Search Results for author:
Found 124 papers, 54 papers with code
Integrating Geodesic Interpolation and Flow Matching for Non-Autoregressive Text Generation in Logit Space
Non-autoregressive language models are emerging as effective alternatives to autoregressive models in the field of natural language processing, facilitating simultaneous token generation.
Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition
We propose a new model for multi-token prediction in transformers, aiming to enhance sampling efficiency without compromising accuracy.
CLEAR: Character Unlearning in Textual and Visual Modalities
To address this, we introduce CLEAR, a new benchmark designed to evaluate MMU methods.
SparseGrad: A Selective Method for Efficient Fine-tuning of MLP Layers
The performance of Transformer models has been enhanced by increasing the number of parameters and the length of the processed text.
Tensor-Train Point Cloud Compression and Efficient Approximate Nearest-Neighbor Search
Nearest-neighbor search in large vector databases is crucial for various machine learning applications.
Sinc Kolmogorov-Arnold Network and Its Applications on Physics-informed Neural Networks
In this paper, we propose to use Sinc interpolation in the context of Kolmogorov-Arnold Networks, neural networks with learnable activation functions, which recently gained attention as alternatives to multilayer perceptron.
Associative memory and dead neurons
We perform a direct analysis of the dynamical system and show how to resolve problems caused by flat directions corresponding to dead neurons: (i) all information about the state vector at a fixed point can be extracted from the energy and Hessian matrix (of Lagrange function), (ii) it is enough to analyze stability in the range of Hessian matrix, (iii) if steady state touching flat region is stable the whole flat region is the basin of attraction.
Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs
It approximates the CE loss for datasets with large-size catalogs, enhancing both time efficiency and memory usage without compromising recommendations quality.
Ranked #1 on
Sequential Recommendation
on Amazon Beauty
Anatomical Positional Embeddings
We train our APE model on 8400 publicly available CT images of abdomen and chest regions.
Spectral Informed Neural Network: An Efficient and Low-Memory PINN
With growing investigations into solving partial differential equations by physics-informed neural networks (PINNs), more accurate and efficient PINNs are required to meet the practical demands of scientific computing.
RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders
Scalability is a major challenge in modern recommender systems.
Inverted Activations: Reducing Memory Footprint in Neural Network Training
A significant challenge in neural network training is the memory footprint associated with activation tensors, particularly in pointwise nonlinearity layers that traditionally save the entire input tensor for the backward pass, leading to substantial memory consumption.
ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equations
We present ConDiff, a novel dataset for scientific machine learning.
Astral: training physics-informed neural networks with error majorants
The primal approach to physics-informed learning is a residual minimization.
Learning from Linear Algebra: A Graph Neural Network Approach to Preconditioner Design for Conjugate Gradient Solvers
In our work, we recall well-established preconditioners from linear algebra and use them as a starting point for training the GNN.
Your Transformer is Secretly Linear
This regularization improves performance metrics on benchmarks like Tiny Stories and SuperGLUE and as well successfully decreases the linearity of the models.
GLiRA: Black-Box Membership Inference Attack via Knowledge Distillation
We observe that the knowledge distillation significantly improves the efficiency of likelihood ratio of membership inference attack, especially in the black-box setting, i. e., when the architecture of the target model is unknown to the attacker.
Certification of Speaker Recognition Models to Additive Perturbations
In this paper, we pioneer applying robustness certification techniques to speaker recognition, originally developed for the image domain.
Quantization of Large Language Models with an Overdetermined Basis
In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation.
OmniFusion Technical Report
We propose an \textit{OmniFusion} model based on a pretrained LLM and adapters for visual modality.
Ranked #88 on
Visual Question Answering
on MM-Vet
Black-Box Approximation and Optimization with Hierarchical Tucker Decomposition
We develop a new method HTBB for the multidimensional black-box approximation and gradient-free optimization, which is based on the low-rank hierarchical Tucker decomposition with the use of the MaxVol indices selection procedure.
Explicit Flow Matching: On The Theory of Flow Matching Algorithms with Applications
In addition, we also investigated simple cases of diffusion generative models by adding a stochastic term and obtained an explicit form of the expression for score.
LoTR: Low Tensor Rank Weight Adaptation
In this paper we generalize and extend an idea of low-rank adaptation (LoRA) of large language models (LLMs) based on Transformer architecture.
Sparse and Transferable Universal Singular Vectors Attack
The research in the field of adversarial attacks and models' vulnerability is one of the fundamental directions in modern machine learning.
Probabilistically Robust Watermarking of Neural Networks
As deep learning (DL) models are widely and effectively used in Machine Learning as a Service (MLaaS) platforms, there is a rapidly growing interest in DL watermarking techniques that can be used to confirm the ownership of a particular model.
Fast gradient-free activation maximization for neurons in spiking neural networks
For a living system, such as a neuron, whose response to a stimulus is unknown and not differentiable, the only way to reveal these features is through a feedback loop that exposes it to a large set of different stimuli.
Run LoRA Run: Faster and Lighter LoRA Implementations
LoRA is a technique that reduces the number of trainable parameters in a neural network by introducing low-rank adapters to linear layers.
Dynamic Collaborative Filtering for Matrix- and Tensor-based Recommender Systems
In production applications of recommender systems, a continuous data flow is employed to update models in real-time.
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models
In this study, we present an investigation into the anisotropy dynamics and intrinsic dimension of embeddings in transformer architectures, focusing on the dichotomy between encoders and decoders.
Memory-efficient particle filter recurrent neural network for object localization
This study proposes a novel memory-efficient recurrent neural network (RNN) architecture specified to solve the object localization problem.
Unsupervised evaluation of GAN sample quality: Introducing the TTJac Score
Evaluation metrics are essential for assessing the performance of generative models in image synthesis.
Robust GAN inversion
Recent advancements in real image editing have been attributed to the exploration of Generative Adversarial Networks (GANs) latent space.
General Lipschitz: Certified Robustness Against Resolvable Semantic Transformations via Transformation-Dependent Randomized Smoothing
Randomized smoothing is the state-of-the-art approach to construct image classifiers that are provably robust against additive adversarial perturbations of bounded magnitude.
Quantization Aware Factorization for Deep Neural Network Compression
Tensor decomposition of convolutional and fully-connected layers is an effective way to reduce parameters and FLOP in neural networks.
Efficient GPT Model Pre-training using Tensor Train Matrix Representation
On the downstream tasks, including language understanding and text summarization, the model performs similarly to the original GPT-2 model.
Spectal Harmonics: Bridging Spectral Embedding and Matrix Completion in Self-Supervised Learning
Self-supervised methods received tremendous attention thanks to their seemingly heuristic approach to learning representations that respect the semantics of the data without any apparent supervision in the form of labels.
Translate your gibberish: black-box adversarial attack on machine translation systems
Neural networks are deployed widely in natural language processing tasks on the industrial scale, and perhaps the most often they are used as compounds of automatic machine translation systems.
Multiparticle Kalman filter for object localization in symmetric environments
Two well-known classes of filtering algorithms to solve the localization problem are Kalman filter-based methods and particle filter-based methods.
Federated Privacy-preserving Collaborative Filtering for On-Device Next App Prediction
Since the data about user experience are distributed among devices, the federated learning setup is used to train the proposed sequential matrix factorization model.
Machine learning methods for prediction of breakthrough curves in reactive porous media
Reactive flows in porous media play an important role in our life and are crucial for many industrial, environmental and biomedical applications.
Mitigating Human and Computer Opinion Fraud via Contrastive Learning
We introduce the novel approach towards fake text reviews detection in collaborative filtering recommender systems.
FMM-Net: neural network architecture based on the Fast Multipole Method
In this paper, we propose a new neural network architecture based on the H2 matrix.
Tensor-based Sequential Learning via Hankel Matrix Representation for Next Item Recommendations
Self-attentive transformer models have recently been shown to solve the next item recommendation task very efficiently.
NAG-GS: Semi-Implicit, Accelerated and Robust Stochastic Optimizer
Classical machine learning models such as deep neural networks are usually trained by using Stochastic Gradient Descent-based (SGD) algorithms.
A case study of spatiotemporal forecasting techniques for weather forecasting
Numerous studies have proposed time series-based models as a viable alternative to numerical forecasts.
T4DT: Tensorizing Time for Learning Temporal 3D Visual Data
We show that low-rank tensor compression is extremely compact to store and query time-varying signed distance functions.
Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AI
The size and complexity of deep neural networks continue to grow exponentially, significantly increasing energy consumption for training and inference by these models.
Tensor networks in machine learning
In this survey we recover the basics of tensor networks and explain the ongoing effort to develop the theory of tensor networks in machine learning.
Tensor-based Collaborative Filtering With Smooth Ratings Scale
Conventional collaborative filtering techniques don't take into consideration the effect of discrepancy in users' rating perception.
Are Quantum Computers Practical Yet? A Case for Feature Selection in Recommender Systems using Tensor Networks
The problem is formulated as a Quadratic Unconstrained Binary Optimization (QUBO) which, due to its NP-hard complexity, is solved using Quantum Annealing on a quantum computer provided by D-Wave.
TTOpt: A Maximum Volume Quantized Tensor Train-based Optimization and its Application to Reinforcement Learning
We present a novel procedure for optimization based on the combination of efficient quantized tensor train representation and a generalized maximum matrix volume principle.
On the Performance of Machine Learning Methods for Breakthrough Curve Prediction
Reactive flows are important part of numerous technical and environmental processes.
Hyperbolic Vision Transformers: Combining Improvements in Metric Learning
Following this line of work, we propose a new hyperbolic-based model for metric learning.
Ranked #1 on
Metric Learning
on CUB-200-2011
Extension of Dynamic Mode Decomposition for dynamic systems with incomplete information based on t-model of optimal prediction
The Dynamic Mode Decomposition has proved to be a very efficient technique to study dynamic data.
Survey on Large Scale Neural Network Training
Modern Deep Neural Networks (DNNs) require significant memory to store weight, activations, and other intermediate tensors during training.
Understanding DDPM Latent Codes Through Optimal Transport
Diffusion models have recently outperformed alternative approaches to model the distribution of natural images, such as GANs.
Smoothed Embeddings for Certified Few-Shot Learning
In this work, we extend randomized smoothing to few-shot learning models that map inputs to normalized embeddings.
Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction
Every modern neural network model has quite a few pointwise nonlinearities in its architecture, and such operation induces additional memory costs which -- as we show -- can be significantly reduced by quantization of the gradients.
Memory-Efficient Backpropagation through Large Linear Layers
Also, we investigate the variance of the gradient estimate induced by the randomized matrix multiplication.
Latent Transformations via NeuralODEs for GAN-based Image Editing
Recent advances in high-fidelity semantic image editing heavily rely on the presumably disentangled latent spaces of the state-of-the-art generative models, such as StyleGAN.
CC-Cert: A Probabilistic Approach to Certify General Robustness of Neural Networks
In safety-critical machine learning applications, it is crucial to defend models against adversarial attacks -- small modifications of the input that change the predictions.
Generation of the NIR spectral Band for Satellite Images with Convolutional Neural Networks
The near-infrared (NIR) spectral range (from 780 to 2500 nm) of the multispectral remote sensing imagery provides vital information for the landcover classification, especially concerning the vegetation assessment.
Object-Based Augmentation Improves Quality of Remote Sensing Semantic Segmentation
To overcome these data restrictions, it is common to consider various approaches including data augmentation techniques.
Dynamic Modeling of User Preferences for Stable Recommendations
In domains where users tend to develop long-term preferences that do not change too frequently, the stability of recommendations is an important factor of the perceived quality of a recommender system.
Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds
In scientific computing and machine learning applications, matrices and more general multidimensional arrays (tensors) can often be approximated with the help of low-rank decompositions.
Meta-Solver for Neural Ordinary Differential Equations
Moreover, we show that the right choice of solver parameterization can significantly affect neural ODEs models in terms of robustness to adversarial attacks.
Disentangled Representations from Non-Disentangled Models
Constructing disentangled representations is known to be a difficult task, especially in the unsupervised scenario.
Functional Space Analysis of Local GAN Convergence
Recent work demonstrated the benefits of studying continuous-time dynamics governing the GAN training.
Two-phase approaches to optimal model-based design of experiments: how many experiments and which ones?
Model-based experimental design is attracting increasing attention in chemical process engineering.
Optimization and Control
Denoising Score Matching with Random Fourier Features
In this work we derive analytical expression for the Denoising Score matching using the Kernel Exponential Family as a model distribution.
Robustness Threats of Differential Privacy
Differential privacy (DP) is a gold-standard concept of measuring and guaranteeing privacy in data analysis.
Adversarial Turing Patterns from Cellular Automata
Furthermore, we propose to use Turing patterns, generated by cellular automata, as universal perturbations, and experimentally show that they significantly degrade the performance of deep learning models.
Tensorized Embedding Layers
The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing.
Learning Elimination Ordering for Tree Decomposition Problem
We propose a Reinforcement Learning-based approach to approximately solve the Tree Decomposition problem.
LETI: Latency Estimation Tool and Investigation of Neural Networks inference on Mobile GPU
In this work, we consider latency approximation on mobile GPU as a data and hardware-specific problem.
Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks
We introduce a simple autoencoder based on hyperbolic geometry for solving standard collaborative filtering problem.
Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network
Most state of the art deep neural networks are overparameterized and exhibit a high computational cost.
Follow the bisector: a simple method for multi-objective optimization
This descent direction is based on the normalized gradients of the individual losses.
Geometry-Inspired Top-k Adversarial Perturbations
The brittleness of deep image classifiers to small adversarial input perturbations has been extensively studied in the last several years.
FREDE: Anytime Graph Embeddings
Low-dimensional representations, or embeddings, of a graph's nodes facilitate several practical data science and data engineering tasks.
Tensorized Transformer for Dynamical Systems Modeling
The identification of nonlinear dynamics from observations is essential for the alignment of the theoretical ideas and experimental data.
Towards Understanding Normalization in Neural ODEs
Normalization is an important and vastly investigated technique in deep learning.
Stochastic gradient algorithms from ODE splitting perspective
We present a different view on stochastic optimization, which goes back to the splitting schemes for approximate solutions of ODE.
Bayesian aggregation improves traditional single image crop classification approaches
We present a comparison between the classical ML approaches and U-Net NN for classifying crops with a single satellite image.
Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs
We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models.
Using Reinforcement Learning in the Algorithmic Trading Problem
A system for trading the fixed volume of a financial instrument is proposed and experimentally tested; this is based on the asynchronous advantage actor-critic method with the use of several neural network architectures.
Deep Representation Learning for Dynamical Systems Modeling
Proper states' representations are the key to the successful dynamics modeling of chaotic systems.
Tensor Completion via Gaussian Process Based Initialization
In this paper, we consider the tensor completion problem representing the solution in the tensor train (TT) format.
Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
Secondly, we propose analyzing the vulnerability of a neural network using active subspace and finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability.
Growing axons: greedy learning of neural networks with application to function approximation
We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions.
Graph Convolutional Policy for Solving Tree Decomposition via Reinforcement Learning Heuristics
We show that the agent builton GCN and trained on a single graph using an Actor-Critic method can efficiently generalize to real-world TD problem instances.
Reduced-Order Modeling of Deep Neural Networks
We introduce a new method for speeding up the inference of deep neural networks.
Predicting dynamical system evolution with residual neural networks
We show how by training neural networks with ResNet-like architecture on the solution samples, models can be developed to predict the ODE system solution further in time.
Empirical study of extreme overfitting points of neural networks
In this paper we propose a method of obtaining points of extreme overfitting - parameters of modern neural networks, at which they demonstrate close to 100 % training accuracy, simultaneously with almost zero accuracy on the test sample.
The Shape of Data: Intrinsic Distance for Data Distributions
The ability to represent and compare machine learning models is crucial in order to quantify subtle model changes, evaluate generative models, and gather insights on neural network architectures.
Universality Theorems for Generative Models
Despite the fact that generative models are extremely successful in practice, the theory underlying this phenomenon is only starting to catch up with practice.
Hyperbolic Image Embeddings
Computer vision tasks such as image classification, image retrieval and few-shot learning are currently dominated by Euclidean and spherical embeddings, so that the final decisions about class belongings or the degree of similarity are made using linear hyperplanes, Euclidean distances, or spherical geodesic distances (cosine similarity).
MUSCO: Multi-Stage Compression of neural networks
The low-rank tensor approximation is very promising for the compression of deep neural networks.
Preconditioning Kaczmarz method by sketching
Therefore, to reduce this complexity, we use random sketching and compare it with the Kaczmarz method without preconditioning.
Numerical Analysis
Tensorized Embedding Layers for Efficient Model Compression
The embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing.
Generalized Tensor Models for Recurrent Neural Networks
Such networks, however, are not very often applied to real life tasks.
Deep Neural Networks Predicting Oil Movement in a Development Unit
We present a novel technique for assessing the dynamics of multiphase fluid flow in the oil reservoir.
PROVEN: Certifying Robustness of Neural Networks with a Probabilistic Approach
With deep neural networks providing state-of-the-art machine learning models for numerous machine learning tasks, quantifying the robustness of these models has become an important area of research.
Adversarial point set registration
We present a novel approach to point set registration which is based on one-shot adversarial learning.
Modelling hidden structure of signals in group data analysis with modified (Lr, 1) and block-term decompositions
This work is devoted to elaboration on the idea to use block term decomposition for group data analysis and to raise the possibility of modelling group activity with (Lr, 1) and Tucker blocks.
Revealing the Unobserved by Linking Collaborative Behavior and Side Knowledge
We propose a tensor-based model that fuses a more granular representation of user preferences with the ability to take additional side information into account.
Comparative study of Discrete Wavelet Transforms and Wavelet Tensor Train decomposition to feature extraction of FTIR data of medicinal plants
Fourier-transform infra-red (FTIR) spectra of samples from 7 plant species were used to explore the influence of preprocessing and feature extraction on efficiency of machine learning algorithms.
HybridSVD: When Collaborative Information is Not Enough
We propose a new hybrid algorithm that allows incorporating both user and item side information within the standard collaborative filtering technique.
Quadrature-based features for kernel approximation
We consider the problem of improving kernel approximation via randomized feature maps.
Geometry Score: A Method For Comparing Generative Adversarial Networks
One of the biggest challenges in the research of generative adversarial networks (GANs) is assessing the quality of generated samples and detecting various levels of mode collapse.
Tensor Train decomposition on TensorFlow (T3F)
Tensor Train decomposition is used across many branches of machine learning.
Mathematical Software Numerical Analysis
Deep Multigrid: learning prolongation and restriction matrices
This paper proposes the method to optimize restriction and prolongation operators in the two-grid method.
Numerical Analysis
Expressive power of recurrent neural networks
In this paper, we prove the expressive power theorem (an exponential lower bound on the width of the equivalent shallow network) for a class of recurrent neural networks -- ones that correspond to the Tensor Train (TT) decomposition.
Neural networks for topology optimization
The main novelty of this work is to state the problem as an image segmentation task.
Art of singular vectors and universal adversarial perturbations
Vulnerability of Deep Neural Networks (DNNs) to adversarial attacks has been attracting a lot of attention in recent studies.
Riemannian Optimization for Skip-Gram Negative Sampling
Skip-Gram Negative Sampling (SGNS) word embedding model, well known by its implementation in "word2vec" software, is usually optimized by stochastic gradient descent.
Efficient Rectangular Maximal-Volume Algorithm for Rating Elicitation in Collaborative Filtering
Cold start problem in Collaborative Filtering can be solved by asking new users to rate a small seed set of representative items or by asking representative users to rate a new item.
Fifty Shades of Ratings: How to Benefit from a Negative Feedback in Top-N Recommendations Tasks
In order to resolve this problem we propose to treat user feedback as a categorical variable and model it with users and items in a ternary way.
Exponential Machines
Modeling interactions between features improves the performance of machine learning solutions in many domains (e. g. recommender systems or sentiment analysis).
Tensor Methods and Recommender Systems
A substantial progress in development of new and efficient tensor factorization techniques has led to an extensive research of their applicability in recommender systems field.
Tensor SimRank for Heterogeneous Information Networks
We propose a generalization of SimRank similarity measure for heterogeneous information networks.
Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition
We propose a simple two-step approach for speeding up convolution layers within large convolutional neural networks based on tensor decomposition and discriminative fine-tuning.
Unifying time evolution and optimization with matrix product states
In particular, our method is compatible with any Hamiltonian for which DMRG can be implemented efficiently and DMRG is obtained as a special case of imaginary time evolution with infinite time step.
Quantum Physics Strongly Correlated Electrons
A projector-splitting integrator for dynamical low-rank approximation
The dynamical low-rank approximation of time-dependent matrices is a low-rank factorization updating technique.
Numerical Analysis 65F30, 65L05, 65L20, 15A23
