Search Results for author: Dmitry Vetrov

Found 61 papers, 33 papers with code

Involutive MCMC: One Way to Derive Them All

no code implementations ICML 2020 Kirill Neklyudov, Max Welling, Evgenii Egorov, Dmitry Vetrov

Markov Chain Monte Carlo (MCMC) is a computational approach to fundamental problems such as inference, integration, optimization, and simulation.

Machine Learning Methods for Spectral Efficiency Prediction in Massive MIMO Systems

no code implementations29 Dec 2021 Evgeny Bobrov, Sergey Troshin, Nadezhda Chirkova, Ekaterina Lobacheva, Sviatoslav Panchenko, Dmitry Vetrov, Dmitry Kropotov

Channel decoding, channel detection, channel assessment, and resource management for wireless multiple-input multiple-output (MIMO) systems are all examples of problems where machine learning (ML) can be successfully applied.

Variational Autoencoders for Precoding Matrices with High Spectral Efficiency

no code implementations23 Nov 2021 Evgeny Bobrov, Alexander Markov, Yulia Novikova, Dmitry Vetrov

The construction of precoding matrices and their distribution for the SE objective function using VAE and CVAE methods is described in the literature for the first time.

Automating Control of Overestimation Bias for Continuous Reinforcement Learning

no code implementations26 Oct 2021 Arsenii Kuznetsov, Alexander Grishin, Artem Tsypin, Arsenii Ashukha, Dmitry Vetrov

Bias correction techniques are used by most of the high-performing methods for off-policy reinforcement learning.

Continuous Control

Quantization of Generative Adversarial Networks for Efficient Inference: a Methodological Study

no code implementations31 Aug 2021 Pavel Andreev, Alexander Fritzler, Dmitry Vetrov

While quantization is well established for discriminative models, the performance of modern quantization techniques in application to GANs remains unclear.

Neural Network Compression Quantization

Mean Embeddings with Test-Time Data Augmentation for Ensembling of Representations

no code implementations15 Jun 2021 Arsenii Ashukha, Andrei Atanov, Dmitry Vetrov

Averaging predictions over a set of models -- an ensemble -- is widely used to improve predictive performance and uncertainty estimation of deep learning models.

Data Augmentation Image Retrieval +1

Towards Practical Credit Assignment for Deep Reinforcement Learning

no code implementations8 Jun 2021 Vyacheslav Alipov, Riley Simmons-Edler, Nikita Putintsev, Pavel Kalinin, Dmitry Vetrov

Based on this exploration, we present a new algorithm, Credit-Constrained Advantage Actor-Critic (C2A2C), which ignores policy updates for actions which don't affect future outcomes based on credit in hindsight, while updating the policy as normal for those that do.

Atari Games

On Power Laws in Deep Ensembles

1 code implementation NeurIPS 2020 Ekaterina Lobacheva, Nadezhda Chirkova, Maxim Kodryan, Dmitry Vetrov

Ensembles of deep neural networks are known to achieve state-of-the-art performance in uncertainty estimation and lead to accuracy improvement.

Involutive MCMC: a Unifying Framework

no code implementations30 Jun 2020 Kirill Neklyudov, Max Welling, Evgenii Egorov, Dmitry Vetrov

Markov Chain Monte Carlo (MCMC) is a computational approach to fundamental problems such as inference, integration, optimization, and simulation.

MARS: Masked Automatic Ranks Selection in Tensor Decompositions

no code implementations18 Jun 2020 Maxim Kodryan, Dmitry Kropotov, Dmitry Vetrov

Tensor decomposition methods are known to be efficient for compressing and accelerating neural networks.

Tensor Decomposition

Deep Ensembles on a Fixed Memory Budget: One Wide Network or Several Thinner Ones?

no code implementations14 May 2020 Nadezhda Chirkova, Ekaterina Lobacheva, Dmitry Vetrov

In this work, we consider a fixed memory budget setting, and investigate, what is more effective: to train a single wide network, or to perform a memory split -- to train an ensemble of several thinner networks, with the same total number of parameters?

Deterministic Decoding for Discrete Data in Variational Autoencoders

1 code implementation4 Mar 2020 Daniil Polykovskiy, Dmitry Vetrov

Variational autoencoders are prominent generative models for modeling discrete data.

Stochasticity in Neural ODEs: An Empirical Study

1 code implementation ICLR Workshop DeepDiffEq 2019 Viktor Oganesyan, Alexandra Volokhova, Dmitry Vetrov

Stochastic regularization of neural networks (e. g. dropout) is a wide-spread technique in deep learning that allows for better generalization.

Data Augmentation Image Classification

Greedy Policy Search: A Simple Baseline for Learnable Test-Time Augmentation

1 code implementation21 Feb 2020 Dmitry Molchanov, Alexander Lyzhov, Yuliya Molchanova, Arsenii Ashukha, Dmitry Vetrov

Test-time data augmentation$-$averaging the predictions of a machine learning model across multiple augmented samples of data$-$is a widely used technique that improves the predictive performance.

Data Augmentation Image Classification

Towards understanding the true loss surface of deep neural networks using random matrix theory and iterative spectral methods

no code implementations ICLR 2020 Diego Granziol, Timur Garipov, Dmitry Vetrov, Stefan Zohren, Stephen Roberts, Andrew Gordon Wilson

This approach is an order of magnitude faster than state-of-the-art methods for spectral visualization, and can be generically used to investigate the spectral properties of matrices in deep learning.

Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution

1 code implementation22 Nov 2019 Artyom Gadetsky, Kirill Struminsky, Christopher Robinson, Novi Quadrianto, Dmitry Vetrov

Learning models with discrete latent variables using stochastic gradient descent remains a challenge due to the high variance of gradient estimates.

Structured Sparsification of Gated Recurrent Neural Networks

no code implementations13 Nov 2019 Ekaterina Lobacheva, Nadezhda Chirkova, Alexander Markovich, Dmitry Vetrov

Recently, a lot of techniques were developed to sparsify the weights of neural networks and to remove networks' structure units, e. g. neurons.

Language Modelling Text Classification

A Prior of a Googol Gaussians: a Tensor Ring Induced Prior for Generative Models

1 code implementation NeurIPS 2019 Maksim Kuznetsov, Daniil Polykovskiy, Dmitry Vetrov, Alexander Zhebrak

Previous works show that the richer family of prior distributions may help to avoid the mode collapse problem in GANs and to improve the evidence lower bound in VAEs.

Subspace Inference for Bayesian Deep Learning

1 code implementation17 Jul 2019 Pavel Izmailov, Wesley J. Maddox, Polina Kirichenko, Timur Garipov, Dmitry Vetrov, Andrew Gordon Wilson

Bayesian inference was once a gold standard for learning with neural networks, providing accurate full predictive distributions and well calibrated uncertainty.

Bayesian Inference Image Classification +1

The Implicit Metropolis-Hastings Algorithm

1 code implementation NeurIPS 2019 Kirill Neklyudov, Evgenii Egorov, Dmitry Vetrov

For any implicit probabilistic model and a target distribution represented by a set of samples, implicit Metropolis-Hastings operates by learning a discriminator to estimate the density-ratio and then generating a chain of samples.

Image Generation

Importance Weighted Hierarchical Variational Inference

1 code implementation NeurIPS 2019 Artem Sobolev, Dmitry Vetrov

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution.

Variational Inference

User-Controllable Multi-Texture Synthesis with Generative Adversarial Networks

no code implementations9 Apr 2019 Aibek Alanov, Max Kochurov, Denis Volkhonskiy, Daniil Yashkov, Evgeny Burnaev, Dmitry Vetrov

We propose a novel multi-texture synthesis model based on generative adversarial networks (GANs) with a user-controllable mechanism.

Texture Synthesis

A Simple Baseline for Bayesian Uncertainty in Deep Learning

7 code implementations NeurIPS 2019 Wesley Maddox, Timur Garipov, Pavel Izmailov, Dmitry Vetrov, Andrew Gordon Wilson

We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning.

Bayesian Inference Transfer Learning

Bayesian Sparsification of Gated Recurrent Neural Networks

1 code implementation NIPS Workshop CDNNRIA 2018 Ekaterina Lobacheva, Nadezhda Chirkova, Dmitry Vetrov

Bayesian methods have been successfully applied to sparsify weights of neural networks and to remove structure units from the networks, e. g. neurons.

ReSet: Learning Recurrent Dynamic Routing in ResNet-like Neural Networks

no code implementations11 Nov 2018 Iurii Kemaev, Daniil Polykovskiy, Dmitry Vetrov

Neural Network is a powerful Machine Learning tool that shows outstanding performance in Computer Vision, Natural Language Processing, and Artificial Intelligence.

Image Classification

Variational Dropout via Empirical Bayes

1 code implementation1 Nov 2018 Valery Kharitonov, Dmitry Molchanov, Dmitry Vetrov

We study the Automatic Relevance Determination procedure applied to deep neural networks.

Bayesian Compression for Natural Language Processing

3 code implementations EMNLP 2018 Nadezhda Chirkova, Ekaterina Lobacheva, Dmitry Vetrov

In natural language processing, a lot of the tasks are successfully solved with recurrent neural networks, but such models have a huge number of parameters.

Metropolis-Hastings view on variational inference and adversarial training

no code implementations ICLR 2019 Kirill Neklyudov, Evgenii Egorov, Pavel Shvechikov, Dmitry Vetrov

From this point of view, the problem of constructing a sampler can be reduced to the question - how to choose a proposal for the MH algorithm?

Bayesian Inference Variational Inference

The Deep Weight Prior

2 code implementations ICLR 2019 Andrei Atanov, Arsenii Ashukha, Kirill Struminsky, Dmitry Vetrov, Max Welling

Bayesian inference is known to provide a general framework for incorporating prior knowledge or specific properties into machine learning models via carefully choosing a prior distribution.

Bayesian Inference Variational Inference

Pairwise Augmented GANs with Adversarial Reconstruction Loss

no code implementations ICLR 2019 Aibek Alanov, Max Kochurov, Daniil Yashkov, Dmitry Vetrov

We experimentally demonstrate that our model generates samples and reconstructions of quality competitive with state-of-the-art on datasets MNIST, CIFAR10, CelebA and achieves good quantitative results on CIFAR10.

Doubly Semi-Implicit Variational Inference

no code implementations5 Oct 2018 Dmitry Molchanov, Valery Kharitonov, Artem Sobolev, Dmitry Vetrov

Unlike discriminator-based and kernel-based approaches to implicit variational inference, DSIVI optimizes a proper lower bound on ELBO that is asymptotically exact.

Variational Inference

Conditional Generators of Words Definitions

1 code implementation ACL 2018 Artyom Gadetsky, Ilya Yakubovskiy, Dmitry Vetrov

We explore recently introduced definition modeling technique that provided the tool for evaluation of different distributed vector representations of words through modeling dictionary definitions of words.

Variational Autoencoder with Arbitrary Conditioning

3 code implementations ICLR 2019 Oleg Ivanov, Michael Figurnov, Dmitry Vetrov

We propose a single neural probabilistic model based on variational autoencoder that can be conditioned on an arbitrary subset of observed features and then sample the remaining features in "one shot".

Image Inpainting Imputation

Averaging Weights Leads to Wider Optima and Better Generalization

14 code implementations14 Mar 2018 Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, Andrew Gordon Wilson

Deep neural networks are typically trained by optimizing a loss function with an SGD variant, in conjunction with a decaying learning rate, until convergence.

Image Classification Stochastic Optimization

Variance Networks: When Expectation Does Not Meet Your Expectations

2 code implementations ICLR 2019 Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov

Ordinary stochastic neural networks mostly rely on the expected values of their weights to make predictions, whereas the induced noise is mostly used to capture the uncertainty, prevent overfitting and slightly boost the performance through test-time averaging.

Efficient Exploration

Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs

9 code implementations NeurIPS 2018 Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, Andrew Gordon Wilson

The loss functions of deep neural networks are complex and their geometric properties are not well understood.

Uncertainty Estimation via Stochastic Batch Normalization

no code implementations13 Feb 2018 Andrei Atanov, Arsenii Ashukha, Dmitry Molchanov, Kirill Neklyudov, Dmitry Vetrov

In this work, we investigate Batch Normalization technique and propose its probabilistic interpretation.

Probabilistic Adaptive Computation Time

no code implementations1 Dec 2017 Michael Figurnov, Artem Sobolev, Dmitry Vetrov

We present a probabilistic model with discrete latent variables that control the computation time in deep learning models such as ResNets and LSTMs.

Bayesian Sparsification of Recurrent Neural Networks

2 code implementations31 Jul 2017 Ekaterina Lobacheva, Nadezhda Chirkova, Dmitry Vetrov

Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights.

Language Modelling Sentiment Analysis

Structured Bayesian Pruning via Log-Normal Multiplicative Noise

5 code implementations NeurIPS 2017 Kirill Neklyudov, Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov

In the paper, we propose a new Bayesian model that takes into account the computational structure of neural networks and provides structured sparsity, e. g. removes neurons and/or convolutional channels in CNNs.

Variational Dropout Sparsifies Deep Neural Networks

13 code implementations ICML 2017 Dmitry Molchanov, Arsenii Ashukha, Dmitry Vetrov

We explore a recently proposed Variational Dropout technique that provided an elegant Bayesian interpretation to Gaussian Dropout.

Sparse Learning

Spatially Adaptive Computation Time for Residual Networks

1 code implementation CVPR 2017 Michael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, Ruslan Salakhutdinov

This paper proposes a deep learning architecture based on Residual Network that dynamically adjusts the number of executed layers for the regions of the image.

General Classification Image Classification +2

Robust Variational Inference

no code implementations28 Nov 2016 Michael Figurnov, Kirill Struminsky, Dmitry Vetrov

Variational inference is a powerful tool for approximate inference.

Variational Inference

Ultimate tensorization: compressing convolutional and FC layers alike

2 code implementations10 Nov 2016 Timur Garipov, Dmitry Podoprikhin, Alexander Novikov, Dmitry Vetrov

Convolutional neural networks excel in image recognition tasks, but this comes at the cost of high computational and memory complexity.

PerforatedCNNs: Acceleration through Elimination of Redundant Convolutions

2 code implementations NeurIPS 2016 Michael Figurnov, Aijan Ibraimova, Dmitry Vetrov, Pushmeet Kohli

We propose a novel approach to reduce the computational cost of evaluation of convolutional neural networks, a factor that has hindered their deployment in low-power devices such as mobile phones.

Breaking Sticks and Ambiguities with Adaptive Skip-gram

3 code implementations25 Feb 2015 Sergey Bartunov, Dmitry Kondrashkin, Anton Osokin, Dmitry Vetrov

Recently proposed Skip-gram model is a powerful method for learning high-dimensional word representations that capture rich semantic relationships between words.

Word Sense Induction

Submodular relaxation for inference in Markov random fields

1 code implementation15 Jan 2015 Anton Osokin, Dmitry Vetrov

In this paper we address the problem of finding the most probable state of a discrete Markov random field (MRF), also known as the MRF energy minimization problem.

Multi-utility Learning: Structured-output Learning with Multiple Annotation-specific Loss Functions

no code implementations23 Jun 2014 Roman Shapovalov, Dmitry Vetrov, Anton Osokin, Pushmeet Kohli

Structured-output learning is a challenging problem; particularly so because of the difficulty in obtaining large datasets of fully labelled instances for training.

Semantic Segmentation Structured Prediction

Spatial Inference Machines

no code implementations CVPR 2013 Roman Shapovalov, Dmitry Vetrov, Pushmeet Kohli

Experimental results show that the spatial dependencies learned by our method significantly improve the accuracy of segmentation.

Semantic Segmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.