Search Results for author:
Found 28 papers, 18 papers with code
Understanding the Diffusion Objective as a Weighted Integral of ELBOs
Diffusion models in the literature are optimized with various objectives that are special cases of a weighted loss, where the weighting function specifies the weight per noise level.
On Distillation of Guided Diffusion Models
For standard diffusion models trained on the pixel-space, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps on ImageNet 64x64 and CIFAR-10, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from.
Imagen Video: High Definition Video Generation with Diffusion Models
We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models.
Ranked #1 on
Video Generation
on LAION-400M
Variational Diffusion Models
In addition, we show that the continuous-time VLB is invariant to the noise schedule, except for the signal-to-noise ratio at its endpoints.
Ranked #1 on
Image Generation
on CIFAR-10
(bits/dimension metric)
How to Train Your Energy-Based Models
Energy-Based Models (EBMs), also known as non-normalized probabilistic models, specify probability density or mass functions up to an unknown normalizing constant.
Learning Energy-Based Models by Diffusion Recovery Likelihood
Inspired by recent progress on diffusion probabilistic models, we present a diffusion recovery likelihood method to tractably learn and sample from a sequence of EBMs trained on increasingly noisy versions of a dataset.
Ranked #13 on
Image Generation
on CelebA 64x64
Score-Based Generative Modeling through Stochastic Differential Equations
Combined with multiple architectural improvements, we achieve record-breaking performance for unconditional image generation on CIFAR-10 with an Inception score of 9. 89 and FID of 2. 20, a competitive likelihood of 2. 99 bits/dim, and demonstrate high fidelity generation of 1024 x 1024 images for the first time from a score-based generative model.
Ranked #11 on
Image Generation
on CIFAR-10
Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesis
We describe a sequence-to-sequence neural network which directly generates speech waveforms from text inputs.
On Linear Identifiability of Learned Representations
Identifiability is a desirable property of a statistical model: it implies that the true model parameters may be estimated to any desired precision, given sufficient computational resources and data.
ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA
We consider the identifiability theory of probabilistic models and establish sufficient conditions under which the representations learned by a very broad family of conditional energy-based models are unique in function space, up to a simple transformation.
Flow Contrastive Estimation of Energy-Based Models
(2) The update of the flow model approximately minimizes the Jensen-Shannon divergence between the flow model and the data distribution.
Variational Autoencoders and Nonlinear ICA: A Unifying Framework
We address this issue by showing that for a broad family of deep latent-variable models, identification of the true joint distribution over observed and latent variables is actually possible up to very simple transformations, thus achieving a principled and powerful form of disentanglement.
An Introduction to Variational Autoencoders
Variational autoencoders provide a principled framework for learning deep latent-variable models and corresponding inference models.
Glow: Generative Flow with Invertible 1x1 Convolutions
Flow-based generative models (Dinh et al., 2014) are conceptually attractive due to tractability of the exact log-likelihood, tractability of exact latent-variable inference, and parallelizability of both training and synthesis.
Ranked #9 on
Image Generation
on CelebA-HQ 256x256
Learning Sparse Neural Networks through L_0 Regularization
We further propose the \emph{hard concrete} distribution for the gates, which is obtained by ``stretching'' a binary concrete distribution and then transforming its samples with a hard-sigmoid.
Learning Sparse Neural Networks through $L_0$ Regularization
We further propose the \emph{hard concrete} distribution for the gates, which is obtained by "stretching" a binary concrete distribution and then transforming its samples with a hard-sigmoid.
PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modifications
1) We use a discretized logistic mixture likelihood on the pixels, rather than a 256-way softmax, which we find to speed up training.
Variational Lossy Autoencoder
Representation learning seeks to expose certain aspects of observed data in a learned representation that's amenable to downstream tasks like classification.
Improving Variational Inference with Inverse Autoregressive Flow
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables.
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks
We present weight normalization: a reparameterization of the weight vectors in a neural network that decouples the length of those weight vectors from their direction.
Variational Dropout and the Local Reparameterization Trick
Our method allows inference of more flexibly parameterized posteriors; specifically, we propose variational dropout, a generalization of Gaussian dropout where the dropout rates are learned, often leading to better models.
Note on Equivalence Between Recurrent Neural Network Time Series Models and Variational Bayesian Models
We observe that the standard log likelihood training objective for a Recurrent Neural Network (RNN) model of time series data is equivalent to a variational Bayesian training objective, given the proper choice of generative and inference models.
Adam: A Method for Stochastic Optimization
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary random variables.
Semi-Supervised Learning with Deep Generative Models
The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis.
Ranked #53 on
Image Classification
on SVHN
Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets
Hierarchical Bayesian networks and neural networks with stochastic hidden units are commonly perceived as two separate types of models.
Auto-Encoding Variational Bayes
First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods.
Ranked #11 on
Image Clustering
on Tiny-ImageNet
Fast Gradient-Based Inference with Continuous Latent Variable Models in Auxiliary Form
We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables.
