Search Results for author:
Found 23 papers, 17 papers with code
Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections
This manifold lens provides both clarity as to why some DGMs (e. g. diffusion models and some generative adversarial networks) empirically surpass others (e. g. likelihood-based models such as variational autoencoders, normalizing flows, or energy-based models) at sample generation, and guidance for devising more performant DGMs.
A Geometric Explanation of the Likelihood OOD Detection Paradox
We also show that this scenario can be identified through local intrinsic dimension (LID) estimation, and propose a method for OOD detection which pairs the likelihoods and LID estimates obtained from a pre-trained DGM.
Data-Efficient Multimodal Fusion on a Single GPU
The goal of multimodal alignment is to learn a single latent space that is shared between multimodal inputs.
Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models
Comparing to 17 modern metrics for evaluating the overall performance, fidelity, diversity, rarity, and memorization of generative models, we find that the state-of-the-art perceptual realism of diffusion models as judged by humans is not reflected in commonly reported metrics such as FID.
TR0N: Translator Networks for 0-Shot Plug-and-Play Conditional Generation
We propose TR0N, a highly general framework to turn pre-trained unconditional generative models, such as GANs and VAEs, into conditional models.
Ranked #31 on
Text-to-Image Generation
on MS COCO
Denoising Deep Generative Models
Likelihood-based deep generative models have recently been shown to exhibit pathological behaviour under the manifold hypothesis as a consequence of using high-dimensional densities to model data with low-dimensional structure.
Relating Regularization and Generalization through the Intrinsic Dimension of Activations
Given a pair of models with similar training set performance, it is natural to assume that the model that possesses simpler internal representations would exhibit better generalization.
CaloMan: Fast generation of calorimeter showers with density estimation on learned manifolds
Precision measurements and new physics searches at the Large Hadron Collider require efficient simulations of particle propagation and interactions within the detectors.
Verifying the Union of Manifolds Hypothesis for Image Data
Assuming that data lies on a single manifold implies intrinsic dimension is identical across the entire data space, and does not allow for subregions of this space to have a different number of factors of variation.
Neural Implicit Manifold Learning for Topology-Aware Density Estimation
We then learn the probability density within $\mathcal{M}$ with a constrained energy-based model, which employs a constrained variant of Langevin dynamics to train and sample from the learned manifold.
On the Normalizing Constant of the Continuous Categorical Distribution
This family enjoys remarkable mathematical simplicity; its density function resembles that of the Dirichlet distribution, but with a normalizing constant that can be written in closed form using elementary functions only.
Diagnosing and Fixing Manifold Overfitting in Deep Generative Models
We propose a class of two-step procedures consisting of a dimensionality reduction step followed by maximum-likelihood density estimation, and prove that they recover the data-generating distribution in the nonparametric regime, thus avoiding manifold overfitting.
Bayesian Nonparametrics for Offline Skill Discovery
Skills or low-level policies in reinforcement learning are temporally extended actions that can speed up learning and enable complex behaviours.
Entropic Issues in Likelihood-Based OOD Detection
This analysis provides further explanation for the success of OOD detection methods based on likelihood ratios, as the problematic entropy term cancels out in expectation.
Rectangular Flows for Manifold Learning
Normalizing flows are invertible neural networks with tractable change-of-volume terms, which allow optimization of their parameters to be efficiently performed via maximum likelihood.
C-Learning: Horizon-Aware Cumulative Accessibility Estimation
In order to address these limitations, we introduce the concept of cumulative accessibility functions, which measure the reachability of a goal from a given state within a specified horizon.
Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning
Modern deep learning is primarily an experimental science, in which empirical advances occasionally come at the expense of probabilistic rigor.
The continuous categorical: a novel simplex-valued exponential family
Simplex-valued data appear throughout statistics and machine learning, for example in the context of transfer learning and compression of deep networks.
Invertible Gaussian Reparameterization: Revisiting the Gumbel-Softmax
The Gumbel-Softmax is a continuous distribution over the simplex that is often used as a relaxation of discrete distributions.
The continuous Bernoulli: fixing a pervasive error in variational autoencoders
Variational autoencoders (VAE) have quickly become a central tool in machine learning, applicable to a broad range of data types and latent variable models.
Deep Random Splines for Point Process Intensity Estimation
Gaussian processes are the leading class of distributions on random functions, but they suffer from well known issues including difficulty scaling and inflexibility with respect to certain shape constraints (such as nonnegativity).
Deep Random Splines for Point Process Intensity Estimation of Neural Population Data
Gaussian processes are the leading class of distributions on random functions, but they suffer from well known issues including difficulty scaling and inflexibility with respect to certain shape constraints (such as nonnegativity).
Maximum Entropy Flow Networks
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge.
