Search Results for author:
Found 25 papers, 15 papers with code
Latent Discretization for Continuous-time Sequence Compression
We empirically verify our approach on multiple domains involving compression of video and motion capture sequences, showing that our approaches can automatically achieve reductions in bit rates by learning how to discretize.
Flow Matching for Generative Modeling
We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale.
Neural Conservation Laws: A Divergence-Free Perspective
We investigate the parameterization of deep neural networks that by design satisfy the continuity equation, a fundamental conservation law.
Latent State Marginalization as a Low-cost Approach for Improving Exploration
While the maximum entropy (MaxEnt) reinforcement learning (RL) framework -- often touted for its exploration and robustness capabilities -- is usually motivated from a probabilistic perspective, the use of deep probabilistic models has not gained much traction in practice due to their inherent complexity.
Unifying Generative Models with GFlowNets
This provides a means for unifying training and inference algorithms, and provides a route to construct an agglomeration of generative models.
Theseus: A Library for Differentiable Nonlinear Optimization
We present Theseus, an efficient application-agnostic open source library for differentiable nonlinear least squares (DNLS) optimization built on PyTorch, providing a common framework for end-to-end structured learning in robotics and vision.
Matching Normalizing Flows and Probability Paths on Manifolds
Continuous Normalizing Flows (CNFs) are a class of generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE).
Semi-Discrete Normalizing Flows through Differentiable Tessellation
Mapping between discrete and continuous distributions is a difficult task and many have had to resort to heuristical approaches.
Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations
We perform scalable approximate inference in continuous-depth Bayesian neural networks.
"Hey, that's not an ODE'": Faster ODE Adjoints with 12 Lines of Code
Neural differential equations may be trained by backpropagating gradients via the adjoint method, which is another differential equation typically solved using an adaptive-step-size numerical differential equation solver.
Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
Flow-based models are powerful tools for designing probabilistic models with tractable density.
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering
Standard first-order stochastic optimization algorithms base their updates solely on the average mini-batch gradient, and it has been shown that tracking additional quantities such as the curvature can help de-sensitize common hyperparameters.
Neural Spatio-Temporal Point Processes
We propose a new class of parameterizations for spatio-temporal point processes which leverage Neural ODEs as a computational method and enable flexible, high-fidelity models of discrete events that are localized in continuous time and space.
Learning Neural Event Functions for Ordinary Differential Equations
The existing Neural ODE formulation relies on an explicit knowledge of the termination time.
"Hey, that's not an ODE": Faster ODE Adjoints via Seminorms
Neural differential equations may be trained by backpropagating gradients via the adjoint method, which is another differential equation typically solved using an adaptive-step-size numerical differential equation solver.
SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models
Standard variational lower bounds used to train latent variable models produce biased estimates of most quantities of interest.
Scalable Gradients for Stochastic Differential Equations
The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.
Ranked #1 on
Video Prediction
on CMU Mocap-2
Neural Networks with Cheap Differential Operators
Gradients of neural networks can be computed efficiently for any architecture, but some applications require differential operators with higher time complexity.
Scalable Gradients and Variational Inference for Stochastic Differential Equations
We derive reverse-mode (or adjoint) automatic differentiation for solutions of stochastic differential equations (SDEs), allowing time-efficient and constant-memory computation of pathwise gradients, a continuous-time analogue of the reparameterization trick.
Latent ODEs for Irregularly-Sampled Time Series
Time series with non-uniform intervals occur in many applications, and are difficult to model using standard recurrent neural networks (RNNs).
Ranked #1 on
Multivariate Time Series Imputation
on PhysioNet Challenge 2012
(mse (10^-3) metric)
Multivariate Time Series Forecasting
Multivariate Time Series Imputation
+1
Residual Flows for Invertible Generative Modeling
Flow-based generative models parameterize probability distributions through an invertible transformation and can be trained by maximum likelihood.
Ranked #2 on
Image Generation
on MNIST
Invertible Residual Networks
We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation.
Ranked #5 on
Image Generation
on MNIST
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
The result is a continuous-time invertible generative model with unbiased density estimation and one-pass sampling, while allowing unrestricted neural network architectures.
Ranked #1 on
Density Estimation
on CIFAR-10
(NLL metric)
Neural Ordinary Differential Equations
Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network.
Ranked #2 on
Multivariate Time Series Forecasting
on MuJoCo
Multivariate Time Series Forecasting
Multivariate Time Series Imputation
Isolating Sources of Disentanglement in Variational Autoencoders
We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables.
