Search Results for author:
Found 17 papers, 8 papers with code
Large Language Models Can Be Strong Differentially Private Learners
Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and attempts at straightforwardly applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Learning from Pseudo Lesion: A Self-supervised Framework for COVID-19 Diagnosis
The pairs of normal and pseudo COVID-19 images were then used to train an encoder-decoder architecture based U-Net for image restoration, which does not require any labelled data.
Learning to Extend Program Graphs to Work-in-Progress Code
Source code spends most of its time in a broken or incomplete state during software development.
Efficient and Accurate Gradients for Neural SDEs
This reduces computational cost (giving up to a $1. 87\times$ speedup) and removes the numerical truncation errors associated with gradient penalty.
Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations
We perform scalable approximate inference in continuous-depth Bayesian neural networks.
Neural SDEs as Infinite-Dimensional GANs
Stochastic differential equations (SDEs) are a staple of mathematical modelling of temporal dynamics.
Neural SDEs Made Easy: SDEs are Infinite-Dimensional GANs
Several authors have introduced \emph{Neural Stochastic Differential Equations} (Neural SDEs), often involving complex theory with various limitations.
When Does Preconditioning Help or Hurt Generalization?
While second order optimizers such as natural gradient descent (NGD) often speed up optimization, their effect on generalization has been called into question.
Scalable Gradients for Stochastic Differential Equations
The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations.
Ranked #1 on
Video Prediction
on CMU Mocap-2
Scalable Gradients and Variational Inference for Stochastic Differential Equations
We derive reverse-mode (or adjoint) automatic differentiation for solutions of stochastic differential equations (SDEs), allowing time-efficient and constant-memory computation of pathwise gradients, a continuous-time analogue of the reparameterization trick.
Stochastic Runge-Kutta Accelerates Langevin Monte Carlo and Beyond
In this paper, we establish the convergence rate of sampling algorithms obtained by discretizing smooth It\^o diffusions exhibiting fast Wasserstein-$2$ contraction, based on local deviation properties of the integration scheme.
A Novel Method for the Absolute Pose Problem with Pairwise Constraints
In this paper, we consider pairwise constraints and propose a globally optimal algorithm for solving the absolute pose estimation problem.
Fast and Globally Optimal Rigid Registration of 3D Point Sets by Transformation Decomposition
However, the existing global methods are slow for two main reasons: the computational complexity of BnB is exponential to the problem dimensionality (which is six for 3D rigid registration), and the bound evaluation used in BnB is inefficient.
The idemetric property: when most distances are (almost) the same
We introduce the \emph{idemetric} property, which formalises the idea that most nodes in a graph have similar distances between them, and which turns out to be quite standard amongst small-world network models.
Social and Information Networks Discrete Mathematics
Isolating Sources of Disentanglement in Variational Autoencoders
We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables.
Inference Suboptimality in Variational Autoencoders
Furthermore, we show that the parameters used to increase the expressiveness of the approximation play a role in generalizing inference rather than simply improving the complexity of the approximation.
