Search Results for author:
Found 85 papers, 40 papers with code
Variable-Bitrate Neural Compression via Bayesian Arithmetic Coding
Deep Bayesian latent variable models have enabled new approaches to both model and data compression.
Unity by Diversity: Improved Representation Learning in Multimodal VAEs
Such architectures impose hard constraints on the model.
On the Challenges and Opportunities in Generative AI
The field of deep generative modeling has grown rapidly and consistently over the years.
Towards Fast Stochastic Sampling in Diffusion Generative Models
We propose Splitting Integrators for fast stochastic sampling in pre-trained diffusion models in augmented spaces.
Position Paper: Bayesian Deep Learning in the Age of Large-Scale AI
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
Precipitation Downscaling with Spatiotemporal Video Diffusion
In climate science and meteorology, high-resolution local precipitation (rain and snowfall) predictions are limited by the computational costs of simulation-based methods.
Anytime-Valid Confidence Sequences for Consistent Uncertainty Estimation in Early-Exit Neural Networks
Early-exit neural networks (EENNs) facilitate adaptive inference by producing predictions at multiple stages of the forward pass.
Understanding and Visualizing Droplet Distributions in Simulations of Shallow Clouds
Thorough analysis of local droplet-level interactions is crucial to better understand the microphysical processes in clouds and their effect on the global climate.
Efficient Integrators for Diffusion Generative Models
We propose two complementary frameworks for accelerating sample generation in pre-trained models: Conjugate Integrators and Splitting Integrators.
Understanding Pathologies of Deep Heteroskedastic Regression
Deep, overparameterized regression models are notorious for their tendency to overfit.
ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation
The dataset is global in coverage, spans multiple years at high sampling frequency, and is designed such that resulting emulators are compatible with downstream coupling into operational climate simulators.
Computationally-Efficient Neural Image Compression with Shallow Decoders
We theoretically formalize the intuition behind, and our experimental results establish a new frontier in the trade-off between rate-distortion and decoding complexity for neural image compression.
Deep Anomaly Detection on Tennessee Eastman Process Data
Our extensive study will facilitate choosing appropriate anomaly detection methods in industrial applications.
A Complete Recipe for Diffusion Generative Models
Score-based Generative Models (SGMs) have demonstrated exceptional synthesis outcomes across various tasks.
Ranked #17 on
Image Generation
on CIFAR-10
Zero-Shot Anomaly Detection via Batch Normalization
Anomaly detection (AD) plays a crucial role in many safety-critical application domains.
Ranked #1 on
Unsupervised Anomaly Detection
on AnoShift
Unsupervised Anomaly Detection
zero-shot anomaly detection
+1
Deep Anomaly Detection under Labeling Budget Constraints
Selecting informative data points for expert feedback can significantly improve the performance of anomaly detection (AD) in various contexts, such as medical diagnostics or fraud detection.
Fully Bayesian Autoencoders with Latent Sparse Gaussian Processes
Autoencoders and their variants are among the most widely used models in representation learning and generative modeling.
Probabilistic Querying of Continuous-Time Event Sequences
Continuous-time event sequences, i. e., sequences consisting of continuous time stamps and associated event types ("marks"), are an important type of sequential data with many applications, e. g., in clinical medicine or user behavior modeling.
Predictive Querying for Autoregressive Neural Sequence Models
In reasoning about sequential events it is natural to pose probabilistic queries such as "when will event A occur next" or "what is the probability of A occurring before B", with applications in areas such as user modeling, medicine, and finance.
Lossy Image Compression with Conditional Diffusion Models
This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models.
Raising the Bar in Graph-level Anomaly Detection
Graph-level anomaly detection has become a critical topic in diverse areas, such as financial fraud detection and detecting anomalous activities in social networks.
Diffusion Probabilistic Modeling for Video Generation
Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation.
SC2 Benchmark: Supervised Compression for Split Computing
With the increasing demand for deep learning models on mobile devices, splitting neural network computation between the device and a more powerful edge server has become an attractive solution.
Hybridizing Physical and Data-driven Prediction Methods for Physicochemical Properties
We apply the new approach to predict activity coefficients at infinite dilution and obtain significant improvements compared to the data-driven and physical baselines and established ensemble methods from the machine learning literature.
Latent Outlier Exposure for Anomaly Detection with Contaminated Data
We propose a strategy for training an anomaly detector in the presence of unlabeled anomalies that is compatible with a broad class of models.
An Introduction to Neural Data Compression
Neural compression is the application of neural networks and other machine learning methods to data compression.
Detecting Anomalies within Time Series using Local Neural Transformations
We develop a new method to detect anomalies within time series, which is essential in many application domains, reaching from self-driving cars, finance, and marketing to medical diagnosis and epidemiology.
Analyzing High-Resolution Clouds and Convection using Multi-Channel VAEs
Understanding the details of small-scale convection and storm formation is crucial to accurately represent the larger-scale planetary dynamics.
Lossless Compression with Probabilistic Circuits
To overcome such problems, we establish a new class of tractable lossless compression models that permit efficient encoding and decoding: Probabilistic Circuits (PCs).
Towards Empirical Sandwich Bounds on the Rate-Distortion Function
By contrast, this paper makes the first attempt at an algorithm for sandwiching the R-D function of a general (not necessarily discrete) source requiring only i. i. d.
Structured Stochastic Gradient MCMC: a hybrid VI and MCMC approach
Unfortunately, VI makes strong assumptions on both the factorization and functional form of the posterior.
Supervised Compression for Resource-Constrained Edge Computing Systems
There has been much interest in deploying deep learning algorithms on low-powered devices, including smartphones, drones, and medical sensors.
Insights from Generative Modeling for Neural Video Compression
While recent machine learning research has revealed connections between deep generative models such as VAEs and rate-distortion losses used in learned compression, most of this work has focused on images.
Structured Stochastic Gradient MCMC
Since practitioners face speed versus accuracy tradeoffs in these models, variational inference (VI) is often the preferable option.
Neural Transformation Learning for Deep Anomaly Detection Beyond Images
Data transformations (e. g. rotations, reflections, and cropping) play an important role in self-supervised learning.
SCALE SPACE FLOW WITH AUTOREGRESSIVE PRIORS
There has been a recent surge of interest in neural video compression models that combines data-driven dimensionality reduction with learned entropy coding.
Lower Bounding Rate-Distortion From Samples
The rate-distortion function tells us the minimal number of bits on average to compress a random object within a given distortion tolerance.
Detecting and Adapting to Irregular Distribution Shifts in Bayesian Online Learning
We consider the problem of online learning in the presence of distribution shifts that occur at an unknown rate and of unknown intensity.
Variational Beam Search for Novelty Detection
We consider the problem of online learning in the presence of sudden distribution shifts, which may be hard to detect and can lead to a slow but steady degradation in model performance.
Generative Video Compression as Hierarchical Variational Inference
Recent work by Marino et al. (2020) showed improved performance in sequential density estimation by combining masked autoregressive flows with hierarchical latent variable models.
User-Dependent Neural Sequence Models for Continuous-Time Event Data
Modeling such data can be very challenging, in particular for applications with many different types of events, since it requires a model to predict the event types as well as the time of occurrence.
Scalable Gaussian Process Variational Autoencoders
Conventional variational autoencoders fail in modeling correlations between data points due to their use of factorized priors.
Variational Dynamic Mixtures
Deep probabilistic time series forecasting models have become an integral part of machine learning.
Hierarchical Autoregressive Modeling for Neural Video Compression
Recent work by Marino et al. (2020) showed improved performance in sequential density estimation by combining masked autoregressive flows with hierarchical latent variable models.
Improving Sequential Latent Variable Models with Autoregressive Flows
We propose an approach for improving sequence modeling based on autoregressive normalizing flows.
Generative Modeling for Atmospheric Convection
While cloud-resolving models can explicitly simulate the details of small-scale storm formation and morphology, these details are often ignored by climate models for lack of computational resources.
Improving Inference for Neural Image Compression
We consider the problem of lossy image compression with deep latent variable models.
Variational Bayesian Quantization
Our experimental results demonstrate the importance of taking into account posterior uncertainties, and show that image compression with the proposed algorithm outperforms JPEG over a wide range of bit rates using only a single standard VAE.
Extreme Classification via Adversarial Softmax Approximation
Training a classifier over a large number of classes, known as 'extreme classification', has become a topic of major interest with applications in technology, science, and e-commerce.
The k-tied Normal Distribution: A Compact Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
Variational Bayesian Inference is a popular methodology for approximating posterior distributions over Bayesian neural network weights.
How Good is the Bayes Posterior in Deep Neural Networks Really?
In this work we cast doubt on the current understanding of Bayes posteriors in popular deep neural networks: we demonstrate through careful MCMC sampling that the posterior predictive induced by the Bayes posterior yields systematically worse predictions compared to simpler methods including point estimates obtained from SGD.
Machine Learning in Thermodynamics: Prediction of Activity Coefficients by Matrix Completion
Activity coefficients, which are a measure of the non-ideality of liquid mixtures, are a key property in chemical engineering with relevance to modeling chemical and phase equilibria as well as transport processes.
Hydra: Preserving Ensemble Diversity for Model Distillation
As a result, the diversity of the ensemble predictions, stemming from each member, is lost.
Deep Amortized Variational Inference for Multivariate Time Series Imputation with Latent Gaussian Process Models
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
Dimensionality Reduction
Multivariate Time Series Imputation
+3
Tightening Bounds for Variational Inference by Revisiting Perturbation Theory
In this paper, we revisit perturbation theory as a powerful way of improving the variational approximation.
On the Parameterization of Gaussian Mean Field Posteriors in Bayesian Neural Networks
Variational Bayesian Inference is a popular methodology for approximating posterior distributions in Bayesian neural networks.
Autoregressive Text Generation Beyond Feedback Loops
Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models.
GP-VAE: Deep Probabilistic Time Series Imputation
Multivariate time series with missing values are common in areas such as healthcare and finance, and have grown in number and complexity over the years.
Dimensionality Reduction
Multivariate Time Series Imputation
+2
A Quantum Field Theory of Representation Learning
Continuous symmetries and their breaking play a prominent role in contemporary physics.
Augmenting and Tuning Knowledge Graph Embeddings
Knowledge graph embeddings rank among the most successful methods for link prediction in knowledge graphs, i. e., the task of completing an incomplete collection of relational facts.
Ranked #5 on
Link Prediction
on FB15k
Deep Generative Video Compression
The usage of deep generative models for image compression has led to impressive performance gains over classical codecs while neural video compression is still in its infancy.
Deep Probabilistic Video Compression
We propose a variational inference approach to deep probabilistic video compression.
Probabilistic Knowledge Graph Embeddings
We develop a probabilistic extension of state-of-the-art embedding models for link prediction in relational knowledge graphs.
Iterative Amortized Inference
The failure of these models to reach fully optimized approximate posterior estimates results in an amortization gap.
Quasi-Monte Carlo Variational Inference
We also propose a new algorithm for Monte Carlo objectives, where we operate with a constant learning rate and increase the number of QMC samples per iteration.
Improving Optimization in Models With Continuous Symmetry Breaking
We show that representation learning models for time series possess an approximate continuous symmetry that leads to slow convergence of gradient descent.
Active Mini-Batch Sampling using Repulsive Point Processes
We first show that the phenomenon of variance reduction by diversified sampling generalizes in particular to non-stationary point processes.
Scalable Generalized Dynamic Topic Models
First, we extend the class of tractable priors from Wiener processes to the generic class of Gaussian processes (GPs).
Improving Optimization for Models With Continuous Symmetry Breaking
We show that representation learning models for time series possess an approximate continuous symmetry that leads to slow convergence of gradient descent.
Disentangled Sequential Autoencoder
This architecture gives us partial control over generating content and dynamics by conditioning on either one of these sets of features.
Learning to Infer
Inference models, which replace an optimization-based inference procedure with a learned model, have been fundamental in advancing Bayesian deep learning, the most notable example being variational auto-encoders (VAEs).
Anomaly Detection with Generative Adversarial Networks
Many anomaly detection methods exist that perform well on low-dimensional problems however there is a notable lack of effective methods for high-dimensional spaces, such as images.
Advances in Variational Inference
Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models.
Bayesian Paragraph Vectors
Word2vec (Mikolov et al., 2013) has proven to be successful in natural language processing by capturing the semantic relationships between different words.
Perturbative Black Box Variational Inference
Black box variational inference (BBVI) with reparameterization gradients triggered the exploration of divergence measures other than the Kullback-Leibler (KL) divergence, such as alpha divergences.
Structured Black Box Variational Inference for Latent Time Series Models
Black box variational inference with reparameterization gradients (BBVI) allows us to explore a rich new class of Bayesian non-conjugate latent time series models; however, a naive application of BBVI to such structured variational models would scale quadratically in the number of time steps.
Factorized Variational Autoencoders for Modeling Audience Reactions to Movies
Matrix and tensor factorization methods are often used for finding underlying low-dimensional patterns from noisy data.
Determinantal Point Processes for Mini-Batch Diversification
The DPP relies on a similarity measure between data points and gives low probabilities to mini-batches which contain redundant data, and higher probabilities to mini-batches with more diverse data.
Stochastic Gradient Descent as Approximate Bayesian Inference
Specifically, we show how to adjust the tuning parameters of constant SGD to best match the stationary distribution to a posterior, minimizing the Kullback-Leibler divergence between these two distributions.
Dynamic Word Embeddings
We present a probabilistic language model for time-stamped text data which tracks the semantic evolution of individual words over time.
Exponential Family Embeddings
In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data.
A Variational Analysis of Stochastic Gradient Algorithms
With constant learning rates, it is a stochastic process that, after an initial phase of convergence, generates samples from a stationary distribution.
Sparse Probit Linear Mixed Model
Formulated as models for linear regression, LMMs have been restricted to continuous phenotypes.
Variational Tempering
Lastly, we develop local variational tempering, which assigns a latent temperature to each data point; this allows for dynamic annealing that varies across data.
Smoothed Gradients for Stochastic Variational Inference
It uses stochastic optimization to fit a variational distribution, following easy-to-compute noisy natural gradients.
