Search Results for author:
Found 30 papers, 7 papers with code
The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability
While the fundamental ideas behind these models come from non-equilibrium physics, variational inference and stochastic calculus, in this paper we show that many aspects of these models can be understood using the tools of equilibrium statistical mechanics.
Stationarity without mean reversion: Improper Gaussian process regression and improper kernels
Stationary covariance functions are favorite in machine learning applications.
In search of dispersed memories: Generative diffusion models are associative memory networks
In this work we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks.
Deterministic training of generative autoencoders using invertible layers
In this work, we provide a deterministic alternative to the stochastic variational training of generative autoencoders.
Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling
We also introduce gated structured layers, which allow bypassing the parts of the models that fail to capture the statistics of the data.
Knowledge is reward: Learning optimal exploration by predictive reward cashing
There is a strong link between the general concept of intelligence and the ability to collect and use information.
Gradient-adjusted Incremental Target Propagation Provides Effective Credit Assignment in Deep Neural Networks
Many of the recent advances in the field of artificial intelligence have been fueled by the highly successful backpropagation of error (BP) algorithm, which efficiently solves the credit assignment problem in artificial neural networks.
Automatic variational inference with cascading flows
We evaluate the performance of the new variational programs in a series of structured inference problems.
Hyperrealistic neural decoding: Reconstruction of face stimuli from fMRI measurements via the GAN latent space
We introduce a new framework for hyperrealistic reconstruction of perceived naturalistic stimuli from brain recordings.
The 3TConv: An Intrinsic Approach to Explainable 3D CNNs
In a 3TConv the 3D convolutional filter is obtained by learning a 2D filter and a set of temporal transformation parameters, resulting in a sparse filter requiring less parameters.
Explainable 3D Convolutional Neural Networks by Learning Temporal Transformations
Finally, we implicitly demonstrate that, in popular ConvNets, the 2DConv can be replaced with a 3TConv and that the weights can be transferred to yield pretrained 3TConvs.
GAIT-prop: A biologically plausible learning rule derived from backpropagation of error
An alternative called target propagation proposes to solve this implausibility by using a top-down model of neural activity to convert an error at the output of a neural network into layer-wise and plausible 'targets' for every unit.
Overcoming the Weight Transport Problem via Spike-Timing-Dependent Weight Inference
We propose a solution to the weight transport problem, which questions the biological plausibility of the backpropagation algorithm.
Automatic structured variational inference
However, the performance of the variational approach depends on the choice of an appropriate variational family.
The Indian Chefs Process
This paper introduces the Indian Chefs Process (ICP), a Bayesian nonparametric prior on the joint space of infinite directed acyclic graphs (DAGs) and orders that generalizes Indian Buffet Processes.
Background Hardly Matters: Understanding Personality Attribution in Deep Residual Networks
It is important that we understand the driving factors behind the predictions, in humans and in deep neural networks.
Temporal Factorization of 3D Convolutional Kernels
3D convolutional neural networks are difficult to train because they are parameter-expensive and data-hungry.
Bayesian nonparametric discontinuity design
Quasi-experimental research designs, such as regression discontinuity and interrupted time series, allow for causal inference in the absence of a randomized controlled trial, at the cost of additional assumptions.
k-GANs: Ensemble of Generative Models with Semi-Discrete Optimal Transport
A possible way of dealing with this problem is to use an ensemble of GANs, where (ideally) each network models a single mode.
Perturbative estimation of stochastic gradients
Furthermore, we introduce a family of variance reduction techniques that can be applied to other gradient estimators.
Wasserstein variational gradient descent: From semi-discrete optimal transport to ensemble variational inference
The solution of the resulting optimal transport problem provides both a particle approximation and a set of optimal transportation densities that map each particle to a segment of the posterior distribution.
Wasserstein Variational Inference
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory.
Forward Amortized Inference for Likelihood-Free Variational Marginalization
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss.
The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables
In the Bayesian filtering example, we show that the method can be used to filter complex nonlinear and non-Gaussian signals defined on manifolds.
GP CaKe: Effective brain connectivity with causal kernels
Here we propose to model this causal interaction using integro-differential equations and causal kernels that allow for a rich analysis of effective connectivity.
Integral Transforms from Finite Data: An Application of Gaussian Process Regression to Fourier Analysis
This is possible because the posterior expectation of Gaussian process regression maps a finite set of samples to a function defined on the whole real line, expressed as a linear combination of covariance functions.
Estimating Nonlinear Dynamics with the ConvNet Smoother
Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering.
Complex-valued Gaussian Process Regression for Time Series Analysis
Furthermore, the complex-valued Gaussian process regression allows to incorporate prior information about the structure in signal and noise and thereby to tailor the analysis to the features of the signal.
Analysis of Nonstationary Time Series Using Locally Coupled Gaussian Processes
In this paper, we introduce a new framework for analyzing nonstationary time series using locally stationary Gaussian process analysis with parameters that are coupled through a hidden Markov model.
Dynamic Decomposition of Spatiotemporal Neural Signals
Neural signals are characterized by rich temporal and spatiotemporal dynamics that reflect the organization of cortical networks.
