Search Results for author:
Found 38 papers, 10 papers with code
Memorization to Generalization: Emergence of Diffusion Models from Associative Memory
In the small data regime the diffusion model exhibits a strong memorization phase, where the network creates distinct basins of attraction around each sample in the training set, akin to the Hopfield model below the critical memory load.
Entropic Time Schedulers for Generative Diffusion Models
In particular, we found that the image quality in pretrained EDM2 models, as evaluated by FID and FD-DINO scores, can be substantially increased by the rescaled entropic time reparameterization without increasing the number of function evaluations, with greater improvements in the few NFEs regime.
Training Consistency Models with Variational Noise Coupling
Consistency Training (CT) has recently emerged as a promising alternative to diffusion models, achieving competitive performance in image generation tasks.
Dynamic Negative Guidance of Diffusion Models
Negative Prompting (NP) is widely utilized in diffusion models, particularly in text-to-image applications, to prevent the generation of undesired features.
Losing dimensions: Geometric memorization in generative diffusion
Generative diffusion processes are state-of-the-art machine learning models deeply connected with fundamental concepts in statistical physics.
Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion
In this paper, we investigate the latent geometry of generative diffusion models under the manifold hypothesis.
Robust and highly scalable estimation of directional couplings from time-shifted signals
The estimation of directed couplings between the nodes of a network from indirect measurements is a central methodological challenge in scientific fields such as neuroscience, systems biology and economics.
The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instability
Generative diffusion models have achieved spectacular performance in many areas of machine learning and generative modeling.
Stationarity without mean reversion in improper Gaussian processes
The main contribution of the paper is the introduction of a large family of smooth non-reverting covariance functions that closely resemble the kernels commonly used in the GP literature (e. g. squared exponential and Mat\'ern class).
In search of dispersed memories: Generative diffusion models are associative memory networks
In this work we show that generative diffusion models can be interpreted as energy-based models and that, when trained on discrete patterns, their energy function is (asymptotically) identical to that of modern Hopfield networks.
Spontaneous Symmetry Breaking in Generative Diffusion Models
In this paper, we show that the dynamics of these models exhibit a spontaneous symmetry breaking that divides the generative dynamics into two distinct phases: 1) A linear steady-state dynamics around a central fixed-point and 2) an attractor dynamics directed towards the data manifold.
Deterministic training of generative autoencoders using invertible layers
In this work, we provide a deterministic alternative to the stochastic variational training of generative autoencoders.
Embedded-model flows: Combining the inductive biases of model-free deep learning and explicit probabilistic modeling
We also introduce gated structured layers, which allow bypassing the parts of the models that fail to capture the statistics of the data.
Knowledge is reward: Learning optimal exploration by predictive reward cashing
There is a strong link between the general concept of intelligence and the ability to collect and use information.
Gradient-adjusted Incremental Target Propagation Provides Effective Credit Assignment in Deep Neural Networks
Many of the recent advances in the field of artificial intelligence have been fueled by the highly successful backpropagation of error (BP) algorithm, which efficiently solves the credit assignment problem in artificial neural networks.
Automatic variational inference with cascading flows
We evaluate the performance of the new variational programs in a series of structured inference problems.
Hyperrealistic neural decoding: Reconstruction of face stimuli from fMRI measurements via the GAN latent space
We introduce a new framework for hyperrealistic reconstruction of perceived naturalistic stimuli from brain recordings.
The 3TConv: An Intrinsic Approach to Explainable 3D CNNs
In a 3TConv the 3D convolutional filter is obtained by learning a 2D filter and a set of temporal transformation parameters, resulting in a sparse filter requiring less parameters.
Explainable 3D Convolutional Neural Networks by Learning Temporal Transformations
Finally, we implicitly demonstrate that, in popular ConvNets, the 2DConv can be replaced with a 3TConv and that the weights can be transferred to yield pretrained 3TConvs.
GAIT-prop: A biologically plausible learning rule derived from backpropagation of error
An alternative called target propagation proposes to solve this implausibility by using a top-down model of neural activity to convert an error at the output of a neural network into layer-wise and plausible 'targets' for every unit.
Overcoming the Weight Transport Problem via Spike-Timing-Dependent Weight Inference
We propose a solution to the weight transport problem, which questions the biological plausibility of the backpropagation algorithm.
Automatic structured variational inference
However, the performance of the variational approach depends on the choice of an appropriate variational family.
The Indian Chefs Process
This paper introduces the Indian Chefs Process (ICP), a Bayesian nonparametric prior on the joint space of infinite directed acyclic graphs (DAGs) and orders that generalizes Indian Buffet Processes.
Background Hardly Matters: Understanding Personality Attribution in Deep Residual Networks
It is important that we understand the driving factors behind the predictions, in humans and in deep neural networks.
Temporal Factorization of 3D Convolutional Kernels
3D convolutional neural networks are difficult to train because they are parameter-expensive and data-hungry.
Bayesian nonparametric discontinuity design
Quasi-experimental research designs, such as regression discontinuity and interrupted time series, allow for causal inference in the absence of a randomized controlled trial, at the cost of additional assumptions.
k-GANs: Ensemble of Generative Models with Semi-Discrete Optimal Transport
A possible way of dealing with this problem is to use an ensemble of GANs, where (ideally) each network models a single mode.
Perturbative estimation of stochastic gradients
Furthermore, we introduce a family of variance reduction techniques that can be applied to other gradient estimators.
Wasserstein variational gradient descent: From semi-discrete optimal transport to ensemble variational inference
The solution of the resulting optimal transport problem provides both a particle approximation and a set of optimal transportation densities that map each particle to a segment of the posterior distribution.
Wasserstein Variational Inference
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory.
Forward Amortized Inference for Likelihood-Free Variational Marginalization
In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss.
The Kernel Mixture Network: A Nonparametric Method for Conditional Density Estimation of Continuous Random Variables
In the Bayesian filtering example, we show that the method can be used to filter complex nonlinear and non-Gaussian signals defined on manifolds.
GP CaKe: Effective brain connectivity with causal kernels
Here we propose to model this causal interaction using integro-differential equations and causal kernels that allow for a rich analysis of effective connectivity.
Integral Transforms from Finite Data: An Application of Gaussian Process Regression to Fourier Analysis
This is possible because the posterior expectation of Gaussian process regression maps a finite set of samples to a function defined on the whole real line, expressed as a linear combination of covariance functions.
Estimating Nonlinear Dynamics with the ConvNet Smoother
Estimating the state of a dynamical system from a series of noise-corrupted observations is fundamental in many areas of science and engineering.
Complex-valued Gaussian Process Regression for Time Series Analysis
Furthermore, the complex-valued Gaussian process regression allows to incorporate prior information about the structure in signal and noise and thereby to tailor the analysis to the features of the signal.
Analysis of Nonstationary Time Series Using Locally Coupled Gaussian Processes
In this paper, we introduce a new framework for analyzing nonstationary time series using locally stationary Gaussian process analysis with parameters that are coupled through a hidden Markov model.
Dynamic Decomposition of Spatiotemporal Neural Signals
Neural signals are characterized by rich temporal and spatiotemporal dynamics that reflect the organization of cortical networks.
