Search Results for author:
Found 7 papers, 2 papers with code
Measuring Feature Sparsity in Language Models
We show our metrics can predict the level of sparsity on synthetic sparse linear activations, and can distinguish between sparse linear data and several other distributions.
Nearly $d$-Linear Convergence Bounds for Diffusion Models via Stochastic Localization
We provide the first convergence bounds which are linear in the data dimension (up to logarithmic factors) assuming only finite second moments of the data distribution.
Error Bounds for Flow Matching Methods
Previous work derived bounds on the approximation error of diffusion models under the stochastic sampling regime, given assumptions on the $L^2$ loss.
From Denoising Diffusions to Denoising Markov Models
We propose a unifying framework generalising this approach to a wide class of spaces and leading to an original extension of score matching.
Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics
We then provide two complementary theoretical analyses of the VR-IWAE bound and thus of the standard IWAE bound.
Polysemanticity and Capacity in Neural Networks
We show that in a toy model the optimal capacity allocation tends to monosemantically represent the most important features, polysemantically represent less important features (in proportion to their impact on the loss), and entirely ignore the least important features.
A Continuous Time Framework for Discrete Denoising Models
We provide the first complete continuous time framework for denoising diffusion models of discrete data.
