A simple connection from loss flatness to compressed representations in neural networks

no code implementations • 3 Oct 2023 • Shirui Chen, Stefano Recanatesi, Eric Shea-Brown

The generalization capacity of deep neural networks has been studied in a variety of ways, including at least two distinct categories of approach: one based on the shape of the loss landscape in parameter space, and the other based on the structure of the representation manifold in feature space (that is, in the space of unit activities).

Expressive probabilistic sampling in recurrent neural networks

1 code implementation • NeurIPS 2023 • Shirui Chen, Linxing Preston Jiang, Rajesh P. N. Rao, Eric Shea-Brown

We show that the firing rate dynamics of a recurrent neural circuit with a separate set of output units can sample from an arbitrary probability distribution.

Denoising