Search Results for author:
Found 5 papers, 1 papers with code
The Relational Bottleneck as an Inductive Bias for Efficient Abstraction
A central challenge for cognitive science is to explain how abstract concepts are acquired from limited experience.
Trainability, Expressivity and Interpretability in Gated Neural ODEs
Using this measure, we explore how the phase-space dimension of the nODEs and the complexity of the function modeling the flow field contribute to expressivity.
Emergence of memory manifolds
The ability to store continuous variables in the state of a biological system (e. g. a neural network) is critical for many behaviours.
Theory of gating in recurrent neural networks
The gate modulating the dimensionality can induce a novel, discontinuous chaotic transition, where inputs push a stable system to strong chaotic activity, in contrast to the typically stabilizing effect of inputs.
Gating creates slow modes and controls phase-space complexity in GRUs and LSTMs
Here, we take the perspective of studying randomly initialized LSTMs and GRUs as dynamical systems, and ask how the salient dynamical properties are shaped by the gates.
