Search Results for author:
Found 12 papers, 1 papers with code
Large language models surpass human experts in predicting neuroscience results
LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts.
Bridging Associative Memory and Probabilistic Modeling
Based on the observation that associative memory's energy functions can be seen as probabilistic modeling's negative log likelihoods, we build a bridge between the two that enables useful flow of ideas in both directions.
Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model
Stepwise inference protocols, such as scratchpads and chain-of-thought, help language models solve complex problems by decomposing them into a sequence of simpler subproblems.
Disentangling Fact from Grid Cell Fiction in Trained Deep Path Integrators
Work on deep learning-based models of grid cells suggests that grid cells generically and robustly arise from optimizing networks to path integrate, i. e., track one's spatial position by integrating self-velocity signals.
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
At both the population and single-cell levels, we find evidence suggesting that neither of the assumptions are likely true in biological neural representations.
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks
Transformers trained on huge text corpora exhibit a remarkable set of capabilities, e. g., performing basic arithmetic.
Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks
Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in shared computational subtasks.
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever larger into the highly overparameterized (data undersampled) regime.
See and Copy: Generation of complex compositional movements from modular and geometric RNN representations
A hallmark of biological intelligence and control is combinatorial generalization: animals are able to learn various things, then piece them together in new combinations to produce appropriate outputs for new tasks.
Winning the lottery with neural connectivity constraints: faster learning across cognitive tasks with spatially constrained sparse RNNs
We study LM-RNNs in a multitask learning setting relevant to cognitive systems neuroscience with a commonly used set of tasks, 20-Cog-tasks [Yang et al., 2019].
Attractor and integrator networks in the brain
In this review, we describe the singular success of attractor neural network models in describing how the brain maintains persistent activity states for working memory, error-corrects, and integrates noisy cues.
Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice
Third, the geometry of RNN dynamics reflects an induced coupling between the two separate inference processes necessary to solve the task.
