Search Results for author:
Found 22 papers, 7 papers with code
Are task representations gated in macaque prefrontal cortex?
A recent paper (Flesch et al, 2022) describes behavioural and neural data suggesting that task representations are gated in the prefrontal cortex in both humans and macaques.
Analyzing Populations of Neural Networks via Dynamical Model Embedding
The specific model to be emulated is determined by a model embedding vector that the meta-model takes as input; these model embedding vectors constitute a manifold corresponding to the given population of models.
Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution
Neuroscience has long been an essential driver of progress in artificial intelligence (AI).
Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems
The results are a trained SLDS variant that closely approximates the RNN, an auxiliary function that can produce a fixed point for each point in state-space, and a trained nonlinear RNN whose dynamics have been regularized such that its first-order terms perform the computation, if possible.
Organizing recurrent network dynamics by task-computation to enable continual learning
Here, we develop a novel learning rule designed to minimize interference between sequentially learned tasks in recurrent networks.
Reverse engineering learned optimizers reveals known and novel mechanisms
Learned optimizers are algorithms that can themselves be trained to solve optimization problems.
The geometry of integration in text classification RNNs
Using tools from dynamical systems analysis, we study recurrent networks trained on a battery of both natural and synthetic text classification tasks.
How recurrent networks implement contextual processing in sentiment analysis
Here, we propose general methods for reverse engineering recurrent neural networks (RNNs) to identify and elucidate contextual processing.
Universality and individuality in neural dynamics across large populations of recurrent networks
To address these foundational questions, we study populations of thousands of networks, with commonly used RNN architectures, trained to solve neuroscientifically motivated tasks and characterize their nonlinear dynamics.
Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics
In this work, we use tools from dynamical systems analysis to reverse engineer recurrent networks trained to perform sentiment classification, a foundational natural language processing task.
Line attractor dynamics in recurrent networks for sentiment classification
Recurrent neural networks (RNNs) are a powerful tool for modeling sequential data.
Hallucinations in Neural Machine Translation
Neural machine translation (NMT) systems have reached state of the art performance in translating text and are in wide deployment.
Task-Driven Convolutional Recurrent Models of the Visual System
Feed-forward convolutional neural networks (CNNs) are currently state-of-the-art for object classification tasks such as ImageNet.
A Dataset and Architecture for Visual Reasoning with a Working Memory
COG is much simpler than the general problem of video analysis, yet it addresses many of the problems relating to visual and logical reasoning and memory -- problems that remain challenging for modern deep learning architectures.
Recurrent Segmentation for Variable Computational Budgets
Importantly, the RNN may be deployed across a range of computational budgets by merely running the model for a variable number of iterations.
An Online Sequence-to-Sequence Model Using Partial Conditioning
However, they are unsuitable for tasks that require incremental predictions to be made as more data arrives or tasks that have long input sequences and output sequences.
Capacity and Trainability in Recurrent Neural Networks
They can store an amount of task information which is linear in the number of parameters, and is approximately 5 bits per parameter.
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability
There exist many problem domains where the interpretability of neural network models is essential for deployment.
Making brain-machine interfaces robust to future neural variability
A major hurdle to clinical translation of brain-machine interfaces (BMIs) is that current decoders, which are trained from a small quantity of recent data, become ineffective when neural recording conditions subsequently change.
LFADS - Latent Factor Analysis via Dynamical Systems
Neuroscience is experiencing a data revolution in which many hundreds or thousands of neurons are recorded simultaneously.
A Neural Transducer
However, they are unsuitable for tasks that require incremental predictions to be made as more data arrives or tasks that have long input sequences and output sequences.
Random Walk Initialization for Training Very Deep Feedforward Networks
We show that the successive application of correctly scaled random matrices to an initial vector results in a random walk of the log of the norm of the resulting vectors, and we compute the scaling that makes this walk unbiased.
