Search Results for author:
Found 11 papers, 8 papers with code
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
We ask: are there performant architectures that can scale sub-quadratically along sequence length and model dimension?
Simple Hardware-Efficient Long Convolutions for Sequence Modeling
We find that a key requirement to achieving high performance is keeping the convolution kernels smooth.
Hungry Hungry Hippos: Towards Language Modeling with State Space Models
First, we use synthetic language modeling tasks to understand the gap between SSMs and attention.
Ranked #2 on
Language Modelling
on The Pile
(Test perplexity metric)
Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Data
At their core, these frameworks learn the dynamics of brain activity by modeling sequences of activity akin to how sequences of text are modeled in NLP.
Differentiable programming for functional connectomics
We introduce a new analytic paradigm and software toolbox that implements common operations used in functional connectomics as fully differentiable processing blocks.
Comparing interpretation methods in mental state decoding analyses with deep learning models
Deep learning (DL) models find increasing application in mental state decoding, where researchers seek to understand the mapping between mental states (e. g., perceiving fear or joy) and brain activity by identifying those brain regions (and networks) whose activity allows to accurately identify (i. e., decode) these states.
Evaluating deep transfer learning for whole-brain cognitive decoding
Here, we systematically evaluate TL for the application of DL models to the decoding of cognitive states (e. g., viewing images of faces or houses) from whole-brain functional Magnetic Resonance Imaging (fMRI) data.
Challenges for cognitive decoding using deep learning methods
In cognitive decoding, researchers aim to characterize a brain region's representations by identifying the cognitive states (e. g., accepting/rejecting a gamble) that can be identified from the region's activity.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Deep Transfer Learning For Whole-Brain fMRI Analyses
Even further, the pre-trained DL model variant is already able to correctly decode 67. 51% of the cognitive states from a test dataset with 100 individuals, when fine-tuned on a dataset of the size of only three subjects.
Analyzing Neuroimaging Data Through Recurrent Deep Learning Models
We further demonstrate DeepLight's ability to study the fine-grained temporo-spatial variability of brain activity over sequences of single fMRI samples.
