Search Results for author: Arjun Desai

Found 7 papers, 4 papers with code

Llamba: Scaling Distilled Recurrent Models for Efficient Language Processing

no code implementations20 Feb 2025 Aviv Bick, Tobias Katsch, Nimit Sohoni, Arjun Desai, Albert Gu

We introduce Llamba, a family of efficient recurrent language models distilled from Llama-3. x into the Mamba architecture.

Mamba

Prospector Heads: Generalized Feature Attribution for Large Models & Data

1 code implementation18 Feb 2024 Gautam Machiraju, Alexander Derry, Arjun Desai, Neel Guha, Amir-Hossein Karimi, James Zou, Russ Altman, Christopher Ré, Parag Mallick

Feature attribution, the ability to localize regions of the input data that are relevant for classification, is an important capability for ML models in scientific and biomedical domains.

Monarch: Expressive Structured Matrices for Efficient and Accurate Training

2 code implementations1 Apr 2022 Tri Dao, Beidi Chen, Nimit Sohoni, Arjun Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra, Christopher Ré

To address these issues, we propose a class of matrices (Monarch) that is hardware-efficient (they are parameterized as products of two block-diagonal matrices for better hardware utilization) and expressive (they can represent many commonly used transforms).

Language Modeling Language Modelling +1

Cannot find the paper you are looking for? You can Submit a new open access paper.