Star Attention: Efficient LLM Inference over Long Sequences

1 code implementation • 26 Nov 2024 • Shantanu Acharya, Fei Jia, Boris Ginsburg

Inference with Transformer-based Large Language Models (LLMs) on long sequences is both costly and slow due to the quadratic complexity of the self-attention mechanism.

Computational Efficiency

RULER: What's the Real Context Size of Your Long-Context Language Models?

4 code implementations • 9 Apr 2024 • Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, Boris Ginsburg

Despite achieving nearly perfect accuracy in the vanilla NIAH test, almost all models exhibit large performance drops as the context length increases.

Long-Context Understanding

Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognition

no code implementations • 6 Oct 2022 • Somshubra Majumdar, Shantanu Acharya, Vitaly Lavrukhin, Boris Ginsburg

Automatic speech recognition models are often adapted to improve their accuracy in a new domain.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Every child should have parents: a taxonomy refinement algorithm based on hyperbolic term embeddings

1 code implementation • ACL 2019 • Rami Aly, Shantanu Acharya, Alexander Ossa, Arne Köhn, Chris Biemann, Alexander Panchenko

We introduce the use of Poincar\'e embeddings to improve existing state-of-the-art approaches to domain-specific taxonomy induction from text as a signal for both relocating wrong hyponym terms within a (pre-induced) taxonomy as well as for attaching disconnected terms in a taxonomy.