Search Results for author:
Found 19 papers, 15 papers with code
ThunderKittens: Simple, Fast, and Adorable AI Kernels
We match CuBLAS and FlashAttention-3 on GEMM and attention inference performance and outperform the strongest baselines by $10-40\%$ on attention backwards, $8\times$ on state space models, and $14\times$ on linear attention.
Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT
Retrieval pipelines-an integral component of many machine learning systems-perform poorly in domains where documents are long (e. g., 10K tokens or more) and where identifying the relevant document requires synthesizing information across the entire text.
Hydragen: High-Throughput LLM Inference with Shared Prefixes
Decoding in this large-batch setting can be bottlenecked by the attention operation, which reads large key-value (KV) caches from memory and computes inefficient matrix-vector products for every sequence in the batch.
FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores
FlashFFTConv uses a matrix decomposition that computes the FFT using matrix multiply units and enables kernel fusion for long sequences, reducing I/O.
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
We ask: are there performant architectures that can scale sub-quadratically along sequence length and model dimension?
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU
As a result, when running OPT-175B on a single 16GB GPU, FlexGen achieves significantly higher throughput compared to state-of-the-art offloading systems, reaching a generation throughput of 1 token/s for the first time with an effective batch size of 144.
Hyena Hierarchy: Towards Larger Convolutional Language Models
Recent advances in deep learning have relied heavily on the use of large Transformers due to their ability to learn at scale.
Ranked #37 on
Language Modelling
on WikiText-103
Simple Hardware-Efficient Long Convolutions for Sequence Modeling
We find that a key requirement to achieving high performance is keeping the convolution kernels smooth.
Hungry Hungry Hippos: Towards Language Modeling with State Space Models
First, we use synthetic language modeling tasks to understand the gap between SSMs and attention.
Ranked #2 on
Language Modelling
on The Pile
(Test perplexity metric)
Lost in Transmission: On the Impact of Networking Corruptions on Video Machine Learning Models
In a simulation study, we investigate (1) what artifacts networking corruptions cause, (2) how such artifacts affect ML models, and (3) whether standard robustness methods can mitigate their negative effects.
TABi: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval
Entity retrieval--retrieving information about entity mentions in a query--is a key step in open-domain tasks, such as question answering or fact checking.
Perfectly Balanced: Improving Transfer and Robustness of Supervised Contrastive Learning
We first prove that adding a weighted class-conditional InfoNCE loss to SupCon controls the degree of spread.
Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision
Despite the black-box nature of foundation models, we prove results characterizing how our approach improves performance and show that lift scales with the smoothness of label distributions in embedding space.
Train and You'll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings
Our goal is to enable machine learning systems to be trained interactively.
Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods
In this work, we show that, for a class of latent variable models highly applicable to weak supervision, we can find a closed-form solution to model parameters, obviating the need for iterative solutions like stochastic gradient descent (SGD).
Multi-Resolution Weak Supervision for Sequential Data
Multi-resolution sources exacerbate this challenge due to complex correlations and sample complexity that scales in the length of the sequence.
Rekall: Specifying Video Events using Compositions of Spatiotemporal Labels
Many real-world video analysis applications require the ability to identify domain-specific events in video, such as interviews and commercials in TV news broadcasts, or action sequences in film.
Influencing Flock Formation in Low-Density Settings
In the interest of learning how to control flocking behavior, recent work in the multiagent systems literature has explored the use of influencing agents for guiding flocking agents to face a target direction.
