Search Results for author:
Found 16 papers, 9 papers with code
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache
Large Language Models (LLMs) are increasingly being deployed on edge devices for long-context settings, creating a growing need for fast and efficient long-context inference.
SALSA: Soup-based Alignment Learning for Stronger Adaptation in RLHF
As a result, policy optimization is often trapped in a narrow region of the parameter space, leading to suboptimal alignment and performance.
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
Large Language Models (LLMs) have transformed natural language processing, but face significant challenges in widespread deployment due to their high runtime cost.
KV Prediction for Improved Time to First Token
In our method, a small auxiliary model is used to process the prompt and produce an approximation of the KV cache used by a base model.
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data
Contrastive learning has emerged as a transformative method for learning effective visual representations through the alignment of image and text embeddings.
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
To this end, we release OpenELM, a state-of-the-art open language model.
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement
While CLIP is scalable, promptable, and robust to distribution shifts on image classification tasks, it lacks object localization capabilities.
Diffusion Models as Masked Audio-Video Learners
Over the past several years, the synchronization between audio and visual signals has been leveraged to learn richer audio-visual representations.
On the Efficacy of Multi-scale Data Samplers for Vision Applications
Multi-scale resolution training has seen an increased adoption across multiple vision tasks, including classification and detection.
Bytes Are All You Need: Transformers Operating Directly On File Bytes
Compared to Perceiver IO, our model requires absolutely no modality-specific processing at inference time, and uses an order of magnitude fewer parameters at equivalent accuracy on ImageNet.
RangeAugment: Efficient Online Augmentation with Range Learning
To answer the open question on the importance of magnitude ranges for each augmentation operation, we introduce RangeAugment that allows us to efficiently learn the range of magnitudes for individual as well as composite augmentation operations.
SPIN: An Empirical Evaluation on Sharing Parameters of Isotropic Networks
In this paper, we perform an empirical evaluation on methods for sharing parameters in isotropic networks (SPIN).
LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time
Our models require no retraining, thus our subspace of models can be deployed entirely on-device to allow adaptive network compression at inference time.
Learning Neural Network Subspaces
Recent observations have advanced our understanding of the neural network optimization landscape, revealing the existence of (1) paths of high accuracy containing diverse solutions and (2) wider minima offering improved performance.
Layer-Wise Data-Free CNN Compression
We also show how to precondition the network to improve the accuracy of our layer-wise compression method.
Label Refinery: Improving ImageNet Classification through Label Progression
Among the three main components (data, labels, and models) of any supervised learning system, data and models have been the main subjects of active research.
