Search Results for author:
Found 9 papers, 4 papers with code
SALSA: Soup-based Alignment Learning for Stronger Adaptation in RLHF
As a result, policy optimization is often trapped in a narrow region of the parameter space, leading to suboptimal alignment and performance.
Computational Bottlenecks of Training Small-scale Large Language Models
While large language models (LLMs) dominate the AI landscape, Small-scale large Language Models (SLMs) are gaining attention due to cost and efficiency demands from consumers.
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
While the performance of LLMs on GSM8K has significantly improved in recent years, it remains unclear whether their mathematical reasoning capabilities have genuinely advanced, raising questions about the reliability of the reported metrics.
Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Models
Large Language Models (LLMs) typically generate outputs token by token using a fixed compute budget, leading to inefficient resource utilization.
Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
In this paper, we introduce HyperCloning, a method that can expand the parameters of a pre-trained language model to those of a larger model with increased hidden dimensions.
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
To this end, we release OpenELM, a state-of-the-art open language model.
LLM in a flash: Efficient Large Language Model Inference with Limited Memory
These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively.
Ranked #67 on
Sentence Completion
on HellaSwag
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models
Large Language Models (LLMs) with billions of parameters have drastically transformed AI applications.
ActiLabel: A Combinatorial Transfer Learning Framework for Activity Recognition
Sensor-based human activity recognition has become a critical component of many emerging applications ranging from behavioral medicine to gaming.
