Search Results for author:
Found 4 papers, 1 papers with code
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
Given the generational gap in available hardware between lay practitioners and the most endowed institutions, LLMs are becoming increasingly inaccessible as they grow in size.
SpacTor-T5: Pre-training T5 Models with Span Corruption and Replaced Token Detection
In this paper, we present SpacTor, a new training procedure consisting of (1) a hybrid objective combining span corruption (SC) and token replacement detection (RTD), and (2) a two-stage curriculum that optimizes the hybrid objective over the initial $\tau$ iterations, then transitions to standard SC loss.
DistillSpec: Improving Speculative Decoding via Knowledge Distillation
Finally, in practical scenarios with models of varying sizes, first using distillation to boost the performance of the target model and then applying DistillSpec to train a well-aligned draft model can reduce decoding latency by 6-10x with minimal performance drop, compared to standard decoding without distillation.
The Practical Challenges of Active Learning: Lessons Learned from Live Experimentation
We tested in a live setting the use of active learning for selecting text sentences for human annotations used in training a Thai segmentation machine learning model.
