Search Results for author:
Found 7 papers, 4 papers with code
Revisiting Intermediate-Layer Matching in Knowledge Distillation: Layer-Selection Strategy Doesn't Matter (Much)
Knowledge distillation (KD) is a popular method of transferring knowledge from a large "teacher" model to a small "student" model.
Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization
Large language models have been increasing in size due to their success in a wide range of applications.
Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsing
We address unsupervised discontinuous constituency parsing, where we observe a high variance in the performance of the only previous model in the literature.
EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation
The ability of zero-shot translation emerges when we train a multilingual model with certain translation directions; the model can then directly translate in unseen directions.
f-Divergence Minimization for Sequence-Level Knowledge Distillation
Experiments across four datasets show that our methods outperform existing KD approaches, and that our symmetric distilling losses can better force the student to learn from the teacher distribution.
An Equal-Size Hard EM Algorithm for Diverse Dialogue Generation
Open-domain dialogue systems aim to interact with humans through natural language texts in an open-ended fashion.
An Empirical Study on the Overlapping Problem of Open-Domain Dialogue Datasets
Open-domain dialogue systems aim to converse with humans through text, and dialogue research has heavily relied on benchmark datasets.
