Search Results for author:
Found 5 papers, 3 papers with code
Knowledge Distillation vs. Pretraining from Scratch under a Fixed (Computation) Budget
We further find that KD yields larger gains over pretraining from scratch when the data must be repeated under the fixed computation budget.
One For All & All For One: Bypassing Hyperparameter Tuning with Model Averaging For Cross-Lingual Transfer
Because of this, model selection based on source-language validation is unreliable: it picks model snapshots with suboptimal target-language performance.
Free Lunch: Robust Cross-Lingual Transfer via Model Checkpoint Averaging
The results indicate that averaging model checkpoints yields systematic and consistent performance gains across diverse target languages in all tasks.
SLICER: Sliced Fine-Tuning for Low-Resource Cross-Lingual Transfer for Named Entity Recognition
Large multilingual language models generally demonstrate impressive results in zero-shot cross-lingual transfer, yet often fail to successfully transfer to low-resource languages, even for token-level prediction tasks like named entity recognition (NER).
Multilingual text classification
named-entity-recognition
+3
SEAGLE: A Platform for Comparative Evaluation of Semantic Encoders for Information Retrieval
We introduce Seagle, a platform for comparative evaluation of semantic text encoding models on information retrieval (IR) tasks.
