Search Results for author:
Found 11 papers, 3 papers with code
Efficient Model Development through Fine-tuning Transfer
In a multilingual model development setting, we show that this approach can significantly increase performance on target-language tasks without retraining, achieving an absolute improvement of 4. 7% and 15. 5% on Global MMLU for Malagasy and Turkish, respectively, compared to Llama 3. 1 8B Instruct.
Self-Vocabularizing Training for Neural Machine Translation
Past vocabulary learning techniques identify relevant vocabulary before training, relying on statistical and entropy-based assumptions that largely neglect the role of model training.
Scaling Parameter-Constrained Language Models with Quality Data
Scaling laws in language modeling traditionally quantify training loss as a function of dataset size and model parameters, providing compute-optimal estimates but often neglecting the impact of data quality on model generalization.
Target-Aware Language Modeling via Granular Data Sampling
A cost-effective and straightforward approach is sampling with low-dimensional data features, which allows to select large-scale pretraining data for domain-specific use cases.
Exploring the Effectiveness and Consistency of Task Selection in Intermediate-Task Transfer Learning
Identifying beneficial tasks to transfer from is a critical step toward successful intermediate-task transfer learning.
Modeling Orthographic Variation Improves NLP Performance for Nigerian Pidgin
We test the effect of this data augmentation on two critical NLP tasks: machine translation and sentiment analysis.
On The Open Prompt Challenge In Conditional Audio Generation
Text-to-audio generation (TTA) produces audio from a text description, learning from pairs of audio samples and hand-annotated text.
In-Context Prompt Editing For Conditional Audio Generation
We show that the framework enhanced the audio quality across the set of collected user prompts, which were edited with reference to the training captions as exemplars.
Low-Resource Cross-Lingual Adaptive Training for Nigerian Pidgin
In this work, we target on improving upon both text classification and translation of Nigerian Pidgin (Naija) by collecting a large-scale parallel English-Pidgin corpus and further propose a framework of cross-lingual adaptive training that includes both continual and task adaptive training so as to adapt a base pre-trained model to low-resource languages.
Revisiting Sample Size Determination in Natural Language Understanding
Knowing exactly how many data points need to be labeled to achieve a certain model performance is a hugely beneficial step towards reducing the overall budgets for annotation.
