Search Results for author:
Found 10 papers, 2 papers with code
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
Large language models (LLMs) are typically trained on general source data for various domains, but a recent surge in domain-specific LLMs has shown their potential to outperform general-purpose models in domain-specific tasks (e. g., biomedicine).
Ranked #10 on
Question Answering
on PubMedQA
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency
Recent research has focused on weight sparsity in neural network training to reduce FLOPs, aiming for improved efficiency (test accuracy w. r. t training FLOPs).
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models
In this work, we show the benefits of using unstructured weight sparsity to train only a subset of weights during pre-training (Sparse Pre-training) and then recover the representational capacity by allowing the zeroed weights to learn (Dense Fine-tuning).
Instance-Level Task Parameters: A Robust Multi-task Weighting Framework
When applied to datasets where one or more tasks can have noisy annotations, the proposed method learns to prioritize learning from clean labels for a given task, e. g. reducing surface estimation errors by up to 60%.
Training With Data Dependent Dynamic Learning Rates
This setting is widely adopted under the assumption that loss functions for each instance are similar in nature, and hence, a common learning rate can be used.
Dynamic curriculum learning via data parameters for noise robust keyword spotting
We propose dynamic curriculum learning via data parameters for noise robust keyword spotting.
Learning Soft Labels via Meta Learning
One-hot labels do not represent soft decision boundaries among concepts, and hence, models trained on them are prone to overfitting.
Data Parameters: A New Family of Parameters for Learning a Differentiable Curriculum
To the best of our knowledge, our work is the first curriculum learning method to show gains on large scale image classification and detection tasks.
Learning Unsupervised Visual Grounding Through Semantic Self-Supervision
Localizing natural language phrases in images is a challenging problem that requires joint understanding of both the textual and visual modalities.
Convolutional Neural Fabrics
Despite the success of CNNs, selecting the optimal architecture for a given task remains an open problem.
