Search Results for author:
Found 6 papers, 4 papers with code
DRoP: Distributionally Robust Pruning
In the era of exceptionally data-hungry models, careful selection of the training data is essential to mitigate the extensive costs of deep learning.
Towards Efficient Active Learning in NLP via Pretrained Representations
Fine-tuning Large Language Models (LLMs) is now a common approach for text classification in a wide range of applications.
Deconstructing the Goldilocks Zone of Neural Network Initialization
The second-order properties of the training loss have a massive impact on the optimization dynamics of deep learning models.
ImpressLearn: Continual Learning via Combined Task Impressions
Starting from existing work on network masking (Wortsman et al., 2020), we show that simply learning a linear combination of a small number of task-specific supermasks (impressions) on a randomly initialized backbone network is sufficient to both retain accuracy on previously learned tasks, as well as achieve high accuracy on unseen tasks.
Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsity
Neural network pruning is a fruitful area of research with surging interest in high sparsity regimes.
Automating Artifact Detection in Video Games
Based on a sample of representative screen corruption examples, the model was able to identify 10 of the most commonly occurring screen artifacts with reasonable accuracy.
