Search Results for author:
Found 14 papers, 4 papers with code
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters.
Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
While code-mixing is a common linguistic practice in many parts of the world, collecting high-quality and low-cost code-mixed data remains a challenge for natural language processing (NLP) research.
Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex
AlphaZero, an approach to reinforcement learning that couples neural networks and Monte Carlo tree search (MCTS), has produced state-of-the-art strategies for traditional board games like chess, Go, shogi, and Hex.
One Venue, Two Conferences: The Separation of Chinese and American Citation Networks
At NeurIPS, American and Chinese institutions cite papers from each other's regions substantially less than they cite endogamously.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation
We conducted a series of experiments to investigate whether and how distillation can be used to accelerate training using ResNet-50 trained on ImageNet and BERT trained on C4 with a masked language modeling objective and evaluated on GLUE, using common enterprise hardware (8x NVIDIA A100).
Efficient Methods for Natural Language Processing: A Survey
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows.
Strengthening Subcommunities: Towards Sustainable Growth in AI Research
AI's rapid growth has been felt acutely by scholarly venues, leading to growing pains within the peer review process.
Model Selection's Disparate Impact in Real-World Deep Learning Applications
Algorithmic fairness has emphasized the role of biased data in automated decision outcomes.
Hyperparameter Optimization Is Deceiving Us, and How to Stop It
Recent empirical work shows that inconsistent results based on choice of hyperparameter optimization (HPO) configuration are a widespread problem in ML research.
Bespoke vs. Prêt-à-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding
The observation of sparse trainable sub-networks within over-parametrized networks - also known as Lottery Tickets (LTs) - has prompted inquiries around their trainability, scaling, uniqueness, and generalization properties.
dagger: A Python Framework for Reproducible Machine Learning Experiment Orchestration
Many research directions in machine learning, particularly in deep learning, involve complex, multi-stage experiments, commonly involving state-mutating operations acting on models along multiple paths of execution.
Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging
Averaging predictions from 10 models reduced variability by nearly 70% (mean coefficient of variation from 0. 543 to 0. 169, t-test 15. 96, p-value < 0. 0001).
The Scientific Method in the Science of Machine Learning
In the quest to align deep learning with the sciences to address calls for rigor, safety, and interpretability in machine learning systems, this contribution identifies key missing pieces: the stages of hypothesis formulation and testing, as well as statistical and systematic uncertainty estimation -- core tenets of the scientific method.
