Search Results for author:
Found 14 papers, 11 papers with code
Probabilistic Active Meta-Learning
However, this setting does not take into account the sequential nature that naturally arises when training a model from scratch in real-life: how do we collect a set of training tasks in a data-efficient manner?
Causal Effect Inference for Structured Treatments
We address the estimation of conditional average treatment effects (CATEs) for structured treatments (e. g., graphs, images, texts).
When Do Flat Minima Optimizers Work?
Recently, flat-minima optimizers, which seek to find parameters in low-loss neighborhoods, have been shown to improve a neural network's generalization performance over stochastic and adaptive gradient-based optimizers.
Evaluating Self-Supervised Learning for Molecular Graph Embeddings
Graph Self-Supervised Learning (GSSL) provides a robust pathway for acquiring embeddings without expert labelling, a capability that carries profound implications for molecular graphs due to the staggering number of potential molecules and the high cost of obtaining labels.
Causal Machine Learning: A Survey and Open Problems
Causal Machine Learning (CausalML) is an umbrella term for machine learning methods that formalize the data-generation process as a structural causal model (SCM).
Stop Wasting My Time! Saving Days of ImageNet and BERT Training with Latest Weight Averaging
Training vision or language models on large datasets can take days, if not weeks.
DAG Learning on the Permutahedron
We propose a continuous optimization framework for discovering a latent directed acyclic graph (DAG) from observational data.
Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases
The problem of spurious correlations (SCs) arises when a classifier relies on non-predictive features that happen to be correlated with the labels in the training data.
The MiniPile Challenge for Data-Efficient Language Models
MiniPile is a 6GB subset of the deduplicated 825GB The Pile corpus.
TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models
In addition, generative data augmentation (GDA) has been shown to produce more diverse and flexible data.
Early Weight Averaging meets High Learning Rates for LLM Pre-training
Specifically, we pre-trained nanoGPT-2 models of varying sizes, small (125M), medium (335M), and large (770M)on the OpenWebText dataset, comprised of 9B tokens.
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models
The computation necessary for training Transformer-based language models has skyrocketed in recent years.
Challenges and Applications of Large Language Models
Due to the fast pace of the field, it is difficult to identify the remaining challenges and already fruitful application areas.
Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models
The in-context learning ability of large language models (LLMs) enables them to generalize to novel downstream tasks with relatively few labeled examples.
