Search Results for author:
Found 22 papers, 12 papers with code
Does `Deep Learning on a Data Diet' reproduce? Overall yes, but GraNd at Initialization does not
Unfortunately, neither the GraNd score at initialization nor the input norm surpasses random pruning in performance.
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning
We examine a simple stochastic strategy for adapting well-known single-point acquisition functions to allow batch active learning.
BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning
We develop BatchBALD, a tractable approximation to the mutual information between a batch of points and model parameters, which we use as an acquisition function to select multiple informative points jointly for the task of deep Bayesian active learning.
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt
But most computation and time is wasted on redundant and noisy points that are already learnt or not learnable.
Deep Deterministic Uncertainty: A Simple Baseline
Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive.
MDP environments for the OpenAI Gym
The OpenAI Gym provides researchers and enthusiasts with simple to use environments for reinforcement learning.
Prediction-Oriented Bayesian Active Learning
Information-theoretic approaches to active learning have traditionally focused on maximising the information gathered about the model parameters, most commonly by optimising the BALD score.
Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data
We introduce causal, Bayesian acquisition functions grounded in information theory that bias data acquisition towards regions with overlapping support to maximize sample efficiency for learning personalized treatment effects.
Black-Box Batch Active Learning for Regression
This approach is compatible with a wide range of machine learning models, including regular and Bayesian deep learning models and non-differentiable models such as random forests.
A Note on "Assessing Generalization of SGD via Disagreement"
Several recent works find empirically that the average test error of deep neural networks can be estimated via the prediction disagreement of models, which does not require labels.
Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantities
Recently proposed methods in data subset selection, that is active learning and active sampling, use Fisher information, Hessians, similarity matrices based on gradients, and gradient lengths to estimate how informative data is for a model's training.
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models.
Unpacking Information Bottlenecks: Surrogate Objectives for Deep Learning
The Information Bottleneck principle offers both a mechanism to explain how deep neural networks train and generalize, as well as a regularized objective with which to train models.
PowerEvaluationBALD: Efficient Evaluation-Oriented Deep (Bayesian) Active Learning with Stochastic Acquisition Functions
We develop BatchEvaluationBALD, a new acquisition function for deep Bayesian active learning, as an expansion of BatchBALD that takes into account an evaluation set of unlabeled data, for example, the pool set.
Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers
Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variables.
A Practical & Unified Notation for Information-Theoretic Quantities in ML
A practical notation can convey valuable intuitions and concisely express new ideas.
Prioritized training on points that are learnable, worth learning, and not yet learned (workshop version)
We introduce Goldilocks Selection, a technique for faster model training which selects a sequence of training points that are "just right".
Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Sampling
They are more realistic than previously suggested ones, building on work by Wen et al. (2021) and Osband et al. (2022), and focus on evaluating the performance of approximate BNNs in an online supervised setting.
Speeding Up BatchBALD: A k-BALD Family of Approximations for Active Learning
One commonly used technique for active learning is BatchBALD, which uses Bayesian neural networks to find the most informative points to label in a pool set.
Deep Deterministic Uncertainty: A New Simple Baseline
Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive.
Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitions
At its core, this thesis aims to enhance the practicality of deep learning by improving the label and training efficiency of deep learning models.
