Search Results for author:
Found 17 papers, 6 papers with code
Leveraging Public Representations for Private Transfer Learning
Motivated by the recent empirical success of incorporating public data into differentially private learning, we theoretically investigate how a shared representation learned from public data can improve private learning.
Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift
Self-training and contrastive learning have emerged as leading techniques for incorporating unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it is absent (semi-supervised learning).
Multitask Learning Can Improve Worst-Group Outcomes
We primarily consider the standard setting of fine-tuning a pre-trained model, where, following recent work \citep{gururangan2020don, dery2023aang}, we multitask the end task with the pre-training objective constructed from the end task data itself.
Deep Neural Networks Tend To Extrapolate Predictably
Rather than extrapolating in arbitrary ways, we observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD.
Contextual Reliability: When Different Features Matter in Different Contexts
We formalize a new setting called contextual reliability which accounts for the fact that the "right" features to use may vary depending on the context.
Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts
Effective machine learning models learn both robust features that directly determine the outcome of interest (e. g., an object with wheels is more likely to be a car), and shortcut features (e. g., an object on a road is more likely to be a car).
Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features
Transfer learning with a small amount of target data is an effective and common approach to adapting a pre-trained model to distribution shifts.
Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts
Some robust training algorithms (e. g., Group DRO) specialize to group shifts and require group information on all training points.
Adversarial Unlearning: Reducing Confidence Along Adversarial Directions
Supervised learning methods trained with maximum likelihood objectives often overfit on training data.
Two Sides of Meta-Learning Evaluation: In vs. Out of Distribution
We categorize meta-learning evaluation into two settings: $\textit{in-distribution}$ [ID], in which the train and test tasks are sampled $\textit{iid}$ from the same underlying task distribution, and $\textit{out-of-distribution}$ [OOD], in which they are not.
Is Support Set Diversity Necessary for Meta-Learning?
Meta-learning is a popular framework for learning with limited data in which an algorithm is produced by training over multiple few-shot learning tasks.
Explaining The Efficacy of Counterfactually Augmented Data
In attempts to produce ML models less reliant on spurious patterns in NLP datasets, researchers have recently proposed curating counterfactually augmented data (CAD) via a human-in-the-loop process in which given some documents and their (initial) labels, humans must revise the text to make a counterfactual label applicable.
Robust Handwriting Recognition with Limited and Noisy Data
Despite the advent of deep learning in computer vision, the general handwriting recognition problem is far from solved.
Nonlinear ISA with Auxiliary Variables for Learning Speech Representations
This paper extends recent work on nonlinear Independent Component Analysis (ICA) by introducing a theoretical framework for nonlinear Independent Subspace Analysis (ISA) in the presence of auxiliary variables.
Covariate Distribution Aware Meta-learning
Based on this model we propose a computationally feasible meta-learning algorithm by introducing meaningful relaxations in our final objective.
Politeness Transfer: A Tag and Generate Approach
This paper introduces a new task of politeness transfer which involves converting non-polite sentences to polite sentences while preserving the meaning.
Better Approximate Inference for Partial Likelihood Models with a Latent Structure
Temporal Point Processes (TPP) with partial likelihoods involving a latent structure often entail an intractable marginalization, thus making inference hard.
