Search Results for author:
Found 106 papers, 46 papers with code
Model-tuning Via Prompts Makes NLP Models Adversarially Robust
In recent years, NLP practitioners have converged on the following practice: (i) import an off-the-shelf pretrained (masked) language model; (ii) append a multilayer perceptron atop the CLS token's hidden representation (with randomly initialized weights); and (iii) fine-tune the entire model on a downstream task (MLP).
Users are the North Star for AI Transparency
We conduct a broad literature survey, identifying many clusters of similar conceptions of transparency, tying each back to our north star with analysis of how it furthers or hinders our ideal AI transparency goals.
Local Causal Discovery for Estimating Causal Effects
Even when the causal graph underlying our data is unknown, we can use observational data to narrow down the possible values that an average treatment effect (ATE) can take by (1) identifying the graph up to a Markov equivalence class; and (2) estimating that ATE for each graph in the class.
RLSbench: Domain Adaptation Under Relaxed Label Shift
Meanwhile, popular deep domain adaptation heuristics tend to falter when faced with shifts in label proportions.
Meta-Learning Mini-Batch Risk Functionals
Supervised learning typically optimizes the expected value risk functional of the loss, but in many cases, we want to optimize for other risk functionals.
Disentangling the Mechanisms Behind Implicit Regularization in SGD
A number of competing hypotheses have been proposed to explain why small-batch Stochastic Gradient Descent (SGD)leads to improved generalization over the full-batch regime, with recent work crediting the implicit regularization of various quantities throughout training.
Model Evaluation in Medical Datasets Over Time
Machine learning models deployed in healthcare systems face data drawn from continually evolving environments.
Domain Adaptation under Missingness Shift
Rates of missing data often depend on record-keeping policies and thus may change across times and locations, even when the underlying features are comparatively stable.
Characterizing Datapoints via Second-Split Forgetting
Popular metrics derived from these dynamics include (i) the epoch at which examples are first correctly classified; (ii) the number of times their predictions flip during training; and (iii) whether their prediction flips if they are held out.
Neural Network Approximations of PDEs Beyond Linearity: Representational Perspective
We show that if composing a function with Barron norm $b$ with $L$ produces a function of Barron norm at most $B_L b^p$, the solution to the PDE can be $\epsilon$-approximated in the $L^2$ sense by a function with Barron norm $O\left(\left(dB_L\right)^{p^{\log(1/\epsilon)}}\right)$.
Learning under Data Drift with Time-Varying Importance Weights
We introduce a novel time-varying importance weight estimator that can detect gradual shifts in the distribution of data.
Downstream Datasets Make Surprisingly Good Pretraining Corpora
For most natural language processing tasks, the dominant practice is to finetune large pretrained transformer models (e. g., BERT) using smaller downstream datasets.
On the State of the Art in Authorship Attribution and Authorship Verification
Despite decades of research on authorship attribution (AA) and authorship verification (AV), inconsistent dataset splits/filtering and mismatched evaluation methods make it difficult to assess the state of the art.
Learning Clinical Concepts for Predicting Risk of Progression to Severe COVID-19
With COVID-19 now pervasive, identification of high-risk individuals is crucial.
Unsupervised Learning under Latent Label Shift
Thus motivated, we introduce a practical algorithm that leverages domain-discriminative models as follows: (i) push examples through domain discriminator $p(d|\mathbf{x})$; (ii) discretize the data by clustering examples in $p(d|\mathbf{x})$ space; (iii) perform non-negative matrix factorization on the discrete data; (iv) combine the recovered $p(y|d)$ with the discriminator outputs $p(d|\mathbf{x})$ to compute $p_d(y|x) \; \forall d$.
Domain Adaptation under Open Set Label Shift
We introduce the problem of domain adaptation under Open Set Label Shift (OSLS) where the label distribution can change arbitrarily and a new class may arrive during deployment, but the class-conditional distributions p(x|y) are domain-invariant.
Supervised Learning with General Risk Functionals
Standard uniform convergence results bound the generalization gap of the expected loss over a hypothesis class.
On the Maximum Hessian Eigenvalue and Generalization
The mechanisms by which certain training interventions, such as increasing learning rates and applying batch normalization, improve the generalization of deep networks remains a mystery.
Resolving the Human Subjects Status of Machine Learning's Crowdworkers
In recent years, machine learning (ML) has come to rely more heavily on crowdworkers, both for building bigger datasets and for addressing research questions requiring human interaction or judgment.
Modeling Attrition in Recommender Systems with Departing Bandits
We first address the case where all users share the same type, demonstrating that a recent UCB-based algorithm is optimal.
Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation
Previous works on Treatment Effect Estimation (TEE) are not in widespread use because they are predominantly theoretical, where strong parametric assumptions are made but untractable for practical application.
Leveraging Unlabeled Data to Predict Out-of-Distribution Performance
Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops.
Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations
Through our evaluation, we observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
Mixture Proportion Estimation and PU Learning: A Modern Approach
Formally, this task is broken down into two subtasks: (i) Mixture Proportion Estimation (MPE) -- determining the fraction of positive examples in the unlabeled data; and (ii) PU-learning -- given such an estimate, learning the desired positive-versus-negative classifier.
Practical Benefits of Feature Feedback Under Distribution Shift
In attempts to develop sample-efficient and interpretable algorithms, researcher have explored myriad mechanisms for collecting and exploiting feature feedback (or rationales) auxiliary annotations provided for training (but not test) instances that highlight salient evidence.
Does Pretraining for Summarization Require Knowledge Transfer?
Pretraining techniques leveraging enormous datasets have driven recent advances in text summarization.
Efficient Online Estimation of Causal Effects by Deciding What to Observe
Researchers often face data fusion problems, where multiple data sources are available, each capturing a distinct subset of variables.
Dive into Deep Learning
This open-source book represents our attempt to make deep learning approachable, teaching readers the concepts, the context, and the code.
Correcting Exposure Bias for Link Recommendation
Link prediction methods are frequently applied in recommender systems, e. g., to suggest citations for academic papers or friends in social networks.
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study
In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions.
RATT: Leveraging Unlabeled Data to Guarantee Generalization
To assess generalization, machine learning scientists typically either (i) bound the generalization gap and then (after training) plug in the empirical risk to obtain a bound on the true risk; or (ii) validate empirically on holdout data.
Off-Policy Risk Assessment in Contextual Bandits
Even when unable to run experiments, practitioners can evaluate prospective policies, using previously logged data.
On the Convergence and Optimality of Policy Gradient for Markov Coherent Risk
Because optimizing the coherent risk is difficult in Markov decision processes, recent work tends to focus on the Markov coherent risk (MCR), a time-consistent surrogate.
Parametric Complexity Bounds for Approximating PDEs with Neural Networks
Recent experiments have shown that deep networks can approximate solutions to high-dimensional PDEs, seemingly escaping the curse of dimensionality.
On Proximal Policy Optimization's Heavy-tailed Gradients
In this paper, we present a detailed empirical study to characterize the heavy-tailed nature of the gradients of the PPO surrogate reward function.
Evaluating Explanations: How much do explanations from the teacher aid students?
While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated.
Decoding and Diversity in Machine Translation
This makes human-level BLEU a misleading benchmark in that modern MT systems cannot approach human-level BLEU while simultaneously maintaining human-level translation diversity.
Rebounding Bandits for Modeling Satiation Effects
Psychological research shows that enjoyment of many goods is subject to satiation, with short-term satisfaction declining after repeated exposures to the same item.
Fair Machine Learning Under Partial Compliance
If k% of employers were to voluntarily adopt a fairness-promoting intervention, should we expect k% progress (in aggregate) towards the benefits of universal adoption, or will the dynamics of partial compliance wash out the hoped-for benefits?
Weakly- and Semi-supervised Evidence Extraction
For many prediction tasks, stakeholders desire not only predictions but also supporting evidence that a human can use to verify its correctness.
Unsupervised Data Augmentation with Naive Augmentation and without Unlabeled Data
Unsupervised Data Augmentation (UDA) is a semi-supervised technique that applies a consistency loss to penalize differences between a model's predictions on (a) observed (unlabeled) examples; and (b) corresponding 'noised' examples produced via data augmentation.
On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment
Modern multilingual models are trained on concatenated text from multiple languages in hopes of conferring benefits to each (positive transfer), with the most pronounced benefits accruing to low-resource languages.
Explaining The Efficacy of Counterfactually Augmented Data
In attempts to produce ML models less reliant on spurious patterns in NLP datasets, researchers have recently proposed curating counterfactually augmented data (CAD) via a human-in-the-loop process in which given some documents and their (initial) labels, humans must revise the text to make a counterfactual label applicable.
Extracting Structured Data from Physician-Patient Conversations By Predicting Noteworthy Utterances
In this exploratory study, we describe a new dataset consisting of conversation transcripts, post-visit summaries, corresponding supporting evidence (in the transcript), and structured labels.
Uncertainty-Aware Lookahead Factor Models for Quantitative Investing
On a periodic basis, publicly traded companies report fundamentals, financial data including revenue, earnings, debt, among others.
Predicting Mortality Risk in Viral and Unspecified Pneumonia to Assist Clinicians with COVID-19 ECMO Planning
Finally, the PEER score is provided in the form of a nomogram for direct calculation of patient risk, and can be used to highlight at-risk patients among critical care patients eligible for ECMO.
Generating SOAP Notes from Doctor-Patient Conversations Using Modular Summarization Techniques
Following each patient visit, physicians draft long semi-structured clinical summaries called SOAP notes.
Estimating Treatment Effects with Observed Confounders and Mediators
We show that it strictly outperforms the backdoor and frontdoor estimators and that this improvement can be unbounded.
A Unified View of Label Shift Estimation
Our contributions include (i) consistency conditions for MLLS, which include calibration of the classifier and a confusion matrix invertibility condition that BBSE also requires; (ii) a unified framework, casting BBSE as roughly equivalent to MLLS for a particular choice of calibration method; and (iii) a decomposition of MLLS's finite-sample error into terms reflecting miscalibration and estimation error.
Causal Inference With Selectively Deconfounded Data
In this paper, we consider the benefit of incorporating a large confounded observational dataset (confounder unobserved) alongside a small deconfounded observational dataset (confounder revealed) when estimating the ATE.
Algorithmic Fairness from a Non-ideal Perspective
Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions.
Smooth Kernels Improve Adversarial Robustness and Perceptually-Aligned Gradients
Recent research has shown that CNNs are often overly sensitive to high-frequency textural patterns.
Game Design for Eliciting Distinguishable Behavior
The ability to inferring latent psychological traits from human behavior is key to developing personalized human-interacting machine learning systems.
Are Perceptually-Aligned Gradients a General Property of Robust Classifiers?
For a standard convolutional neural network, optimizing over the input pixels to maximize the score of some target class will generally produce a grainy-looking version of the original image.
Accelerating Deep Learning by Focusing on the Biggest Losers
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration.
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e. g., mentions of genre), models trained on the combined data are less sensitive to this signal.
Learning to Deceive with Attention-Based Explanations
Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing.
Entity Projection via Machine Translation for Cross-Lingual NER
Although over 100 languages are supported by strong off-the-shelf machine translation systems, only a subset of them possess large annotated corpora for named entity recognition.
AmazonQA: A Review-Based Question Answering Task
Observing that many questions can be answered based upon the available product reviews, we propose the task of review-based QA.
Estimating brain age based on a healthy population with deep learning and structural MRI
Numerous studies have established that estimated brain age, as derived from statistical models trained on healthy populations, constitutes a valuable biomarker that is predictive of cognitive decline and various neurological diseases.
Learning Causal State Representations of Partially Observable Environments
In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP).
Learning Robust Global Representations by Penalizing Local Predictive Power
Despite their renowned predictive power on i. i. d.
Ranked #90 on
Domain Generalization
on PACS
Efficient candidate screening under multiple tests and implications for fairness
We analyze the optimal employer policy both when the employer sets a fixed number of tests per candidate and when the employer can set a dynamic policy, assigning further tests adaptively based on results from the previous tests.
Combating Adversarial Misspellings with Robust Word Recognition
To combat adversarial spelling mistakes, we propose placing a word recognition model in front of the downstream classifier.
Embryo staging with weakly-supervised region selection and dynamically-decoded predictions
First, noting that in each image the embryo occupies a small subregion, we jointly train a region proposal network with the downstream classifier to isolate the embryo.
Learning Robust Representations by Projecting Superficial Statistics Out
We test our method on the battery of standard domain generalization data sets and, interestingly, achieve comparable or better performance as compared to other domain generalization methods that explicitly require samples from the target distribution for training.
Ranked #99 on
Domain Generalization
on PACS
What is the Effect of Importance Weighting in Deep Learning?
Importance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning.
Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
We might hope that when faced with unexpected inputs, well-designed software systems would fire off warnings.
Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Study
This paper provides a large scale empirical study of deep active learning, addressing multiple tasks and, for each, multiple datasets, multiple models, and a full suite of acquisition functions.
How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
Many recent papers address reading comprehension, where examples consist of (question, passage, answer) tuples.
Learning Noise-Invariant Representations for Robust Speech Recognition
Despite rapid advances in speech recognition, current models remain brittle to superficial perturbations to their inputs.
Practical Obstacles to Deploying Active Learning
Active learning (AL) is a widely-used training strategy for maximizing predictive performance subject to a fixed annotation budget.
Troubling Trends in Machine Learning Scholarship
Collectively, machine learning (ML) researchers are engaged in the creation and dissemination of knowledge about data-driven algorithms.
Surprising Negative Results for Generative Adversarial Tree Search
We deploy this model and propose generative adversarial tree search (GATS) a deep RL algorithm that learns the environment model and implements Monte Carlo tree search (MCTS) on the learned model for planning.
Born Again Neural Networks
Knowledge distillation (KD) consists of transferring knowledge from one machine learning model (the teacher}) to another (the student).
Correction by Projection: Denoising Images with Generative Adversarial Networks
In this paper, we propose to denoise corrupted images by finding the nearest point on the GAN manifold, recovering latent vectors by minimizing distances in image space.
Stochastic Activation Pruning for Robust Adversarial Defense
Neural networks are known to be vulnerable to adversarial examples.
Active Learning with Partial Feedback
While many active learning papers assume that the learner can simply ask for a label and receive it, real annotation often presents a mismatch between the form of a label (say, one among many classes), and the form of an annotation (typically yes/no binary feedback).
Detecting and Correcting for Label Shift with Black Box Predictors
Faced with distribution shift between training and test set, we wish to detect and quantify the shift, and to correct our classifiers without test set labels.
Avoiding Catastrophic States with Intrinsic Fear
Many practical reinforcement learning problems contain catastrophic states that the optimal policy visits infrequently or never.
Learning From Noisy Singly-labeled Data
We propose a new algorithm for jointly modeling labels and worker quality from noisy crowd-sourced data.
The Doctor Just Won't Accept That!
For the field of interpretable machine learning to advance, we must ask the following questions: What precisely won't various stakeholders accept?
Does mitigating ML's impact disparity require treatment disparity?
Following related work in law and policy, two notions of disparity have come to shape the study of fairness in algorithmic decision-making.
Improving Factor-Based Quantitative Investing by Forecasting Company Fundamentals
Academic research has identified some factors, i. e. computed features of the reported data, that are known through retrospective analysis to outperform the market average.
Estimating Reactions and Recommending Products with Generative Models of Reviews
Natural language approaches that model information like product reviews have proved to be incredibly useful in improving the performance of such methods, as reviews provide valuable auxiliary information that can be used to better estimate latent user preferences and item properties.
Tensor Regression Networks
First, we introduce Tensor Contraction Layers (TCLs) that reduce the dimensionality of their input while preserving their multilinear structure using tensor contraction.
Deep Active Learning for Named Entity Recognition
In this work, we demonstrate that the amount of labeled training data can be drastically reduced when deep learning is combined with active learning.
Tensor Contraction Layers for Parsimonious Deep Nets
Specifically, we propose the Tensor Contraction Layer (TCL), the first attempt to incorporate tensor contractions as end-to-end trainable neural network layers.
Semantically Decomposing the Latent Spaces of Generative Adversarial Networks
Corresponding samples from the real dataset consist of two distinct photographs of the same subject.
Dance Dance Convolution
For the step placement task, we combine recurrent and convolutional neural networks to ingest spectrograms of low-level audio features to predict steps, conditioned on chart difficulty.
Predicting Surgery Duration with Neural Heteroscedastic Regression
Scheduling surgeries is a challenging task due to the fundamental uncertainty of the clinical environment, as well as the risks and costs associated with under- and over-booking.
Precise Recovery of Latent Vectors from Generative Adversarial Networks
Generative adversarial networks (GANs) transform latent vectors into visually plausible images.
A User Simulator for Task-Completion Dialogues
Then, one can train reinforcement learning agents in an online fashion as they interact with the simulator.
Combating Reinforcement Learning's Sisyphean Curse with Intrinsic Fear
We introduce intrinsic fear (IF), a learned reward shaping that guards DRL agents against periodic catastrophes.
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems.
Context Matters: Refining Object Detection in Video with Recurrent Neural Networks
Then we train a recurrent neural network that takes as input sequences of pseudo-labeled frames and optimizes an objective that encourages both accuracy on the target frame and consistency across consecutive frames.
Modeling Missing Data in Clinical Time Series with RNNs
For linear models, we show an alternative strategy to capture this signal.
The Mythos of Model Interpretability
First, we examine the motivations underlying interest in interpretability, finding them to be diverse and occasionally discordant.
Stuck in a What? Adventures in Weight Space
As neural networks are typically over-complete, it's easy to show the existence of vast continuous regions through weight space with equal loss.
Learning to Diagnose with LSTM Recurrent Neural Networks
We present the first study to empirically evaluate the ability of LSTMs to recognize patterns in multivariate time series of clinical measurements.
Generative Concatenative Nets Jointly Learn to Write and Classify Reviews
A recommender system's basic task is to estimate how users will respond to unseen items.
Phenotyping of Clinical Time Series with LSTM Recurrent Neural Networks
We present a novel application of LSTM recurrent neural networks to multilabel classification of diagnoses given variable-length time series of clinical measurements.
A Critical Review of Recurrent Neural Networks for Sequence Learning
Recurrent neural networks (RNNs) are connectionist models that capture the dynamics of sequences via cycles in the network of nodes.
Efficient Elastic Net Regularization for Sparse Linear Models
This paper provides closed-form updates for the popular squared norm $\ell^2_2$ and elastic net regularizers.
Differential Privacy and Machine Learning: a Survey and Review
The objective of machine learning is to extract useful information from data, while privacy is preserved by concealing information.
