Search Results for author:
Found 65 papers, 36 papers with code
Robustness and Adversarial Examples in Natural Language Processing
In particular, we will review recent studies on analyzing the weakness of NLP systems when facing adversarial inputs and data with a distribution shift.
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs).
Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks World
Language agents that interact with the world on their own have great potential for automating digital tasks.
Parallel Structures in Pre-training Data Yield In-Context Learning
Pre-trained language models (LMs) are capable of in-context learning (ICL): they can adapt to a task with only a few examples given in the prompt without any parameter update.
Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning
We propose explanation-consistency finetuning (EC-finetuning), a method that adapts LLMs to generate more consistent natural-language explanations on related examples.
Pragmatic Radiology Report Generation
When pneumonia is not found on a chest X-ray, should the report describe this negative observation or omit it?
Show Your Work with Confidence: Confidence Bands for Tuning Curves
We present the first method to construct valid confidence bands for tuning curves.
Personas as a Way to Model Truthfulness in Language Models
This allows the model to separate truth from falsehoods and controls the truthfulness of its generation.
Does Writing with Language Models Reduce Content Diversity?
We develop a set of diversity metrics and find that writing with InstructGPT (but not the GPT3) results in a statistically significant reduction in diversity.
Leveraging Implicit Feedback from Deployment Data in Dialogue
We study improving social conversational agents by learning from natural dialogue between users and a deployed model, without extra annotations.
Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations
To answer these questions, we propose to evaluate $\textbf{counterfactual simulatability}$ of natural language explanations: whether an explanation can enable humans to precisely infer the model's outputs on diverse counterfactuals of the explained input.
Efficient Shapley Values Estimation by Amortization for Text Classification
In practice, Shapley Values are often estimated with a small number of stochastic model evaluations.
Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples
Given the intractably large size of the space of proofs, any model that is capable of general deductive reasoning must generalize to proofs of greater complexity.
Measuring Inductive Biases of In-Context Learning with Underspecified Demonstrations
We investigate the inductive biases of ICL from the perspective of feature bias: which feature ICL is more likely to use given a set of underspecified demonstrations in which two features are equally predictive of the labels.
How do decoding algorithms distribute information in dialogue responses?
We find that (i) surprisingly, model-generated responses follow the UID principle to a greater extent than human responses, and (ii) decoding algorithms that promote UID do not generate higher-quality responses.
Extrapolative Controlled Sequence Generation via Iterative Refinement
We study the problem of extrapolative controlled generation, i. e., generating sequences with attribute values beyond the range seen in training.
Reward Gaming in Conditional Text Generation
To align conditional text generation model outputs with desired behaviors, there has been an increasing focus on training the model using reinforcement learning (RL) with reward functions learned from human annotations.
Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing
The core component of our system is a language model fine-tuned on a diverse collection of instructions for poetry writing.
Are All Spurious Features in Natural Language Alike? An Analysis through a Causal Lens
In case (i), we want the model to be invariant to the feature, which is neither necessary nor sufficient for prediction.
Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining
To benchmark the performance of pretrained multilingual language models, we construct noisy datasets covering five languages and four NLP tasks and observe a clear gap in the performance between clean and noisy data in the zero-shot cross-lingual setting.
Data Augmentation
Pretrained Multilingual Language Models
+1
Nuisances via Negativa: Adjusting for Spurious Correlations via Data Augmentation
In prediction tasks, there exist features that are related to the label in the same way across different settings for that task; these are semantic features or semantics.
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
Large language models (LLMs) have shown remarkable reasoning capabilities given chain-of-thought prompts (examples with intermediate reasoning steps).
On the Relation between Sensitivity and Accuracy in In-context Learning
In-context learning (ICL) suffers from oversensitivity to the prompt, making it unreliable in real-world scenarios.
Exploring the Role of Task Transferability in Large-Scale Multi-Task Learning
Recent work has found that multi-task training with a large number of diverse tasks can uniformly improve downstream performance on unseen target tasks.
Amortized Noisy Channel Neural Machine Translation
For all three approaches, the generated translations fail to achieve rewards comparable to BSR, but the translation quality approximated by BLEU and BLEURT is similar to the quality of BSR-produced translations.
QuALITY: Question Answering with Long Input Texts, Yes!
To enable building and testing models on long-document comprehension, we introduce QuALITY, a multiple-choice QA dataset with context passages in English that have an average length of about 5, 000 tokens, much longer than typical current models can process.
IRM---when it works and when it doesn't: A test case of natural language inference
Invariant Risk Minimization (IRM) is a recently proposed framework for out-of-distribution (o. o. d) generalization.
Machine-in-the-Loop Rewriting for Creative Image Captioning
Machine-in-the-loop writing aims to enable humans to collaborate with models to complete their writing tasks more effectively.
Meta-learning via Language Model In-context Tuning
The goal of meta-learning is to learn to adapt to a new task with only a few labeled examples.
SeqPATE: Differentially Private Text Generation via Knowledge Distillation
Differentially private (DP) learning algorithms provide guarantees on identifying the existence of a training sample from model outputs.
Types of Out-of-Distribution Texts and How to Detect Them
Despite agreement on the importance of detecting out-of-distribution (OOD) examples, there is little consensus on the formal definition of OOD examples and how to best detect them.
Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization
Despite recent progress in abstractive summarization, systems still suffer from faithfulness errors.
An Investigation of the (In)effectiveness of Counterfactually Augmented Data
While pretrained language models achieve excellent performance on natural language understanding benchmarks, they tend to rely on spurious correlations and generalize poorly to out-of-distribution (OOD) data.
Unsupervised Extractive Summarization using Pointwise Mutual Information
Unsupervised approaches to extractive summarization usually rely on a notion of sentence importance defined by the semantic similarity between a sentence and the document.
Text Generation by Learning from Demonstrations
Current approaches to text generation largely rely on autoregressive models and maximum likelihood estimation.
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Recent work has shown that pre-trained language models such as BERT improve robustness to spurious correlations in the dataset.
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization
We tackle the problem of evaluating faithfulness of a generated summary given its source document.
Different Set Domain Adaptation for Brain-Computer Interfaces: A Label Alignment Approach
Currently, most domain adaptation approaches require the source domains to have the same feature space and label space as the target domain, which limits their applications, as the auxiliary data may have different feature spaces and/or different label spaces.
A Dynamic Strategy Coach for Effective Negotiation
We consider a bargaining scenario where a seller and a buyer negotiate the price of an item for sale through a text-based dialog.
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
We first learn a biased model that only uses features that are known to relate to dataset bias.
Multi-View Broad Learning System for Primate Oculomotor Decision Decoding
Multi-view learning improves the learning performance by utilizing multi-view data: data collected from multiple sources, or feature sets extracted from the same data source.
GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing
We present GluonCV and GluonNLP, the deep learning toolkits for computer vision and natural language processing based on Apache MXNet (incubating).
Pun Generation with Surprise
We tackle the problem of generating a pun sentence given a pair of homophones (e. g., "died" and "dyed").
Quizbowl: The Case for Incremental Question Answering
Throughout this paper, we show that collaborations with the vibrant trivia community have contributed to the quality of our dataset, spawned new research directions, and doubled as an exciting way to engage the public with research in machine learning and natural language processing.
QuAC: Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
Decoupling Strategy and Generation in Negotiation Dialogues
We consider negotiation settings in which two agents use natural language to bargain on goods.
QuAC : Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
Transfer Learning for Brain-Computer Interfaces: A Euclidean Space Data Alignment Approach
Our approach has three desirable properties: 1) it aligns the EEG trials directly in the Euclidean space, and any signal processing, feature extraction and machine learning algorithms can then be applied to the aligned trials; 2) its computational cost is very low; and, 3) it is unsupervised and does not need any label information from the new subject.
Transfer Learning Enhanced Common Spatial Pattern Filtering for Brain Computer Interfaces (BCIs): Overview and a New Approach
The electroencephalogram (EEG) is the most widely used input for brain computer interfaces (BCIs), and common spatial pattern (CSP) is frequently used to spatially filter it to increase its signal-to-noise ratio.
Spatial Filtering for Brain Computer Interfaces: A Comparison between the Common Spatial Pattern and Its Variant
The electroencephalogram (EEG) is the most popular form of input for brain computer interfaces (BCIs).
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context
We know very little about how neural language models (LM) use prior linguistic context.
Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
We consider the task of text attribute transfer: transforming a sentence to alter a specific attribute (e. g., sentiment) while preserving its attribute-independent content (e. g., changing "screen is just the right size" to "screen is too small").
Ranked #1 on
Unsupervised Text Style Transfer
on Yelp2018
Learning Symmetric Collaborative Dialogue Agents with Dynamic Knowledge Graph Embeddings
To model both structured knowledge and unstructured language, we propose a neural model with dynamic knowledge graph embeddings that evolve as the dialogue progresses.
Opponent Modeling in Deep Reinforcement Learning
Opponent modeling is necessary in multi-agent settings where secondary agents with competing goals also adapt their strategies, yet it remains challenging because strategies interact with each other and change.
Active Information Acquisition
We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task.
General Reinforcement Learning
Reinforcement Learning (RL)
+1
Learning to Search for Dependencies
We demonstrate that a dependency parser can be built using a credit assignment compiler which removes the burden of worrying about low-level machine learning details from the parser implementation.
Learning to Search in Branch and Bound Algorithms
Branch-and-bound is a widely used method in combinatorial optimization, including mixed integer programming, structured prediction and MAP inference.
A Credit Assignment Compiler for Joint Prediction
Many machine learning applications involve jointly predicting multiple mutually dependent output variables.
Imitation Learning by Coaching
However, it is important to note that these guarantees depend on how well the policy we found can imitate the oracle on the training data.
