Search Results for author:
Found 90 papers, 28 papers with code
On Writing a Textbook on Natural Language Processing
There are thousands of papers about natural language processing and computational linguistics, but very few textbooks.
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment
In this work, we evaluate a simple approach for zero-shot cross-lingual alignment, where a reward model is trained on preference data in one source language and directly applied to other target languages.
Transforming and Combining Rewards for Aligning Large Language Models
A common approach for aligning language models to human preferences is to first learn a reward model from preference data, and then use this reward model to update the language model.
Theoretical guarantees on the best-of-n alignment policy
A commonly used analytical expression in the literature claims that the KL divergence between the best-of-$n$ policy and the base policy is equal to $\log (n) - (n-1)/n.$ We disprove the validity of this claim, and show that it is an upper bound on the actual KL divergence.
Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
However, even pretrain reward ensembles do not eliminate reward hacking: we show several qualitative reward hacking phenomena that are not mitigated by ensembling because all reward models in the ensemble exhibit similar error patterns.
Selectively Answering Ambiguous Questions
We investigate question answering from this perspective, focusing on answering a subset of questions with a high degree of accuracy, from a set of questions in which many are inherently ambiguous.
MD3: The Multi-Dialect Dataset of Dialogues
We introduce a new dataset of conversational speech representing English from India, Nigeria, and the United States.
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development.
Dialect-robust Evaluation of Generated Text
Evaluation metrics that are not robust to dialect variation make it impossible to tell how well systems perform for many groups of users, and can even penalize systems for producing text in lower-resource dialects.
Predicting Long-Term Citations from Short-Term Linguistic Influence
A standard measure of the influence of a research paper is the number of times it is cited.
Pre-trained Sentence Embeddings for Implicit Discourse Relation Classification
Implicit discourse relations bind smaller linguistic units into coherent texts.
Classification
Implicit Discourse Relation Classification
+4
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model
But what sorts of rationales are useful and how can we train systems to produce them?
Informativeness and Invariance: Two Perspectives on Spurious Correlations in Natural Language
Spurious correlations are a threat to the trustworthiness of natural language processing systems, motivating research into methods for identifying and eliminating them.
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
A fundamental goal of scientific research is to learn about causal relationships.
Learning to Look Inside: Augmenting Token-Based Encoders with Character-Level Information
Commonly-used transformer language models depend on a tokenization schema which sets an unchangeable subword vocabulary prior to pre-training, destined to be applied to all downstream tasks regardless of domain shift, novel word formations, or other sources of vocabulary mismatch.
The MultiBERTs: BERT Reproductions for Robustness Analysis
Experiments with pre-trained models such as BERT are often based on a single checkpoint.
Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer
Zero-shot cross-lingual transfer is emerging as a practical solution: pre-trained models later fine-tuned on one transfer language exhibit surprising performance when tested on many target languages.
Time-Aware Language Models as Temporal Knowledge Bases
We introduce a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time and highlight problems with LMs at either end of the spectrum -- those trained on specific slices of temporal data, as well as those trained on a wide range of temporal data.
Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Tests
We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions.
Counterfactual Invariance to Spurious Correlations in Text Classification
We introduce counterfactual invariance as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions.
Abolitionist Networks: Modeling Language Change in Nineteenth-Century Activist Newspapers
This paper supplements recent qualitative work on the role of women in abolition's vanguard, as well as the role of the Black press, with a quantitative text modeling approach.
Tuiteamos o pongamos un tuit? Investigating the Social Constraints of Loanword Integration in Spanish Social Media
In social media, we find that speaker background and expectations of formality explain loanword and native word integration, such that authors who use more Spanish and who write to a wider audience tend to use integrated verb forms more often.
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains.
Learning to Recognize Dialect Features
Evaluation on a test set of 22 dialect features of Indian English demonstrates that these models learn to recognize many features with high accuracy, and that a few minimal pairs can be as effective for training as thousands of labeled examples.
Will it Unblend?
Natural language processing systems often struggle with out-of-vocabulary (OOV) terms, which do not appear in training data.
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation
In this paper, we propose a new adversarial augmentation method for Neural Machine Translation (NMT).
Ranked #22 on
Machine Translation
on WMT2014 English-German
Sparse, Dense, and Attentional Representations for Text Retrieval
Dual encoders perform retrieval by encoding documents and queries into dense lowdimensional vectors, scoring each document by its inner product with the query.
Characterizing Collective Attention via Descriptor Context: A Case Study of Public Discussions of Crisis Events
But according to rationalist models of natural language communication, the collective salience of each entity will be expressed not only in how often it is mentioned, but in the form that those mentions take.
Follow the Leader: Documents on the Leading Edge of Semantic Change Get More Citations
However, simply knowing that a word has changed in meaning is insufficient to identify the instances of word usage that convey the historical or the newer meaning.
How we do things with words: Analyzing text as social and cultural data
Second, we hope to provide a set of best practices for working with thick social and cultural concepts.
Clinical Concept Extraction for Document-Level Coding
The text of clinical notes can be a valuable source of patient information and clinical assessments.
Measuring and Modeling Language Change
Such questions are fundamental to the social sciences and humanities, and scholars in these disciplines are increasingly turning to computational techniques for answers.
Correcting Whitespace Errors in Digitized Historical Texts
Whitespace errors are common to digitized archives.
Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling
To address this scenario, we propose domain-adaptive fine-tuning, in which the contextualized embeddings are adapted by masked language modeling on text from the target domain.
Character Eyes: Seeing Language through Character-Level Taggers
Character-level models have been used extensively in recent years in NLP tasks as both supplements and replacements for closed-vocabulary token-level word representations.
Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation
We consider the problem of making machine translation more robust to character-level variation at the source side, such as typos.
The Referential Reader: A Recurrent Entity Network for Anaphora Resolution
We present a new architecture for storing and accessing entity mentions during online text processing.
Interactional Stancetaking in Online Forums
Theories of interactional stancetaking have been put forward as holistic accounts, but until now, these theories have been applied only through detailed qualitative analysis of (portions of) a few individual conversations.
Making ``fetch'' happen: The influence of social and linguistic context on nonstandard word growth and decline
In an online community, new words come and go: today{'}s {``}haha{''} may be replaced by tomorrow{'}s {``}lol.
Mind Your POV: Convergence of Articles and Editors Towards Wikipedia's Neutrality Norm
Do NPOV corrections encourage editors to adopt this style?
Predicting Semantic Relations using Global Graph Properties
Semantic graphs, such as WordNet, are resources which curate natural language on two distinguishable layers.
Ranked #14 on
Link Prediction
on WN18RR
Si O No, Que Penses? Catalonian Independence and Linguistic Identity on Social Media
We also find that Catalan is used more often in referendum-related discourse than in other contexts, contrary to prior findings on language variation.
Stylistic Variation in Social Media Part-of-Speech Tagging
While prior work found that similar approaches yield performance improvements in sentiment analysis and entity linking, we were unable to obtain performance improvements in part-of-speech tagging, despite strong evidence for the link between part-of-speech error rates and social network structure.
Sí o no, què penses? Catalonian Independence and Linguistic Identity on Social Media
We also find that Catalan is used more often in referendum-related discourse than in other contexts, contrary to prior findings on language variation.
Detecting Social Influence in Event Cascades by Comparing Discriminative Rankers
However, detecting social influence from observational data is challenging due to confounds like homophily and practical issues like missing data.
Explainable Prediction of Medical Codes from Clinical Text
Our method aggregates information across the document using a convolutional neural network, and uses an attention mechanism to select the most relevant segments for each of the thousands of possible codes.
Ranked #10 on
Medical Code Prediction
on MIMIC-III
Making "fetch" happen: The influence of social and linguistic context on nonstandard word growth and decline
In an online community, new words come and go: today's "haha" may be replaced by tomorrow's "lol."
#anorexia, #anarexia, #anarexyia: Characterizing Online Community Practices with Orthographic Variation
We also demonstrate the utility of orthographic variation as a new lens to study sociolinguistic change in online communities, particularly when the change results from an exogenous force such as a content ban.
Making "fetch" happen: The influence of social and linguistic context on nonstandard word growth and decline
In an online community, new words come and go: today's "haha" may be replaced by tomorrow's "lol."
Mimicking Word Embeddings using Subword RNNs
In this paper, we present MIMICK, an approach to generating OOV word embeddings compositionally, by learning a function from spellings to distributional embeddings.
A Multidimensional Lexicon for Interpersonal Stancetaking
The sociolinguistic construct of stancetaking describes the activities through which discourse participants create and signal relationships to their interlocutors, to the topic of discussion, and to the talk itself.
Unsupervised Learning for Lexicon-Based Classification
In lexicon-based classification, documents are assigned labels by comparing the number of words that appear from two opposed lexicons, such as positive and negative sentiment.
Toward Socially-Infused Information Extraction: Embedding Authors, Mentions, and Entities
Entity linking is the task of identifying mentions of entities in text, and linking them to entries in a knowledge base.
The Social Dynamics of Language Change in Online Networks
Language change is a complex social phenomenon, revealing pathways of communication and sociocultural influence.
Morphological Priors for Probabilistic Neural Word Embeddings
Word embeddings allow natural language processing systems to share statistical information across related words.
Shallow Discourse Parsing Using Distributed Argument Representations and Bayesian Optimization
This paper describes the Georgia Tech team's approach to the CoNLL-2016 supplementary evaluation on discourse relation sense classification.
Part-of-Speech Tagging for Historical English
We evaluate several domain adaptation methods on the task of tagging Early Modern English and Modern British English texts in the Penn Corpora of Historical English.
A Latent Variable Recurrent Neural Network for Discourse Relation Language Models
This paper presents a novel latent variable recurrent neural network architecture for jointly modeling sequences of words and (possibly latent) discourse relations between adjacent sentences.
A Kernel Independence Test for Geographical Language Variation
Quantifying the degree of spatial dependence for linguistic variables is a key task for analyzing dialectal variation.
Nonparametric Bayesian Storyline Detection from Microtexts
News events and social media are composed of evolving storylines, which capture public attention for a limited period of time.
Overcoming Language Variation in Sentiment Analysis with Social Attention
Variation in language is ubiquitous, particularly in newer forms of writing such as social media.
Document Context Language Models
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure.
Emoticons vs. Emojis on Twitter: A Causal Inference Approach
Online writing lacks the non-verbal cues present in face-to-face communication, which provide additional contextual information about the utterance, such as the speaker's intention or affective state.
Better Document-level Sentiment Analysis from RST Discourse Parsing
Discourse structure is the hidden link between surface features and document-level properties, such as sentiment polarity.
Confounds and Consequences in Geotagged Twitter Data
Twitter is often used in quantitative studies that identify geographically-preferred topics, writing styles, and entities.
One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations
A more subtle challenge is that it is not enough to represent the meaning of each argument of a discourse relation, because the relation may depend on links between lowerlevel components, such as entity mentions.
Entity-Augmented Distributional Semantics for Discourse Relations
A more subtle challenge is that it is not enough to represent the meaning of each sentence of a discourse relation, because the relation may depend on links between lower-level elements, such as entity mentions.
Unsupervised Domain Adaptation with Feature Embeddings
Representation learning is the dominant technique for unsupervised domain adaptation, but existing approaches often require the specification of "pivot features" that generalize across domains, which are selected by task-specific heuristics.
One Vector is Not Enough: Entity-Augmented Distributional Semantics for Discourse Relations
Discourse relations bind smaller linguistic units into coherent texts.
Learning Document-Level Semantic Properties from Free-Text Annotations
The paraphrase structure is linked with a latent topic model of the review texts, enabling the system to predict the properties of unannotated documents and to effectively aggregate the semantic properties of multiple reviews.
Multilingual Part-of-Speech Tagging: Two Unsupervised Approaches
We demonstrate the effectiveness of multilingual learning for unsupervised part-of-speech tagging.
Diffusion of Lexical Change in Social Media
Computer-mediated communication is driving fundamental changes in the nature of written language.
Gender identity and lexical variation in social media
Examining individuals whose language does not match the classifier's model for their gender, we find that they have social networks that include significantly fewer same-gender social connections and that, in general, social network homophily is correlated with the use of same-gender language markers.
