Search Results for author:
Found 86 papers, 58 papers with code
Where Are We in Discourse Relation Recognition?
Discourse parsers recognize the intentional and inferential relationships that organize extended texts.
The Role of Context and Uncertainty in Shallow Discourse Parsing
We find that including annotator accuracy and confidence improves model accuracy, and incorporating confidence in the model’s temperature function can lead to models with significantly better-calibrated confidence measures.
Political Ideology and Polarization: A Multi-dimensional Approach
Analyzing ideology and polarization is of critical importance in advancing our grasp of modern politics.
longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attacks.
Developing methods to adversarially challenge NLP systems is a promising avenue for improving both model performance and interpretability.
Behavioral Analysis of Information Salience in Large Language Models
Large Language Models (LLMs) excel at text summarization, a task that requires models to select content based on its importance.
QuaLLM-Health: An Adaptation of an LLM-Based Framework for Quantitative Data Extraction from Online Health Discussions
In this work, we present an adapted framework from QuaLLM into QuaLLM-Health for extracting clinically relevant quantitative data from Reddit discussions about glucagon-like peptide-1 (GLP-1) receptor agonists using large language models (LLMs).
Learning to Refine with Fine-Grained Natural Language Feedback
Recent work has explored the capability of large language models (LLMs) to identify and correct errors in LLM-generated responses.
Detection and Measurement of Syntactic Templates in Generated Text
Recent work on evaluating the diversity of text generated by LLMs has focused on word-level features.
Do they mean 'us'? Interpreting Referring Expressions in Intergroup Bias
We curate a unique dataset of over 6 million game-time comments from opposing perspectives (the teams in the game), each comment grounded in a non-linguistic description of the events that precipitated these comments (live win probabilities for each team).
Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning CodeLLMs
In this work, we introduce ROBO-INSTRUCT that preserves the diversity of programs generated by an LLM while providing the correctness of simulator-based checking.
Which questions should I answer? Salience Prediction of Inquisitive Questions
QSALIENCE is instruction-tuned over our dataset of linguist-annotated salience scores of 1, 766 (context, question) pairs.
Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided
Large language models (LLMs) have offered new opportunities for emotional support, and recent work has shown that they can produce empathic responses to people in distress.
Large Language Models Produce Responses Perceived to be Empathic
Large Language Models (LLMs) have demonstrated surprising performance on many tasks, including writing supportive messages that display empathy.
FactPICO: Factuality Evaluation for Plain Language Summarization of Medical Evidence
But how factual are these summaries in a high-stakes domain like medicine?
InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification
Text simplification aims to make technical texts more accessible to laypeople but often results in deletion of information and vagueness.
Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion
Situations and events evoke emotions in humans, but to what extent do they inform the prediction of emotion detection models?
Wrapper Boxes: Faithful Attribution of Model Predictions to Training Data
Our pipeline thus preserves the predictive performance of neural language models while faithfully attributing classic model decisions to training data.
QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsing
Questions Under Discussion (QUD) is a versatile linguistic framework in which discourse progresses as continuously asking questions and answering them.
Evaluating Subjective Cognitive Appraisals of Emotions from Large Language Models
The emotions we experience involve complex processes; besides physiological aspects, research in psychology has studied cognitive appraisals where people assess their situations subjectively, according to their own values (Scherer, 2005).
Sarcasm Detection in a Disaster Context
This contempt is in some cases expressed as sarcasm or irony.
Multilingual Code Co-Evolution Using Large Language Models
In this paper, we target a novel task: translating code changes from one programming language to another using large language models (LLMs).
Unsupervised Extractive Summarization of Emotion Triggers
Second, we develop new unsupervised learning models that can jointly detect emotions and summarize their triggers.
Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias
While existing work on studying bias in NLP focues on negative or pejorative language use, Govindarajan et al. (2023) offer a revised framing of bias in terms of intergroup social context, and its effects on language behavior.
Using Natural Language Explanations to Rescale Human Judgments
These scores should reflect the annotators' underlying assessments of the example.
Multilingual Simplification of Medical Texts
This work addresses this limitation via multilingual simplification, i. e., directly simplifying complex texts into simplified texts in multiple languages.
Elaborative Simplification as Implicit Questions Under Discussion
Automated text simplification, a technique useful for making text more accessible to people such as children and emergent bilinguals, is often thought of as a monolingual translation task from complex sentences to simplified sentences using encoder-decoder models.
Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)
Large language models, particularly GPT-3, are able to produce high quality summaries of general domain news articles in few- and zero-shot settings.
Learning Deep Semantics for Test Completion
We formalize the novel task of test completion to automatically complete the next statement in a test method based on the context of prior statements and the code under test.
Using Developer Discussions to Guide Fixing Bugs in Software
Automatically fixing software bugs is a challenging task.
Why Do You Feel This Way? Summarizing Triggers of Emotions in Social Media Posts
This paper takes a novel angle, namely, emotion detection and trigger summarization, aiming to both detect perceived emotions in text, and summarize events and their appraisals that trigger each emotion.
Discourse Analysis via Questions and Answers: Parsing Dependency Structures of Questions Under Discussion
Human evaluation results show that QUD dependency parsing is possible for language models trained with this crowdsourced, generalizable annotation scheme.
News Summarization and Evaluation in the Era of GPT-3
Finally, we evaluate models on a setting beyond generic summarization, specifically keyword-based summarization, and show how dominant fine-tuning approaches compare to prompting.
How people talk about each other: Modeling Generalized Intergroup Bias and Emotion
Current studies of bias in NLP rely mainly on identifying (unwanted or negative) bias towards a specific demographic group.
Text Simplification of College Admissions Instructions: A Professionally Simplified and Verified Corpus
We present PSAT (Professionally Simplified Admissions Texts), a dataset with 112 admissions instructions randomly selected from higher education institutions across the US.
CoditT5: Pretraining for Source Code and Natural Language Editing
Pretrained language models have been shown to be effective in many software-related generation tasks; however, they are not well-suited for editing tasks as they are not designed to reason about edits.
longhorns at DADC 2022: How many linguists does it take to fool a Question Answering model? A systematic approach to adversarial attacks
Developing methods to adversarially challenge NLP systems is a promising avenue for improving both model performance and interpretability.
SNaC: Coherence Error Detection for Narrative Summarization
In this work, we introduce SNaC, a narrative coherence evaluation framework rooted in fine-grained annotations for long summaries.
Evaluating Factuality in Text Simplification
We find that errors often appear in both that are not captured by existing evaluation metrics, motivating a need for research into ensuring the factual accuracy of automated simplification models.
ProtoTEx: Explaining Model Decisions with Prototype Tensors
We present ProtoTEx, a novel white-box NLP classification architecture based on prototype networks.
How Do We Answer Complex Questions: Discourse Structure of Long-form Answers
Long-form answers, consisting of multiple sentences, can provide nuanced and comprehensive answers to a broader set of questions.
Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections
While there has been substantial progress in text comprehension through simple factoid question answering, more holistic comprehension of a discourse still presents a major challenge (Dunietz et al., 2020).
Training Dynamics for Text Summarization Models
Across different datasets (CNN/DM, XSum, MediaSum) and summary properties, such as abstractiveness and hallucination, we study what the model learns at different stages of its fine-tuning process.
Learning to Describe Solutions for Bug Reports Based on Developer Discussions
When a software bug is reported, developers engage in a discussion to collaboratively resolve it.
AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses
This is in stark contrast to recent probing studies on pre-trained representation learning models, which show that rich linguistic features such as parts-of-speech and morphology are encoded by them.
The Case for Claim Difficulty Assessment in Automatic Fact Checking
Fact-checking is the process of evaluating the veracity of claims (i. e., purported facts).
Impact of Evaluation Methodologies on Code Summarization
This may lead to evaluations that are inconsistent with the intended use cases.
Emotion analysis and detection during COVID-19
Crises such as natural disasters, global pandemics, and social unrest continuously threaten our world and emotionally affect millions of people worldwide in distinct ways.
Political Ideology and Polarization of Policy Positions: A Multi-dimensional Approach
Analyzing ideology and polarization is of critical importance in advancing our grasp of modern politics.
Paragraph-level Simplification of Medical Texts
In this work we introduce a new corpus of parallel texts in English comprising technical and lay summaries of all published evidence pertaining to different clinical topics.
Did they answer? Subjective acts and intents in conversational discourse
Discourse signals are often implicit, leaving it up to the interpreter to draw the required inferences.
Learning to Generate Code Comments from Class Hierarchies
Descriptive code comments are essential for supporting code comprehension and maintenance.
Roosterize: Suggesting Lemma Names for Coq Verification Projects Using Deep Learning
Naming conventions are an important concern in large verification projects using proof assistants, such as Coq.
My Teacher Thinks The World Is Flat! Interpreting Automatic Essay Scoring Mechanism
However, little research has been put to understand and interpret the black-box nature of these deep-learning based scoring models.
Elaborative Simplification: Content Addition and Explanation Generation in Text Simplification
Much of modern-day text simplification research focuses on sentence-level simplification, transforming original, more complex sentences into simplified versions.
Help! Need Advice on Identifying Advice
Humans use language to accomplish a wide variety of tasks - asking for and giving advice being one of them.
Deep Just-In-Time Inconsistency Detection Between Comments and Source Code
For extrinsic evaluation, we show the usefulness of our approach by combining it with a comment update model to build a more comprehensive automatic comment maintenance system which can both detect and resolve inconsistent comments based on code changes.
Inquisitive Question Generation for High Level Text Comprehension
Inquisitive probing questions come naturally to humans in a variety of settings, but is a challenging task for automatic systems.
Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems
This number is increasing further due to COVID-19 and the associated automation of education and testing.
Learning to Format Coq Code Using Language Models
Should arguments to the rewrite tactic be separated by a single space?
An Annotated Dataset of Discourse Modes in Hindi Stories
In this paper, we present a new corpus consisting of sentences from Hindi short stories annotated for five different discourse modes argumentative, narrative, descriptive, dialogic and informative.
Detecting Perceived Emotions in Hurricane Disasters
Natural disasters (e. g., hurricanes) affect millions of people each year, causing widespread destruction in their wake.
Assessing Discourse Relations in Language Generation from GPT-2
Recent advances in NLP have been attributed to the emergence of large-scale pre-trained language models.
Learning to Update Natural Language Comments Based on Code Changes
We formulate the novel task of automatically updating an existing natural language comment based on changes in the body of code it accompanies.
Deep Generation of Coq Lemma Names Using Elaborated Terms
Our results show that Roosterize substantially outperforms baselines for suggesting lemma names, highlighting the importance of using multi-input models and elaborated terms.
Unsupervised Adversarial Domain Adaptation for Implicit Discourse Relation Classification
Implicit discourse relations are not only more challenging to classify, but also to annotate, than their explicit counterparts.
Associating Natural Language Comment and Source Code Entities
Comments are an integral part of software development; they are natural language descriptions associated with source code elements.
Discourse Level Factors for Sentence Deletion in Text Simplification
We inspect various document and discourse factors associated with sentence deletion, using a new manually annotated sentence alignment corpus we collected.
Adaptive Ensembling: Unsupervised Domain Adaptation for Political Document Analysis
Insightful findings in political science often require researchers to analyze documents of a certain subject or type, yet these documents are usually contained in large corpora that do not distinguish between pertinent and non-pertinent documents.
Evaluating Discourse in Structured Text Representations
Discourse structure is integral to understanding a text and is helpful in many NLP tasks.
Plain English Summarization of Contracts
We propose the task of summarizing such legal documents in plain English, which would enable users to have a better understanding of the terms they are accepting.
Linguistically-Informed Specificity and Semantic Plausibility for Dialogue Generation
Sequence-to-sequence models for open-domain dialogue generation tend to favor generic, uninformative responses.
From News to Medical: Cross-domain Discourse Segmentation
The first step in discourse analysis involves dividing a text into segments.
Domain Agnostic Real-Valued Specificity Prediction
Sentence specificity quantifies the level of detail in a sentence, characterizing the organization of information in discourse.
Why Swear? Analyzing and Inferring the Intentions of Vulgar Expressions
Vulgar words are employed in language use for several different functions, ranging from expressing aggression to signaling group identity or the informality of the communication.
Expressively vulgar: The socio-dynamics of vulgarity and its effects on sentiment analysis in social media
Vulgarity is a common linguistic expression and is used to perform several linguistic functions.
A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature
We present a corpus of 5, 000 richly annotated abstracts of medical articles describing clinical randomized controlled trials.
Aggregating and Predicting Sequence Labels from Crowd Annotations
Despite sequences being core to NLP, scant work has considered how to handle noisy sequence labels from multiple annotators for the same text.
Improving the Annotation of Sentence Specificity
We found that the lack of specificity distributes evenly among immediate prior context, long distance prior context and no prior context.
