Search Results for author:
Found 125 papers, 65 papers with code
Learning with Limited Text Data
Natural Language Processing (NLP) has achieved great progress in the past decade on the basis of neural models, which often make use of large amounts of labeled data to achieve state-of-the-art performance.
“This is a Problem, Don’t You Agree?” Framing and Bias in Human Evaluation for Natural Language Generation
Despite recent efforts reviewing current human evaluation practices for natural language generation (NLG) research, the lack of reported question wording and potential for framing effects or cognitive biases influencing results has been widely overlooked.
Personalized Response Generation with Tensor Factorization
Personalized response generation is essential for more human-like conversations.
Explaining Toxic Text via Knowledge Enhanced Text Generation
Warning: This paper contains content that is offensive and may be upsetting. Biased or toxic speech can be harmful to various demographic groups.
HypMix: Hyperbolic Interpolative Data Augmentation
Interpolation-based regularisation methods for data augmentation have proven to be effective for various tasks and modalities.
SPORTSINTERVIEW: A Large-Scale Sports Interview Benchmark for Entity-centric Dialogues
We propose a novel knowledge grounded dialogue (interview) dataset SPORTSINTERVIEW set in the domain of sports interview.
One Document, Many Revisions: A Dataset for Classification and Description of Edit Intents
Document authoring involves a lengthy revision process, marked by individual edits that are frequently linked to comments.
Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension
Through benchmarking with QG models, we show that the QG model trained on FairytaleQA is capable of asking high-quality and more diverse questions.
Focus on the Action: Learning to Highlight and Summarize Jointly for Email To-Do Items Summarization
Automatic email to-do item generation is the task of generating to-do items from a given email to help people overview emails and schedule daily work.
Frustratingly Simple but Surprisingly Strong: Using Language-Independent Features for Zero-shot Cross-lingual Semantic Parsing
The availability of corpora has led to significant advances in training semantic parsers in English.
Planning and Generating Natural and Diverse Disfluent Texts as Augmentation for Disfluency Detection
Existing approaches to disfluency detection heavily depend on human-annotated data.
DMix: Adaptive Distance-aware Interpolative Mixup
Interpolation-based regularisation methods such as Mixup, which generate virtual training samples, have proven to be effective for various tasks and modalities. We extend Mixup and propose DMix, an adaptive distance-aware interpolative Mixup that selects samples based on their diversity in the embedding space.
Simple Conversational Data Augmentation for Semi-supervised Abstractive Dialogue Summarization
Abstractive conversation summarization has received growing attention while most current state-of-the-art summarization models heavily rely on human-annotated summaries.
WIKIBIAS: Detecting Multi-Span Subjective Biases in Language
Biases continue to be prevalent in modern text and media, especially subjective bias – a special type of bias that introduces improper attitudes or presents a statement with the presupposition of truth.
Best Practices and Lessons Learned on Synthetic Data for Language Models
The success of AI models relies on the availability of large, diverse, and high-quality datasets, which can be challenging to obtain due to data scarcity, privacy concerns, and high costs.
Social Skill Training with Large Language Models
People rely on social skills like conflict resolution to communicate effectively and to thrive in both work and personal life.
Mapping the Increasing Use of LLMs in Scientific Papers
To address this gap, we conduct the first systematic, large-scale analysis across 950, 965 papers published between January 2020 and February 2024 on the arXiv, bioRxiv, and Nature portfolio journals, using a population-level statistical framework to measure the prevalence of LLM-modified content over time.
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs?
Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors
We further design a self-improvement method on top of large language models to enhance the automatic generation of feedback.
A Safe Harbor for AI Evaluation and Red Teaming
Independent evaluation and red teaming are critical for identifying the risks posed by generative AI systems.
Design2Code: How Far Are We From Automating Front-End Engineering?
In this work, we formalize this as a Design2Code task and conduct comprehensive benchmarking.
Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future
As Natural Language Processing (NLP) systems become increasingly integrated into human social life, these technologies will need to increasingly rely on social intelligence.
Unintended Impacts of LLM Alignment on Global Representation
Before being deployed for user-facing applications, developers align Large Language Models (LLMs) to user preferences through a variety of procedures, such as Reinforcement Learning From Human Feedback (RLHF) and Direct Preference Optimization (DPO).
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs
This paper introduces a new perspective to jailbreak LLMs as human-like communicators, to explore this overlooked intersection between everyday language interaction and AI safety.
Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach
Using COVID-19 as a testbed domain, our experiments demonstrate a significant alignment between the susceptibility scores estimated by our computational modeling and human judgments, confirming the effectiveness of this latent modeling approach.
Grounding Gaps in Language Model Generations
To understand the roots of the identified grounding gap, we examine the role of instruction tuning and preference optimization, finding that training on contemporary preference data leads to a reduction in generated grounding acts.
A Material Lens on Coloniality in NLP
Coloniality, the continuation of colonial harms beyond "official" colonization, has pervasive effects across society and scientific fields.
Task-Agnostic Low-Rank Adapters for Unseen English Dialects
Large Language Models (LLMs) are trained on corpora disproportionally weighted in favor of Standard American English.
Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data, however, this process might suffer from privacy issues and violations of data protection regulations.
Impressions: Understanding Visual Semiotics and Aesthetic Impact
We present Impressions, a novel dataset through which to investigate the semiotics of images, and how specific visual features and design choices can elicit specific emotions, thoughts and beliefs.
CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation
Annotated data plays a critical role in Natural Language Processing (NLP) in training models and evaluating their performance.
CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations
Recent work has aimed to capture nuances of human behavior by using LLMs to simulate responses from particular demographics in settings like social science experiments and public opinion surveys.
"Mistakes Help Us Grow": Facilitating and Evaluating Growth Mindset Supportive Language in Classrooms
Teachers' growth mindset supportive language (GMSL)--rhetoric emphasizing that one's skills can be improved over time--has been shown to significantly reduce disparities in academic achievement and enhance students' learning outcomes.
Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency
Developing an educational test can be expensive and time-consuming, as each item must be written by experts and then evaluated by collecting hundreds of student responses.
MIDDAG: Where Does Our News Go? Investigating Information Diffusion via Community-Level Information Pathways
We present MIDDAG, an intuitive, interactive system that visualizes the information propagation paths on social media triggered by COVID-19-related news articles accompanied by comprehensive insights, including user/community susceptibility level, as well as events and popular opinions raised by the crowd while propagating the information.
Dynamic LLM-Agent Network: An LLM-agent Collaboration Framework with Agent Team Optimization
We further design an automatic agent team optimization algorithm based on an unsupervised metric termed $\textit{Agent Importance Score}$, enabling the selection of best agents based on the contribution each agent makes.
DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
Moreover, DyVal-generated samples are not only evaluation sets, but also helpful data for fine-tuning to improve the performance of LLMs on existing benchmarks.
Rehearsal: Simulating Conflict to Teach Conflict Resolution
Compared to a control group with lecture material covering the same IRP theory, participants with simulated training from Rehearsal significantly improved their performance in the unaided conflict: they reduced their use of escalating competitive strategies by an average of 67%, while doubling their use of cooperative strategies.
Anchor Points: Benchmarking Models with Much Fewer Examples
Moreover, just several anchor points can be used to estimate model per-class predictions on all other points in a dataset with low mean absolute error, sufficient for gauging where the model is likely to fail.
Identifying and Mitigating the Security Risks of Generative AI
However, GenAI can be used just as well by attackers to generate new attacks and increase the velocity and efficacy of existing attacks.
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
Instruction tuning unlocks the superior capability of Large Language Models (LLM) to interact with humans.
Modeling Cross-Cultural Pragmatic Inference with Codenames Duet
Prior work uses simple reference games to test models of pragmatic reasoning, often with unidentified speakers and listeners.
Forgotten Knowledge: Examining the Citational Amnesia in NLP
Citing papers is the primary method through which modern scientific writing discusses and builds on past work.
Training Socially Aligned Language Models on Simulated Social Interactions
Social alignment in AI systems aims to ensure that these models behave according to established societal values.
TADA: Task-Agnostic Dialect Adapters for English
Large Language Models, the dominant starting point for Natural Language Processing (NLP) applications, fail at a higher rate for speakers of English dialects other than Standard American English (SAE).
NormBank: A Knowledge Bank of Situational Social Norms
We present NormBank, a knowledge bank of 155k situational norms.
Benchmarking LLM-based Machine Translation on Cultural Awareness
Furthermore, we devise a novel evaluation metric to assess the understandability of translations in a reference-free manner by GPT-4.
DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules
We show that DADA is effective for both single task and instruction finetuned language models, offering an extensible and interpretable framework for adapting existing LLMs to different English dialects.
Helping the Helper: Supporting Peer Counselors via AI-Empowered Practice and Feedback
Millions of users come to online peer counseling platforms to seek support on diverse topics ranging from relationship stress to anxiety.
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible.
Can Large Language Models Transform Computational Social Science?
We conclude that the performance of today's LLMs can augment the CSS research pipeline in two ways: (1) serving as zero-shot data annotators on human annotation teams, and (2) bootstrapping challenging creative generation tasks (e. g., explaining the underlying attributes of a text).
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
Diffusion models that are based on iterative denoising have been recently proposed and leveraged in various generation tasks like image generation.
Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints
The limits of open-ended generative models are unclear, yet increasingly important.
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Spurred by advancements in scale, large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot -- i. e., without adaptation on downstream data.
Auditing Gender Presentation Differences in Text-to-Image Models
Text-to-image models, which can generate high-quality images based on textual input, have recently enabled various content-creation tools.
Parameter-Efficient Fine-Tuning Design Spaces
We discover the following design patterns: (i) group layers in a spindle pattern; (ii) allocate the number of trainable parameters to layers uniformly; (iii) tune all the groups; (iv) assign proper tuning strategies to different groups.
Human-in-the-loop Abstractive Dialogue Summarization
Specifically, we ask humans to highlight the salient information to be included in summaries to provide the local feedback , and to make overall comparisons among summaries in terms of coherence, accuracy, coverage, concise and overall quality, as the global feedback.
Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games
We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes.
Multi-VALUE: A Framework for Cross-Dialectal English NLP
First, we use this system to stress tests question answering, machine translation, and semantic parsing.
DAMP: Doubly Aligned Multilingual Parser for Task-Oriented Dialogue
Modern virtual assistants use internal semantic parsing engines to convert user utterances to actionable commands.
On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Generating a Chain of Thought (CoT) has been shown to consistently improve large language model (LLM) performance on a wide range of NLP tasks.
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning
This leads to reasoning mistakes, which need to be corrected as they occur by teaching VL models the missing SVLC skills; often this must be done using private data where the issue was found, which naturally leads to a data-free continual (no task-id) VL learning setting.
Modeling Motivational Interviewing Strategies On An Online Peer-to-Peer Counseling Platform
This work provides a deeper understanding of the use of motivational interviewing techniques on peer-to-peer counselor platforms and sheds light on how to build better training programs for volunteer counselors on online platforms.
WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain
To this end, we contribute the Financial Language Understanding Evaluation (FLUE), an open-source comprehensive suite of benchmarks for the financial domain.
Geographic Citation Gaps in NLP Research
Similar disparities are also believed to exist for paper citation counts.
Robustness of Demonstration-based Learning Under Limited Data Scenario
Demonstration-based learning has shown great potential in stimulating pretrained language models' ability under limited data scenario.
Shapley Head Pruning: Identifying and Removing Interference in Multilingual Transformers
However, as a fixed-size model acquires more languages, its performance across all languages degrades, a phenomenon termed interference.
DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification
This paper proposes a simple yet effective interpolation-based data augmentation approach termed DoubleMix, to improve the robustness of models in text classification.
A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch
We empirically demonstrate that using an input sketch (even a poorly drawn one) in addition to text considerably increases retrieval recall compared to traditional text-based image retrieval.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models
SeqZero achieves SOTA performance of BART-based models on GeoQuery and EcommerceQuery, which are two few-shot datasets with compositional data split.
TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding
Data augmentation is an effective approach to tackle over-fitting.
SUBS: Subtree Substitution for Compositional Semantic Parsing
Although sequence-to-sequence models often achieve good performance in semantic parsing for i. i. d.
Inducing Positive Perspectives with Text Reframing
Sentiment transfer is one popular example of a text style transfer task, where the goal is to reverse the sentiment polarity of a text.
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
In this work, we introduce a new resource, not to authoritatively resolve moral ambiguities, but instead to facilitate systematic understanding of the intuitions, values and moral judgments reflected in the utterances of dialogue systems.
VALUE: Understanding Dialect Disparity in NLU
To understand disparities in current models and to facilitate more dialect-competent NLU systems, we introduce the VernAcular Language Understanding Evaluation (VALUE) benchmark, a challenging variant of GLUE that we created with a set of lexical and morphosyntactic transformation rules.
Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension
Through benchmarking with QG models, we show that the QG model trained on FairytaleQA is capable of asking high-quality and more diverse questions.
Ranked #1 on
Question Generation
on FairytaleQA
Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognition
We found that state-of-the-art NER systems trained on CoNLL 2003 training data drop performance dramatically on our challenging set.
Continual Sequence Generation with Adaptive Compositional Modules
Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks.
Measure and Improve Robustness in NLP Models: A Survey
Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research.
Interpreting Deep Learning Models in Natural Language Processing: A Review
Neural network models have achieved state-of-the-art performances in a wide range of natural language processing (NLP) tasks.
Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models
Recently, NLP models have achieved remarkable progress across a variety of tasks; however, they have also been criticized for being not robust.
GNN is a Counter? Revisiting GNN for Question Answering
To open the black box of GNN and investigate these problems, we dissect state-of-the-art GNN modules for QA and analyze their reasoning capability.
Ranked #12 on
Question Answering
on OpenBookQA
Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework
While previous work has focused on a single fairness criteria, we propose to use additional descriptive fairness metrics to better understand the source of these biases.
Semantic Categorization of Social Knowledge for Commonsense Question Answering
Large pre-trained language models (PLMs) have led to great success on various commonsense question answering (QA) tasks in an end-to-end fashion.
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
Hate speech has grown significantly on social media, causing serious consequences for victims of all demographics.
To Protect and To Serve? Analyzing Entity-Centric Framing of Police Violence
Framing has significant but subtle effects on public opinion and policy.
Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond
A fundamental goal of scientific research is to learn about causal relationships.
A Search Engine for Discovery of Scientific Challenges and Directions
To address this problem, we present a novel task of extraction and search of scientific challenges and directions, to facilitate rapid knowledge discovery.
Linguistic Characterization of Divisive Topics Online: Case Studies on Contentiousness in Abortion, Climate Change, and Gun Control
As public discourse continues to move and grow online, conversations about divisive topics on social media platforms have also increased.
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability
Fine-tuning large pre-trained models with task-specific data has achieved great success in NLP.
Quantifying the Impact of Human Capital, Job History, and Language Factors on Job Seniority with a Large-scale Analysis of Resumes
As job markets worldwide have become more competitive and applicant selection criteria have become more opaque, and different (and sometimes contradictory) information and advice is available for job seekers wishing to progress in their careers, it has never been more difficult to determine which factors in a r\'esum\'e most effectively help career progression.
An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets.
Disfl-QA: A Benchmark Dataset for Understanding Disfluencies in Question Answering
Disfluencies is an under-studied topic in NLP, even though it is ubiquitous in human conversation.
The Importance of Modeling Social Factors of Language: Theory and Practice
We show that current NLP systems systematically break down when faced with interpreting the social factors of language.
Personalized Response Generation via Generative Split Memory Network
Despite the impressive successes of generation and dialogue systems, how to endow a text generation system with particular personality traits to deliver more personalized responses remains under-investigated.
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization
Fine-tuning large pre-trained models with task-specific data has achieved great success in NLP.
Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs
Abstractive conversation summarization has received much attention recently.
Continual Learning for Text Classification with Information Disentanglement Based Regularization
Continual learning has become increasingly important as it enables NLP models to constantly learn and gain knowledge over time.
Putting Humans in the Natural Language Processing Loop: A Survey
How can we design Natural Language Processing (NLP) systems that learn from human feedback?
RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization
With the widespread use of toxic language online, platforms are increasingly using automated systems that leverage advances in natural language processing to automatically flag and remove toxic comments.
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics.
Ranked #1 on
Extreme Summarization
on GEM-XSum
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
Weakly-Supervised Hierarchical Models for Predicting Persuasive Strategies in Good-faith Textual Requests
Modeling persuasive language has the potential to better facilitate our decision-making processes.
Tuiteamos o pongamos un tuit? Investigating the Social Constraints of Loanword Integration in Spanish Social Media
In social media, we find that speaker background and expectations of formality explain loanword and native word integration, such that authors who use more Spanish and who write to a wider audience tend to use integrated verb forms more often.
Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization
Formality style transfer is the task of converting informal sentences to grammatically-correct formal sentences, which can be used to improve performance of many downstream NLP tasks.
Examining the Ordering of Rhetorical Strategies in Persuasive Requests
We find that specific (orderings of) strategies interact uniquely with a request's content to impact success rate, and thus the persuasiveness of a request.
Local Additivity Based Data Augmentation for Semi-supervised NER
Named Entity Recognition (NER) is one of the first stages in deep language understanding yet current NER models heavily rely on human-annotated data.
Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization
Text summarization is one of the most challenging and interesting problems in NLP.
Evaluating Graph Vulnerability and Robustness using TIGER
By democratizing the tools required to study network robustness, our goal is to assist researchers and practitioners in analyzing their own networks; and facilitate the development of new research in the field.
Racism is a Virus: Anti-Asian Hate and Counterspeech in Social Media during the COVID-19 Crisis
The spread of COVID-19 has sparked racism and hate on social media targeted towards Asian communities.
ToTTo: A Controlled Table-To-Text Generation Dataset
We present ToTTo, an open-domain English table-to-text dataset with over 120, 000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description.
Ranked #3 on
Data-to-Text Generation
on ToTTo
MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification
This paper presents MixText, a semi-supervised learning method for text classification, which uses our newly designed data augmentation method called TMix.
Semi-Supervised Models via Data Augmentationfor Classifying Interactive Affective Responses
We present semi-supervised models with data augmentation (SMDA), a semi-supervised text classification system to classify interactive affective responses.
RECAST: Interactive Auditing of Automatic Toxicity Detection Models
As toxic language becomes nearly pervasive online, there has been increasing interest in leveraging the advancements in natural language processing (NLP), from very large transformer models to automatically detecting and removing toxic comments.
Automatically Neutralizing Subjective Bias in Text
To address this issue, we introduce a novel testbed for natural language generation: automatically bringing inappropriately subjective text into a neutral point of view ("neutralizing" biased text).
Characterizing Collective Attention via Descriptor Context: A Case Study of Public Discussions of Crisis Events
But according to rationalist models of natural language communication, the collective salience of each entity will be expressed not only in how often it is mentioned, but in the form that those mentions take.
Let's Make Your Request More Persuasive: Modeling Persuasive Strategies via Semi-Supervised Neural Nets on Crowdfunding Platforms
Modeling what makes a request persuasive - eliciting the desired response from a reader - is critical to the study of propaganda, behavioral economics, and advertising.
Identifying Semantic Edit Intentions from Revisions in Wikipedia
Most studies on human editing focus merely on syntactic revision operations, failing to capture the intentions behind revision changes, which are essential for facilitating the single and collaborative writing process.
Edit Categories and Editor Role Identification in Wikipedia
In this work, we introduced a corpus for categorizing edit types in Wikipedia.
