Search Results for author: Nanyun Peng

Found 192 papers, 113 papers with code

AESOP: Paraphrase Generation with Adaptive Syntactic Control

1 code implementation EMNLP 2021 Jiao Sun, Xuezhe Ma, Nanyun Peng

We propose to control paraphrase generation through carefully chosen target syntactic structures to generate more proper and higher quality paraphrases.

Data Augmentation Language Modelling +2

ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations

no code implementations EMNLP 2021 Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

While these tasks partially evaluate machines’ ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning.

Machine Reading Comprehension Natural Language Queries +1

Improving Pre-trained Vision-and-Language Embeddings for Phrase Grounding

no code implementations EMNLP 2021 Zi-Yi Dou, Nanyun Peng

Phrase grounding aims to map textual phrases to their associated image regions, which can be a prerequisite for multimodal reasoning and can benefit tasks requiring identifying objects based on language.

Multimodal Reasoning Phrase Grounding

Evaluating Cultural and Social Awareness of LLM Web Agents

no code implementations30 Oct 2024 Haoyi Qiu, Alexander R. Fabbri, Divyansh Agarwal, Kung-Hsiang Huang, Sarah Tan, Nanyun Peng, Chien-Sheng Wu

To address these, we introduce CASA, a benchmark designed to assess LLM agents' sensitivity to cultural and social norms across two web-based tasks: online shopping and social discussion forums.

Benchmarking Navigate

Guiding Through Complexity: What Makes Good Supervision for Hard Reasoning Tasks?

1 code implementation27 Oct 2024 Xuan He, Da Yin, Nanyun Peng

Specifically, training on hard task supervision with the same outcome error rates but disparate step-wise error rates can lead to a 30\% accuracy gap on MATH benchmark.

Data Augmentation Math

Vulnerability of LLMs to Vertically Aligned Text Manipulations

no code implementations26 Oct 2024 Zhecheng Li, Yiwei Wang, Bryan Hooi, Yujun Cai, Zhen Xiong, Nanyun Peng, Kai-Wei Chang

Text classification involves categorizing a given text, such as determining its sentiment or identifying harmful content.

Few-Shot Learning text-classification +1

Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarization

no code implementations26 Oct 2024 Zhecheng Li, Yiwei Wang, Bryan Hooi, Yujun Cai, Naifan Cheung, Nanyun Peng, Kai-Wei Chang

To resolve this question, we fully explore the potential of large language models on cross-lingual summarization task for low-resource languages through our four-step zero-shot method: Summarization, Improvement, Translation and Refinement (SITR) with correspondingly designed prompts.

SPEED++: A Multilingual Event Extraction Framework for Epidemic Prediction and Preparedness

no code implementations24 Oct 2024 Tanmay Parekh, Jeffrey Kwan, Jiarui Yu, Sparsh Johri, Hyosang Ahn, Sreya Muppalla, Kai-Wei Chang, Wei Wang, Nanyun Peng

However, these works only focused on English posts, while epidemics can occur anywhere in the world, and early discussions are often in the local, non-English languages.

Event Extraction Misinformation

BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression

1 code implementation20 Oct 2024 Yuankai Li, Jia-Chen Gu, Di wu, Kai-Wei Chang, Nanyun Peng

Based on our synthetic data built entirely by open-source models, BRIEF generates more concise summaries and enables a range of LLMs to achieve exceptional open-domain question answering (QA) performance.

In-Context Learning Long-Context Understanding +3

MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models

no code implementations10 Oct 2024 WenBo Hu, Jia-Chen Gu, Zi-Yi Dou, Mohsen Fayyaz, Pan Lu, Kai-Wei Chang, Nanyun Peng

In this paper, we introduce a multimodal retrieval-augmented generation benchmark, MRAG-Bench, in which we systematically identify and categorize scenarios where visually augmented knowledge is better than textual knowledge, for instance, more images from varying viewpoints.

Multiple-choice Question Answering +1

LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints

no code implementations9 Oct 2024 Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, Nanyun Peng

To address the performance gap between open-source and proprietary models, we propose the Decompose, Critique and Refine (DeCRIM) self-correction pipeline, which enhances LLMs' ability to follow constraints.

Instruction Following

Control Large Language Models via Divide and Conquer

no code implementations6 Oct 2024 Bingxuan Li, Yiwei Wang, Tao Meng, Kai-Wei Chang, Nanyun Peng

This paper investigates controllable generation for large language models (LLMs) with prompt-based control, focusing on Lexically Constrained Generation (LCG).

Text Generation

Detecting Machine-Generated Long-Form Content with Latent-Space Variables

no code implementations4 Oct 2024 Yufei Tian, Zeyu Pan, Nanyun Peng

The increasing capability of large language models (LLMs) to generate fluent long-form texts is presenting new challenges in distinguishing machine-generated outputs from human-written ones, which is crucial for ensuring authenticity and trustworthiness of expressions.

Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding

no code implementations5 Sep 2024 Cheng Wang, Yiwei Wang, Bryan Hooi, Yujun Cai, Nanyun Peng, Kai-Wei Chang

The training data in large language models is key to their success, but it also presents privacy and security risks, as it may contain sensitive information.

REFFLY: Melody-Constrained Lyrics Editing Model

no code implementations30 Aug 2024 Songyan Zhao, Bingxuan Li, Yufei Tian, Nanyun Peng

Automatic melody-to-lyric generation aims to produce lyrics that align with a given melody.

ARMADA: Attribute-Based Multimodal Data Augmentation

no code implementations19 Aug 2024 Xiaomeng Jin, Jeonghwan Kim, Yu Zhou, Kuan-Hao Huang, Te-Lin Wu, Nanyun Peng, Heng Ji

To address these issues, we propose Attribute-based Multimodal Data Augmentation (ARMADA), a novel multimodal data augmentation method via knowledge-guided manipulation of visual attributes of the mentioned entities.

Attribute Data Augmentation

Unlocking Exocentric Video-Language Data for Egocentric Video Representation Learning

no code implementations7 Aug 2024 Zi-Yi Dou, Xitong Yang, Tushar Nagarajan, Huiyu Wang, Jing Huang, Nanyun Peng, Kris Kitani, Fu-Jen Chu

We present EMBED (Egocentric Models Built with Exocentric Data), a method designed to transform exocentric video-language data for egocentric video representation learning.

Multi-Instance Retrieval Representation Learning +1

QUDSELECT: Selective Decoding for Questions Under Discussion Parsing

no code implementations2 Aug 2024 Ashima Suvarna, Xiao Liu, Tanmay Parekh, Kai-Wei Chang, Nanyun Peng

In QUD parsing, each sentence is viewed as an answer to a question triggered by an anchor sentence in prior context.

Sentence

Are Large Language Models Capable of Generating Human-Level Narratives?

1 code implementation18 Jul 2024 Yufei Tian, Tenghao Huang, Miri Liu, Derek Jiang, Alexander Spangher, Muhao Chen, Jonathan May, Nanyun Peng

This paper investigates the capability of LLMs in storytelling, focusing on narrative development and plot progression.

Diversity

Evaluating Human Alignment and Model Faithfulness of LLM Rationale

no code implementations28 Jun 2024 Mohsen Fayyaz, Fan Yin, Jiao Sun, Nanyun Peng

We study how well large language models (LLMs) explain their generations through rationales -- a set of tokens extracted from the input text that reflect the decision-making process of LLMs.

Decision Making

LLM-A*: Large Language Model Enhanced Incremental Heuristic Search on Path Planning

1 code implementation20 Jun 2024 Silin Meng, Yiwei Wang, Cheng-Fu Yang, Nanyun Peng, Kai-Wei Chang

Path planning is a fundamental scientific problem in robotics and autonomous navigation, requiring the derivation of efficient routes from starting to destination points while avoiding obstacles.

Autonomous Navigation Language Modelling +1

VDebugger: Harnessing Execution Feedback for Debugging Visual Programs

1 code implementation19 Jun 2024 Xueqing Wu, Zongyu Lin, Songyan Zhao, Te-Lin Wu, Pan Lu, Nanyun Peng, Kai-Wei Chang

Visual programs are executable code generated by large language models to address visual reasoning problems.

Visual Reasoning

Adaptable Logical Control for Large Language Models

1 code implementation19 Jun 2024 Honghua Zhang, Po-Nien Kung, Masahiro Yoshida, Guy Van Den Broeck, Nanyun Peng

Despite the success of Large Language Models (LLMs) on various tasks following human instructions, controlling model generation at inference time poses a persistent challenge.

Math Text Generation

Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation

1 code implementation19 Jun 2024 Di wu, Jia-Chen Gu, Fan Yin, Nanyun Peng, Kai-Wei Chang

Retrieval-augmented language models (RALMs) have shown strong performance and wide applicability in knowledge-intensive tasks.

Retrieval Uncertainty Quantification

Measuring Psychological Depth in Language Models

1 code implementation18 Jun 2024 Fabrice Harel-Canada, Hanyu Zhou, Sreya Muppalla, Zeynep Yildiz, Miryung Kim, Amit Sahai, Nanyun Peng

By shifting the focus from text to reader, the Psychological Depth Scale is a validated, automated, and systematic means of measuring the capacity of LLMs to connect with humans through the stories they tell.

REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy

no code implementations11 Jun 2024 Haw-Shiuan Chang, Nanyun Peng, Mohit Bansal, Anil Ramakrishna, Tagyoung Chung

If a LLM's entropy is higher than the asymptotic entropy (i. e., the LLM is more uncertain than it should be), the THF model predicts a high hallucination hazard, which leads to a lower p threshold in REAL sampling.

Diversity Hallucination

CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation

no code implementations8 Jun 2024 I-Hung Hsu, Zifeng Wang, Long T. Le, Lesly Miculicich, Nanyun Peng, Chen-Yu Lee, Tomas Pfister

Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses by accurately citing verifiable sources.

Open-Domain Question Answering

Re-ReST: Reflection-Reinforced Self-Training for Language Agents

1 code implementation3 Jun 2024 Zi-Yi Dou, Cheng-Fu Yang, Xueqing Wu, Kai-Wei Chang, Nanyun Peng

Finetuning language agents with reasoning-action trajectories is effective, but obtaining these trajectories from human annotations or stronger models is costly and sometimes impractical.

Code Generation Multi-hop Question Answering +3

Matryoshka Query Transformer for Large Vision-Language Models

1 code implementation29 May 2024 WenBo Hu, Zi-Yi Dou, Liunian Harold Li, Amita Kamath, Nanyun Peng, Kai-Wei Chang

This raises the question: can we achieve flexibility in the number of visual tokens to suit different tasks and computational resources?

Language Modelling Representation Learning

FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation

no code implementations8 May 2024 Xuehai He, Jian Zheng, Jacob Zhiyuan Fang, Robinson Piramuthu, Mohit Bansal, Vicente Ordonez, Gunnar A Sigurdsson, Nanyun Peng, Xin Eric Wang

Controllable text-to-image (T2I) diffusion models generate images conditioned on both text prompts and semantic inputs of other modalities like edge maps.

Text-to-Image Generation

Medical Vision-Language Pre-Training for Brain Abnormalities

no code implementations27 Apr 2024 Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang

Vision-language models have become increasingly powerful for tasks that require an understanding of both visual and linguistic elements, bridging the gap between these modalities.

Language Modelling

Weak-to-Strong Extrapolation Expedites Alignment

1 code implementation25 Apr 2024 Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, Nanyun Peng

Through experiments with twelve open-source LLMs on HuggingFace, we demonstrate that ExPO consistently improves off-the-shelf DPO/RLHF models, as evaluated on the mainstream LLM benchmarks AlpacaEval 2. 0 and MT-Bench.

VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models

1 code implementation22 Apr 2024 Haoyi Qiu, WenBo Hu, Zi-Yi Dou, Nanyun Peng

Our work also highlights the critical balance between faithfulness and coverage of model outputs, and encourages future works to address hallucinations in LVLMs while keeping their outputs informative.

Hallucination Informativeness +2

GenEARL: A Training-Free Generative Framework for Multimodal Event Argument Role Labeling

no code implementations7 Apr 2024 Hritik Bansal, Po-Nien Kung, P. Jeffrey Brantingham, Kai-Wei Chang, Nanyun Peng

In this paper, we propose GenEARL, a training-free generative framework that harness the power of the modern generative models to understand event task descriptions given image contexts to perform the EARL task.

Language Modelling Large Language Model +1

PhonologyBench: Evaluating Phonological Skills of Large Language Models

no code implementations3 Apr 2024 Ashima Suvarna, Harshita Khandelwal, Nanyun Peng

To this end, we present PhonologyBench, a novel benchmark consisting of three diagnostic tasks designed to explicitly test the phonological skills of LLMs in English: grapheme-to-phoneme conversion, syllable counting, and rhyme word generation.

Language Modelling Large Language Model

Event Detection from Social Media for Epidemic Prediction

1 code implementation2 Apr 2024 Tanmay Parekh, Anh Mac, Jiarui Yu, Yuxuan Dong, Syed Shahriar, Bonnie Liu, Eric Yang, Kuan-Hao Huang, Wei Wang, Nanyun Peng, Kai-Wei Chang

In our work, we pioneer exploiting Event Detection (ED) for better preparedness and early warnings of any upcoming epidemic by developing a framework to extract and analyze epidemic-related events from social media posts.

Event Detection

Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization

1 code implementation31 Mar 2024 Hritik Bansal, Ashima Suvarna, Gantavya Bhatt, Nanyun Peng, Kai-Wei Chang, Aditya Grover

A common technique for aligning large language models (LLMs) relies on acquiring human preferences by comparing multiple generations conditioned on a fixed context.

Argument-Aware Approach To Event Linking

no code implementations22 Mar 2024 I-Hung Hsu, Zihan Xue, Nilay Pochh, Sahil Bansal, Premkumar Natarajan, Jayanth Srinivasa, Nanyun Peng

Event linking connects event mentions in text with relevant nodes in a knowledge base (KB).

Entity Linking

Improving Event Definition Following For Zero-Shot Event Detection

no code implementations5 Mar 2024 Zefan Cai, Po-Nien Kung, Ashima Suvarna, Mingyu Derek Ma, Hritik Bansal, Baobao Chang, P. Jeffrey Brantingham, Wei Wang, Nanyun Peng

We hypothesize that a diverse set of event types and definitions are the key for models to learn to follow event definitions while existing event extraction datasets focus on annotating many high-quality examples for a few event types.

Event Detection Event Extraction

DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation

1 code implementation4 Mar 2024 Xueqing Wu, Rui Zheng, Jingzhen Sha, Te-Lin Wu, Hanyu Zhou, Mohan Tang, Kai-Wei Chang, Nanyun Peng, Haoran Huang

We construct the DACO dataset, containing (1) 440 databases (of tabular data) collected from real-world scenarios, (2) ~2k query-answer pairs that can serve as weak supervision for model training, and (3) a concentrated but high-quality test set with human refined annotations that serves as our main evaluation benchmark.

2k Code Generation

On Prompt-Driven Safeguarding for Large Language Models

2 code implementations31 Jan 2024 Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie zhou, Kai-Wei Chang, Minlie Huang, Nanyun Peng

In this work, we investigate how LLMs' behavior (i. e., complying with or refusing user queries) is affected by safety prompts from the perspective of model representation.

ConTextual: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models

1 code implementation24 Jan 2024 Rohan Wadhawan, Hritik Bansal, Kai-Wei Chang, Nanyun Peng

We conduct experiments to assess the performance of 14 foundation models (GPT-4V, Gemini-Pro-Vision, LLaVA-Next) and establish a human performance baseline.

Visual Reasoning

DeepEdit: Knowledge Editing as Decoding with Constraints

2 code implementations19 Jan 2024 Yiwei Wang, Muhao Chen, Nanyun Peng, Kai-Wei Chang

How to edit the knowledge in multi-step reasoning has become the major challenge in the knowledge editing (KE) of large language models (LLMs).

Informativeness knowledge editing +2

Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue

1 code implementation9 Jan 2024 Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, Nanyun Peng

Model editing is a technique that edits the large language models (LLMs) with updated knowledge to alleviate hallucinations without resource-intensive retraining.

Model Editing Natural Language Inference +1

New Job, New Gender? Measuring the Social Bias in Image Generation Models

no code implementations1 Jan 2024 Wenxuan Wang, Haonan Bai, Jen-tse Huang, Yuxuan Wan, Youliang Yuan, Haoyi Qiu, Nanyun Peng, Michael R. Lyu

BiasPainter uses a diverse range of seed images of individuals and prompts the image generation models to edit these images using gender, race, and age-neutral queries.

Bias Detection Fairness +1

AMRFact: Enhancing Summarization Factuality Evaluation with AMR-Driven Negative Samples Generation

1 code implementation16 Nov 2023 Haoyi Qiu, Kung-Hsiang Huang, Jingnong Qu, Nanyun Peng

Prior works on evaluating factual consistency of summarization often take the entailment-based approaches that first generate perturbed (factual inconsistent) summaries and then train a classifier on the generated data to detect the factually inconsistencies during testing time.

Abstractive Text Summarization Natural Language Inference +1

TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction

1 code implementation16 Nov 2023 Kuan-Hao Huang, I-Hung Hsu, Tanmay Parekh, Zhiyu Xie, Zixuan Zhang, Premkumar Natarajan, Kai-Wei Chang, Nanyun Peng, Heng Ji

In this work, we identify and address evaluation challenges, including inconsistency due to varying data assumptions or preprocessing steps, the insufficiency of current evaluation frameworks that may introduce dataset or data split bias, and the low reproducibility of some previous approaches.

Benchmarking Event Extraction

Tracking the Newsworthiness of Public Documents

no code implementations16 Nov 2023 Alexander Spangher, Emilio Ferrara, Ben Welsh, Nanyun Peng, Serdar Tumgoren, Jonathan May

Journalists must find stories in huge amounts of textual data (e. g. leaks, bills, press releases) as part of their jobs: determining when and why text becomes news can help us understand coverage patterns and help us build assistive tools.

Retrieval

Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks

1 code implementation1 Nov 2023 Po-Nien Kung, Fan Yin, Di wu, Kai-Wei Chang, Nanyun Peng

Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models (LLMs) on a massive amount of diverse tasks with instructions.

Informativeness Out-of-Distribution Generalization +1

BOOST: Harnessing Black-Box Control to Boost Commonsense in LMs' Generation

no code implementations25 Oct 2023 Yufei Tian, Felix Zhang, Nanyun Peng

Large language models (LLMs) such as GPT-3 have demonstrated a strong capability to generate coherent and contextually relevant text.

Language Modelling Sentence

Localizing Active Objects from Egocentric Vision with Symbolic World Knowledge

1 code implementation23 Oct 2023 Te-Lin Wu, Yu Zhou, Nanyun Peng

The ability to actively ground task instructions from an egocentric view is crucial for AI agents to accomplish tasks or assist humans virtually.

Phrase Grounding World Knowledge

Evaluating Large Language Models on Controlled Generation Tasks

1 code implementation23 Oct 2023 Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Frederick Wieting, Nanyun Peng, Xuezhe Ma

While recent studies have looked into the abilities of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc, there have been few studies looking into the controllability of large language models on generation tasks.

Question Generation Question-Generation +2

"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters

1 code implementation13 Oct 2023 Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, Nanyun Peng

Through benchmarking evaluation on 2 popular LLMs- ChatGPT and Alpaca, we reveal significant gender biases in LLM-generated recommendation letters.

Benchmarking Fairness +1

Mitigating Bias for Question Answering Models by Tracking Bias Influence

no code implementations13 Oct 2023 Mingyu Derek Ma, Jiun-Yu Kao, Arpit Gupta, Yu-Hsiang Lin, Wenbo Zhao, Tagyoung Chung, Wei Wang, Kai-Wei Chang, Nanyun Peng

Based on the intuition that a model would lean to be more biased if it learns from a biased example, we measure the bias level of a query instance by observing its influence on another instance.

Multiple-choice Multi-Task Learning +1

Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systems

1 code implementation8 Oct 2023 Yixin Wan, Jieyu Zhao, Aman Chadha, Nanyun Peng, Kai-Wei Chang

Recent advancements in Large Language Models empower them to follow freeform instructions, including imitating generic or specific demographic personas in conversations.

Benchmarking

MIDDAG: Where Does Our News Go? Investigating Information Diffusion via Community-Level Information Pathways

no code implementations4 Oct 2023 Mingyu Derek Ma, Alexander K. Taylor, Nuan Wen, Yanchen Liu, Po-Nien Kung, Wenna Qin, Shicheng Wen, Azure Zhou, Diyi Yang, Xuezhe Ma, Nanyun Peng, Wei Wang

We present MIDDAG, an intuitive, interactive system that visualizes the information propagation paths on social media triggered by COVID-19-related news articles accompanied by comprehensive insights, including user/community susceptibility level, as well as events and popular opinions raised by the crowd while propagating the information.

Contextual Label Projection for Cross-Lingual Structured Prediction

1 code implementation16 Sep 2023 Tanmay Parekh, I-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, Nanyun Peng

Label projection, which involves obtaining translated labels and texts jointly, is essential for leveraging machine translation to facilitate cross-lingual transfer in structured prediction tasks.

Event Argument Extraction Machine Translation +6

RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment

2 code implementations24 Jul 2023 Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian

We propose Reinforcement Learning from Contrastive Distillation (RLCD), a method for aligning language models to follow principles expressed in natural language (e. g., to be more harmless) without using human feedback.

Language Modelling reinforcement-learning +1

DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models

1 code implementation20 Jun 2023 Sidi Lu, Wenbo Zhao, Chenyang Tao, Arpit Gupta, Shanchan Wu, Tagyoung Chung, Nanyun Peng

NeurAlly-Decomposed Oracle (NADO) is a powerful approach for controllable generation with large language models.

Machine Translation

Open-Domain Text Evaluation via Contrastive Distribution Methods

1 code implementation20 Jun 2023 Sidi Lu, Hongyi Liu, Asli Celikyilmaz, Tianlu Wang, Nanyun Peng

We investigate CDM for open-domain text generation evaluation under two paradigms: 1) _Generative_ CDM, which harnesses the contrast of two language models' distributions to generate synthetic examples for training discriminator-based metrics; 2) _Discriminative_ CDM, which directly uses distribution disparities between two language models for evaluation.

Abstractive Text Summarization Coherence Evaluation +1

Are Fairy Tales Fair? Analyzing Gender Bias in Temporal Narrative Event Chains of Children's Fairy Tales

no code implementations26 May 2023 Paulina Toro Isaza, Guangxuan Xu, Akintoye Oloko, Yufang Hou, Nanyun Peng, Dakuo Wang

Social biases and stereotypes are embedded in our culture in part through their presence in our stories, as evidenced by the rich history of humanities and social science literature analyzing such biases in children stories.

AMPERE: AMR-Aware Prefix for Generation-Based Event Argument Extraction Model

1 code implementation26 May 2023 I-Hung Hsu, Zhiyu Xie, Kuan-Hao Huang, Prem Natarajan, Nanyun Peng

However, existing generation-based EAE models mostly focus on problem re-formulation and prompt design, without incorporating additional information that has been shown to be effective for classification-based models, such as the abstract meaning representation (AMR) of the input passages.

Abstract Meaning Representation Event Argument Extraction

Code-Switched Text Synthesis in Unseen Language Pairs

no code implementations26 May 2023 I-Hung Hsu, Avik Ray, Shubham Garg, Nanyun Peng, Jing Huang

In this work, we study the problem of synthesizing code-switched texts for language pairs absent from the training data.

Machine Translation

Identifying Informational Sources in News Articles

1 code implementation24 May 2023 Alexander Spangher, Nanyun Peng, Jonathan May, Emilio Ferrara

News articles are driven by the informational sources journalists use in reporting.

Text Generation

Gender Biases in Automatic Evaluation Metrics for Image Captioning

1 code implementation24 May 2023 Haoyi Qiu, Zi-Yi Dou, Tianlu Wang, Asli Celikyilmaz, Nanyun Peng

Model-based evaluation metrics (e. g., CLIPScore and GPTScore) have demonstrated decent correlations with human judgments in various language generation tasks.

Fairness Image Captioning +1

Masked Path Modeling for Vision-and-Language Navigation

no code implementations23 May 2023 Zi-Yi Dou, Feng Gao, Nanyun Peng

In this paper, we introduce a masked path modeling (MPM) objective, which pretrains an agent using self-collected data for downstream navigation tasks.

Action Generation Navigate +1

Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning

no code implementations19 May 2023 Po-Nien Kung, Nanyun Peng

Our experiments show that models trained on simplified task definition or delusive examples can achieve comparable performance to the ones trained on the original instructions and examples.

Zero-Shot Learning

ACCENT: An Automatic Event Commonsense Evaluation Metric for Open-Domain Dialogue Systems

1 code implementation12 May 2023 Sarik Ghazarian, Yijia Shao, Rujun Han, Aram Galstyan, Nanyun Peng

We take the first step by focusing on event commonsense that considers events and their relations, and is crucial in both dialogues and general commonsense reasoning.

Unsupervised Melody-Guided Lyrics Generation

no code implementations12 May 2023 Yufei Tian, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Gunnar Sigurdsson, Chenyang Tao, Wenbo Zhao, Tagyoung Chung, Jing Huang, Nanyun Peng

At inference time, we leverage the crucial alignments between melody and lyrics and compile the given melody into constraints to guide the generation process.

Text Generation

Tractable Control for Autoregressive Language Generation

1 code implementation15 Apr 2023 Honghua Zhang, Meihua Dang, Nanyun Peng, Guy Van Den Broeck

To overcome this challenge, we propose to use tractable probabilistic models (TPMs) to impose lexical constraints in autoregressive text generation models, which we refer to as GeLaTo (Generating Language with Tractable Constraints).

Text Generation

Sequentially Controlled Text Generation

no code implementations5 Jan 2023 Alexander Spangher, Xinyu Hua, Yao Ming, Nanyun Peng

While GPT-2 generates sentences that are remarkably human-like, longer documents can ramble and do not follow human-like writing structure.

Text Generation

DOC: Improving Long Story Coherence With Detailed Outline Control

1 code implementation20 Dec 2022 Kevin Yang, Dan Klein, Nanyun Peng, Yuandong Tian

In human evaluations of automatically generated stories, DOC substantially outperforms a strong Re3 baseline (Yang et al., 2022) on plot coherence (22. 5% absolute gain), outline relevance (28. 2%), and interestingness (20. 7%).

Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts

1 code implementation3 Dec 2022 Arshiya Aggarwal, Jiao Sun, Nanyun Peng

These fixed prefix templates could themselves be specific in terms of styles or linguistic structures, which may lead to unreliable fairness conclusions that are not representative of the general trends from tone varying prompts.

Fairness Text Generation

A Moral- and Event- Centric Inspection of Gender Bias in Fairy Tales at A Large Scale

no code implementations25 Nov 2022 Zhixuan Zhou, Jiao Sun, Jiaxin Pei, Nanyun Peng, JinJun Xiong

Our analysis further reveal stereotypical portrayals of both male and female characters in terms of moral foundations and events.

Fairness

ExPUNations: Augmenting Puns with Keywords and Explanations

1 code implementation24 Oct 2022 Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Tagyoung Chung, Jing Huang, Yang Liu, Nanyun Peng

The tasks of humor understanding and generation are challenging and subjective even for humans, requiring commonsense and real-world knowledge to master.

Explanation Generation Natural Language Understanding +1

Context-Situated Pun Generation

1 code implementation24 Oct 2022 Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Shuyang Gao, Tagyoung Chung, Jing Huang, Yang Liu, Nanyun Peng

In this work, we propose a new task, context-situated pun generation, where a specific context represented by a set of keywords is provided, and the task is to first identify suitable pun words that are appropriate for the context, then generate puns based on the context keywords and the identified pun words.

Retrieval

A Unified Framework for Pun Generation with Humor Principles

1 code implementation24 Oct 2022 Yufei Tian, Divyanshu Sheth, Nanyun Peng

We propose a unified framework to generate both homophonic and homographic puns to resolve the split-up in existing works.

EnDex: Evaluation of Dialogue Engagingness at Scale

1 code implementation22 Oct 2022 Guangxuan Xu, Ruibo Liu, Fabrice Harel-Canada, Nischal Reddy Chandra, Nanyun Peng

We propose EnDex, the first human-reaction based model to evaluate dialogue engagingness.

Character-Centric Story Visualization via Visual Planning and Token Alignment

2 code implementations16 Oct 2022 Hong Chen, Rujun Han, Te-Lin Wu, Hideki Nakayama, Nanyun Peng

This task requires machines to 1) understand long text inputs and 2) produce a globally consistent image sequence that illustrates the contents of the story.

Story Visualization Text-to-Image Generation

Re3: Generating Longer Stories With Recursive Reprompting and Revision

1 code implementation13 Oct 2022 Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein

We consider the problem of automatically generating longer stories of over two thousand words.

Language Modelling

Controllable Text Generation for Open-Domain Creativity and Fairness

no code implementations24 Sep 2022 Nanyun Peng

Recent advances in large pre-trained language models have demonstrated strong results in generating natural languages and significantly improved performances for many natural language generation (NLG) applications such as machine translation and text summarization.

Fairness Machine Translation +2

NECE: Narrative Event Chain Extraction Toolkit

no code implementations17 Aug 2022 Guangxuan Xu, Paulina Toro Isaza, Moshi Li, Akintoye Oloko, Bingsheng Yao, Cassia Sanctos, Aminat Adebiyi, Yufang Hou, Nanyun Peng, Dakuo Wang

To understand a narrative, it is essential to comprehend the temporal event flows, especially those associated with main characters; however, this can be challenging with lengthy and unstructured narrative texts.

Question Answering

NewsEdits: A News Article Revision Dataset and a Document-Level Reasoning Challenge

1 code implementation14 Jun 2022 Alexander Spangher, Xiang Ren, Jonathan May, Nanyun Peng

News article revision histories provide clues to narrative and factual evolution in news articles.

FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation

1 code implementation NAACL 2022 Zi-Yi Dou, Nanyun Peng

The speaker-follower models have proven to be effective in vision-and-language navigation, where a speaker model is used to synthesize new instructions to augment the training data for a follower navigation model.

Vision and Language Navigation

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

4 code implementations9 Jun 2022 Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, ZiRui Wang, Ziyi Wu

BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.

Common Sense Reasoning Math +1

Controllable Text Generation with Neurally-Decomposed Oracle

1 code implementation27 May 2022 Tao Meng, Sidi Lu, Nanyun Peng, Kai-Wei Chang

We propose a general and efficient framework to control auto-regressive generation models with NeurAlly-Decomposed Oracle (NADO).

Language Modelling Machine Translation +1

Helpfulness and Fairness of Task-Oriented Dialogue Systems

no code implementations25 May 2022 Jiao Sun, Yu Hou, Jiin Kim, Nanyun Peng

Then, we collect human annotations for the helpfulness of dialogue responses based on our definition and build a classifier to automatically determine the helpfulness of a response.

Fairness Goal-Oriented Dialogue Systems +1

TAGPRIME: A Unified Framework for Relational Structure Extraction

1 code implementation25 May 2022 I-Hung Hsu, Kuan-Hao Huang, Shuning Zhang, Wenxin Cheng, Premkumar Natarajan, Kai-Wei Chang, Nanyun Peng

In this work, we propose to take a unified view of all these tasks and introduce TAGPRIME to address relational structure extraction problems.

Event Argument Extraction Language Modelling +2

Learning Action Conditions from Instructional Manuals for Instruction Understanding

1 code implementation25 May 2022 Te-Lin Wu, Caiqi Zhang, Qingyuan Hu, Alex Spangher, Nanyun Peng

The ability to infer pre- and postconditions of an action is vital for comprehending complex instructions, and is essential for applications such as autonomous instruction-guided agents and assistive AI that supports humans to perform physical tasks.

GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles

1 code implementation25 May 2022 Tanmay Parekh, I-Hung Hsu, Kuan-Hao Huang, Kai-Wei Chang, Nanyun Peng

We utilize this ontology to further introduce GENEVA, a diverse generalizability benchmarking dataset comprising four test suites, aimed at evaluating models' ability to handle limited data and unseen event type generalization.

Benchmarking Event Argument Extraction +1

Sibylvariant Transformations for Robust Text Classification

1 code implementation Findings (ACL) 2022 Fabrice Harel-Canada, Muhammad Ali Gulzar, Nanyun Peng, Miryung Kim

The vast majority of text transformation techniques in NLP are inherently limited in their ability to expand input space coverage due to an implicit constraint to preserve the original class label.

Adversarial Robustness Defect Detection +2

AmbiPun: Generating Humorous Puns with Ambiguous Context

1 code implementation NAACL 2022 Anirudh Mittal, Yufei Tian, Nanyun Peng

In this paper, we propose a simple yet effective way to generate pun sentences that does not require any training on existing puns.

Reverse Dictionary

Zero-shot Sonnet Generation with Discourse-level Planning and Aesthetics Features

1 code implementation NAACL 2022 Yufei Tian, Nanyun Peng

Poetry generation, and creative language generation in general, usually suffers from the lack of large training data.

Sonnet Generation

DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations

1 code implementation ACL 2022 Sarik Ghazarian, Nuan Wen, Aram Galstyan, Nanyun Peng

We also show that DEAM can distinguish between coherent and incoherent dialogues generated by baseline manipulations, whereas those baseline models cannot detect incoherent examples generated by DEAM.

Abstract Meaning Representation Coherence Evaluation +1

Zero-shot Commonsense Question Answering with Cloze Translation and Consistency Optimization

1 code implementation1 Jan 2022 Zi-Yi Dou, Nanyun Peng

In this paper, we instead focus on better utilizing the \textit{implicit knowledge} stored in pre-trained language models.

Natural Questions Question Answering +4

Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manuals

no code implementations ACL 2022 Te-Lin Wu, Alex Spangher, Pegah Alipoormolabashi, Marjorie Freedman, Ralph Weischedel, Nanyun Peng

The ability to sequence unordered events is an essential skill to comprehend and reason about real world task procedures, which often requires thorough understanding of temporal common sense and multimodal information, as these procedures are often communicated through a combination of texts and images.

Common Sense Reasoning Open-Ended Question Answering

On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark

1 code implementation Findings (ACL) 2022 Hao Sun, Guangxuan Xu, Jiawen Deng, Jiale Cheng, Chujie Zheng, Hao Zhou, Nanyun Peng, Xiaoyan Zhu, Minlie Huang

We propose a taxonomy for dialogue safety specifically designed to capture unsafe behaviors in human-bot dialogue settings, with focuses on context-sensitive unsafety, which is under-explored in prior works.

HyperExpan: Taxonomy Expansion with Hyperbolic Representation Learning

no code implementations Findings (EMNLP) 2021 Mingyu Derek Ma, Muhao Chen, Te-Lin Wu, Nanyun Peng

Taxonomies are valuable resources for many applications, but the limited coverage due to the expensive manual curation process hinders their general applicability.

Graph Neural Network Representation Learning +1

Document-level Entity-based Extraction as Template Generation

1 code implementation EMNLP 2021 Kung-Hsiang Huang, Sam Tang, Nanyun Peng

Document-level entity-based extraction (EE), aiming at extracting entity-centric information such as entity roles and entity relations, is key to automatic knowledge acquisition from text corpora for various domains.

4-ary Relation Extraction Binary Relation Extraction +1

Paraphrase Generation as Unsupervised Machine Translation

no code implementations COLING 2022 Xiaofei Sun, Yufei Tian, Yuxian Meng, Nanyun Peng, Fei Wu, Jiwei Li, Chun Fan

Then based on the paraphrase pairs produced by these UMT models, a unified surrogate model can be trained to serve as the final \sts model to generate paraphrases, which can be directly used for test in the unsupervised setup, or be finetuned on labeled datasets in the supervised setup.

Paraphrase Generation Sentence +3

DEGREE: A Data-Efficient Generation-Based Event Extraction Model

2 code implementations NAACL 2022 I-Hung Hsu, Kuan-Hao Huang, Elizabeth Boschee, Scott Miller, Prem Natarajan, Kai-Wei Chang, Nanyun Peng

Given a passage and a manually designed prompt, DEGREE learns to summarize the events mentioned in the passage into a natural sentence that follows a predefined pattern.

Event Extraction Sentence +2

On Measures of Biases and Harms in NLP

no code implementations7 Aug 2021 Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang

Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality.

Men Are Elected, Women Are Married: Events Gender Bias on Wikipedia

1 code implementation ACL 2021 Jiao Sun, Nanyun Peng

Human activities can be seen as sequences of events, which are crucial to understanding societies.

Event Detection

COM2SENSE: A Commonsense Reasoning Benchmark with Complementary Sentences

1 code implementation Findings (ACL) 2021 Shikhar Singh, Nuan Wen, Yu Hou, Pegah Alipoormolabashi, Te-Lin Wu, Xuezhe Ma, Nanyun Peng

To this end, we introduce a new commonsense reasoning benchmark dataset comprising natural language true/false statements, with each sample paired with its complementary counterpart, resulting in 4k sentence pairs.

4k Sentence

Metaphor Generation with Conceptual Mappings

1 code implementation ACL 2021 Kevin Stowe, Tuhin Chakrabarty, Nanyun Peng, Smaranda Muresan, Iryna Gurevych

Guided by conceptual metaphor theory, we propose to control the generation process by encoding conceptual mappings between cognitive domains to generate meaningful metaphoric expressions.

Sentence

``Nice Try, Kiddo'': Investigating Ad Hominems in Dialogue Responses

no code implementations NAACL 2021 Emily Sheng, Kai-Wei Chang, Prem Natarajan, Nanyun Peng

Ad hominem attacks are those that target some feature of a person{'}s character instead of the position the person is maintaining.

Abusive Language

Societal Biases in Language Generation: Progress and Challenges

1 code implementation ACL 2021 Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng

Technology for language generation has advanced rapidly, spurred by advancements in pre-training large models on massive amounts of data and the need for intelligent agents to communicate in a natural manner.

Fairness Text Generation

"Don't quote me on that": Finding Mixtures of Sources in News Articles

1 code implementation19 Apr 2021 Alexander Spangher, Nanyun Peng, Jonathan May, Emilio Ferrara

Journalists publish statements provided by people, or \textit{sources} to contextualize current events, help voters make informed decisions, and hold powerful individuals accountable.

Clustering

Modeling "Newsworthiness" for Lead-Generation Across Corpora

no code implementations19 Apr 2021 Alexander Spangher, Nanyun Peng, Jonathan May, Emilio Ferrara

Journalists obtain "leads", or story ideas, by reading large corpora of government records: court cases, proposed bills, etc.

Revealing Persona Biases in Dialogue Systems

1 code implementation18 Apr 2021 Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang, Nanyun Peng

Dialogue systems in the form of chatbots and personal assistants are being increasingly integrated into people's lives.

Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Training

1 code implementation EMNLP 2021 Kuan-Hao Huang, Wasi Uddin Ahmad, Nanyun Peng, Kai-Wei Chang

Pre-trained multilingual language encoders, such as multilingual BERT and XLM-R, show great potential for zero-shot cross-lingual transfer.

Sentence text-classification +4