Search Results for author: Asli Celikyilmaz

Found 77 papers, 28 papers with code

Making PPO even better: Value-Guided Monte-Carlo Tree Search decoding

no code implementations26 Sep 2023 Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, Asli Celikyilmaz

The key idea is not to throw out the value network, a byproduct of PPO training for evaluating partial output sequences, when decoding text out of the policy network.

Text Generation

Chain-of-Verification Reduces Hallucination in Large Language Models

no code implementations20 Sep 2023 Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston

Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models.

Text Generation

Shepherd: A Critic for Language Model Generation

1 code implementation8 Aug 2023 Tianlu Wang, Ping Yu, Xiaoqing Ellen Tan, Sean O'Brien, Ramakanth Pasunuru, Jane Dwivedi-Yu, Olga Golovneva, Luke Zettlemoyer, Maryam Fazel-Zarandi, Asli Celikyilmaz

As large language models improve, there is increasing interest in techniques that leverage these models' capabilities to refine their own outputs.

Language Modelling

RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment

1 code implementation24 Jul 2023 Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian

We propose Reinforcement Learning from Contrast Distillation (RLCD), a method for aligning language models to follow natural language principles without using human feedback.

Language Modelling reinforcement-learning

Understanding In-Context Learning via Supportive Pretraining Data

no code implementations26 Jun 2023 Xiaochuang Han, Daniel Simig, Todor Mihaylov, Yulia Tsvetkov, Asli Celikyilmaz, Tianlu Wang

We observe that a continued pretraining on this small subset significantly improves the model's ICL ability, by up to 18%.

Open-Domain Text Evaluation via Meta Distribution Modeling

no code implementations20 Jun 2023 Sidi Lu, Asli Celikyilmaz, Tianlu Wang, Nanyun Peng

We investigate MDM for open-domain text generation evaluation under two paradigms: 1) \emph{Generative} MDM, which leverages the Meta-Distribution Methods to generate in-domain negative samples for training discriminator-based metrics; 2) \emph{Discriminative} MDM, which directly uses distribution discrepancies between two language models for evaluation.

Abstractive Text Summarization Text Generation

Gender Biases in Automatic Evaluation Metrics: A Case Study on Image Captioning

no code implementations24 May 2023 Haoyi Qiu, Zi-Yi Dou, Tianlu Wang, Asli Celikyilmaz, Nanyun Peng

Pretrained model-based evaluation metrics have demonstrated strong performance with high correlations with human judgments in various natural language generation tasks such as image captioning.

Fairness Image Captioning +1

Look-back Decoding for Open-Ended Text Generation

no code implementations22 May 2023 Nan Xu, Chunting Zhou, Asli Celikyilmaz, Xuezhe Ma

Given a prefix (context), open-ended generation aims to decode texts that are coherent, which don't abruptly drift from previous topics, and informative, which don't suffer from undesired repetitions.

Story Generation

OPT-R: Exploring the Role of Explanations in Finetuning and Prompting for Reasoning Skills of Large Language Models

no code implementations19 May 2023 Badr AlKhamissi, Siddharth Verma, Ping Yu, Zhijing Jin, Asli Celikyilmaz, Mona Diab

Our study entails finetuning three different sizes of OPT on a carefully curated reasoning corpus, resulting in two sets of finetuned models: OPT-R, finetuned without explanations, and OPT-RE, finetuned with explanations.

RECKONING: Reasoning through Dynamic Knowledge Encoding

no code implementations10 May 2023 Zeming Chen, Gail Weiss, Eric Mitchell, Asli Celikyilmaz, Antoine Bosselut

In the outer loop, the model learns to use the updated weights to reproduce and answer reasoning questions about the memorized knowledge.

Large Language Model Programs

no code implementations9 May 2023 Imanol Schlag, Sainbayar Sukhbaatar, Asli Celikyilmaz, Wen-tau Yih, Jason Weston, Jürgen Schmidhuber, Xian Li

In recent years, large pre-trained language models (LLMs) have demonstrated the ability to follow instructions and perform novel tasks from a few examples.

Language Modelling Large Language Model +1

STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension

no code implementations24 Dec 2022 Borui Wang, Chengcheng Feng, Arjun Nair, Madelyn Mao, Jai Desai, Asli Celikyilmaz, Haoran Li, Yashar Mehdad, Dragomir Radev

Abstractive dialogue summarization has long been viewed as an important standalone task in natural language processing, but no previous work has explored the possibility of whether abstractive dialogue summarization can also be used as a means to boost an NLP system's performance on other important dialogue comprehension tasks.

Abstractive Dialogue Summarization Question Answering

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

no code implementations22 Dec 2022 Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, Xian Li, Brian O'Horo, Gabriel Pereyra, Jeff Wang, Christopher Dewan, Asli Celikyilmaz, Luke Zettlemoyer, Ves Stoyanov

To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks.

Language Modelling Meta-Learning +2

Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences

no code implementations19 Dec 2022 Asish Ghoshal, Arash Einolghozati, Ankit Arun, Haoran Li, Lili Yu, Yashar Mehdad, Scott Wen-tau Yih, Asli Celikyilmaz

Lack of factual correctness is an issue that still plagues state-of-the-art summarization systems despite their impressive progress on generating seemingly fluent summaries.

Abstractive Text Summarization

ALERT: Adapting Language Models to Reasoning Tasks

no code implementations16 Dec 2022 Ping Yu, Tianlu Wang, Olga Golovneva, Badr Alkhamissy, Gargi Ghosh, Mona Diab, Asli Celikyilmaz

Current large language models can perform reasonably well on complex tasks that require step-by-step reasoning with few-shot learning.

Few-Shot Learning Language Modelling +1

MURMUR: Modular Multi-Step Reasoning for Semi-Structured Data-to-Text Generation

no code implementations16 Dec 2022 Swarnadeep Saha, Xinyan Velocity Yu, Mohit Bansal, Ramakanth Pasunuru, Asli Celikyilmaz

We propose MURMUR, a neuro-symbolic modular approach to text generation from semi-structured data with multi-step reasoning.

Data-to-Text Generation

ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning

1 code implementation15 Dec 2022 Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, Asli Celikyilmaz

Large language models show improved downstream task performance when prompted to generate step-by-step reasoning to justify their final answers.

Informativeness Text Generation

Complementary Explanations for Effective In-Context Learning

1 code implementation25 Nov 2022 Xi Ye, Srinivasan Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, Ramakanth Pasunuru

Large language models (LLMs) have exhibited remarkable capabilities in learning from explanations in prompts, but there has been limited understanding of exactly how these explanations function or why they are effective.

ED-FAITH: Evaluating Dialogue Summarization on Faithfulness

no code implementations15 Nov 2022 Sicong Huang, Asli Celikyilmaz, Haoran Li

Abstractive summarization models typically generate content unfaithful to the input, thus highlighting the significance of evaluating the faithfulness of generated summaries.

Abstractive Text Summarization Language Modelling

ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection

no code implementations25 May 2022 Badr AlKhamissi, Faisal Ladhak, Srini Iyer, Ves Stoyanov, Zornitsa Kozareva, Xian Li, Pascale Fung, Lambert Mathias, Asli Celikyilmaz, Mona Diab

Hate speech detection is complex; it relies on commonsense reasoning, knowledge of stereotypes, and an understanding of social nuance that differs from one culture to the next.

Cultural Vocal Bursts Intensity Prediction Few-Shot Learning +1

A Review on Language Models as Knowledge Bases

no code implementations12 Apr 2022 Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, Marjan Ghazvininejad

Recently, there has been a surge of interest in the NLP community on the use of pretrained Language Models (LMs) as Knowledge Bases (KBs).

Know Thy Strengths: Comprehensive Dialogue State Tracking Diagnostics

2 code implementations15 Dec 2021 Hyundong Cho, Chinnadhurai Sankar, Christopher Lin, Kaushik Ram Sadagopan, Shahin Shayandeh, Asli Celikyilmaz, Jonathan May, Ahmad Beirami

Recent works that revealed the vulnerability of dialogue state tracking (DST) models to distributional shifts have made holistic comparisons on robustness and qualitative analyses increasingly important for understanding their relative performance.

Ranked #4 on Multi-domain Dialogue State Tracking on MULTIWOZ 2.1 (using extra training data)

Dialogue State Tracking Multi-domain Dialogue State Tracking +1

Discourse-Aware Soft Prompting for Text Generation

no code implementations10 Dec 2021 Marjan Ghazvininejad, Vladimir Karpukhin, Vera Gor, Asli Celikyilmaz

We show that soft-prompt based conditional text generation can be improved with simple and efficient methods that simulate modeling the discourse structure of human written text.

Blocking Conditional Text Generation +1

Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs

1 code implementation26 Nov 2021 Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer

In this paper, we discuss approaches to detecting when models have beliefs about the world, and we improve on methods for updating model beliefs to be more truthful, with a focus on methods based on learned optimizers or hypernetworks.

Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries

no code implementations NAACL 2022 Xiangru Tang, Alexander Fabbri, Haoran Li, Ziming Mao, Griffin Thomas Adams, Borui Wang, Asli Celikyilmaz, Yashar Mehdad, Dragomir Radev

Current pre-trained models applied to summarization are prone to factual inconsistencies which either misrepresent the source text or introduce extraneous information.

An Exploratory Study on Long Dialogue Summarization: What Works and What's Next

1 code implementation10 Sep 2021 Yusen Zhang, Ansong Ni, Tao Yu, Rui Zhang, Chenguang Zhu, Budhaditya Deb, Asli Celikyilmaz, Ahmed Hassan Awadallah, Dragomir Radev

Dialogue summarization helps readers capture salient information from long conversations in meetings, interviews, and TV series.


EmailSum: Abstractive Email Thread Summarization

1 code implementation ACL 2021 Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao, Mohit Bansal

Furthermore, we find that widely used automatic evaluation metrics (ROUGE, BERTScore) are weakly correlated with human judgments on this email thread summarization task.

Abstractive Text Summarization Email Thread Summarization

Enriching Transformers with Structured Tensor-Product Representations for Abstractive Summarization

1 code implementation NAACL 2021 Yichen Jiang, Asli Celikyilmaz, Paul Smolensky, Paul Soulos, Sudha Rao, Hamid Palangi, Roland Fernandez, Caitlin Smith, Mohit Bansal, Jianfeng Gao

On several syntactic and semantic probing tasks, we demonstrate the emergent structural information in the role vectors and improved syntactic interpretability in the TPR layer outputs.

Abstractive Text Summarization

QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization

1 code implementation NAACL 2021 Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, Dragomir Radev

As increasing numbers of meetings are recorded and transcribed, meeting summaries have become essential to remind those who may or may not have attended the meetings about the key decisions made and the tasks to be completed.

Meeting Summarization

Data Augmentation for Abstractive Query-Focused Multi-Document Summarization

1 code implementation2 Mar 2021 Ramakanth Pasunuru, Asli Celikyilmaz, Michel Galley, Chenyan Xiong, Yizhe Zhang, Mohit Bansal, Jianfeng Gao

The progress in Query-focused Multi-Document Summarization (QMDS) has been limited by the lack of sufficient largescale high-quality training datasets.

Data Augmentation Document Summarization +1

The Amazing World of Neural Language Generation

no code implementations EMNLP 2020 Yangfeng Ji, Antoine Bosselut, Thomas Wolf, Asli Celikyilmaz

Neural Language Generation (NLG) {--} using neural network models to generate coherent text {--} is among the most promising methods for automated text creation.

Language Modelling Text Generation +1

RMM: A Recursive Mental Model for Dialogue Navigation

1 code implementation Findings of the Association for Computational Linguistics 2020 Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz, Jianfeng Gao

In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers.

Answer Generation Instruction Following

GO FIGURE: A Meta Evaluation of Factuality in Summarization

no code implementations Findings (ACL) 2021 Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, Jianfeng Gao

While neural language models can generate text with remarkable fluency and coherence, controlling for factual correctness in generation remains an open research question.

Common Sense Reasoning Document Summarization +1

Substance over Style: Document-Level Targeted Content Transfer

1 code implementation EMNLP 2020 Allison Hegel, Sudha Rao, Asli Celikyilmaz, Bill Dolan

Existing language models excel at writing from scratch, but many real-world scenarios require rewriting an existing document to fit a set of constraints.

Language Modelling Text Generation

Evaluation of Text Generation: A Survey

no code implementations26 Jun 2020 Asli Celikyilmaz, Elizabeth Clark, Jianfeng Gao

The paper surveys evaluation methods of natural language generation (NLG) systems that have been developed in the last few years.

Text Generation Text Summarization

Reparameterized Variational Divergence Minimization for Stable Imitation

no code implementations18 Jun 2020 Dilip Arumugam, Debadeepta Dey, Alekh Agarwal, Asli Celikyilmaz, Elnaz Nouri, Bill Dolan

While recent state-of-the-art results for adversarial imitation-learning algorithms are encouraging, recent works exploring the imitation learning from observation (ILO) setting, where trajectories \textit{only} contain expert observations, have not been met with the same success.

Continuous Control Imitation Learning

A Recipe for Creating Multimodal Aligned Datasets for Sequential Tasks

1 code implementation ACL 2020 Angela S. Lin, Sudha Rao, Asli Celikyilmaz, Elnaz Nouri, Chris Brockett, Debadeepta Dey, Bill Dolan

Learning to align these different instruction sets is challenging because: a) different recipes vary in their order of instructions and use of ingredients; and b) video instructions can be noisy and tend to contain far more information than text instructions.


RMM: A Recursive Mental Model for Dialog Navigation

1 code implementation2 May 2020 Homero Roman Roman, Yonatan Bisk, Jesse Thomason, Asli Celikyilmaz, Jianfeng Gao

In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers.

Answer Generation Instruction Following

PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking

2 code implementations EMNLP 2020 Hannah Rashkin, Asli Celikyilmaz, Yejin Choi, Jianfeng Gao

We propose the task of outline-conditioned story generation: given an outline as a set of phrases that describe key characters and events to appear in a story, the task is to generate a coherent narrative that is consistent with the provided outline.

Story Generation

AREDSUM: Adaptive Redundancy-Aware Iterative Sentence Ranking for Extractive Document Summarization

3 code implementations EACL 2021 Keping Bi, Rahul Jha, W. Bruce Croft, Asli Celikyilmaz

Redundancy-aware extractive summarization systems score the redundancy of the sentences to be included in a summary either jointly with their salience information or separately as an additional sentence scoring step.

Document Summarization Extractive Document Summarization +2

Working Memory Graphs

no code implementations ICML 2020 Ricky Loynd, Roland Fernandez, Asli Celikyilmaz, Adith Swaminathan, Matthew Hausknecht

Transformers have increasingly outperformed gated RNNs in obtaining new state-of-the-art results on supervised tasks involving text sequences.

Decision Making

Robust Navigation with Language Pretraining and Stochastic Sampling

1 code implementation IJCNLP 2019 Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi

Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments.

Vision and Language Navigation

Discourse Understanding and Factual Consistency in Abstractive Summarization

no code implementations EACL 2021 Saadia Gabriel, Antoine Bosselut, Jeff Da, Ari Holtzman, Jan Buys, Kyle Lo, Asli Celikyilmaz, Yejin Choi

We introduce a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary.

Abstractive Text Summarization

Sentence Mover's Similarity: Automatic Evaluation for Multi-Sentence Texts

no code implementations ACL 2019 Elizabeth Clark, Asli Celikyilmaz, Noah A. Smith

For evaluating machine-generated texts, automatic methods hold the promise of avoiding collection of human judgments, which can be expensive and time-consuming.

Semantic Similarity Semantic Textual Similarity +1

Learning Compressed Sentence Representations for On-Device Text Processing

1 code implementation ACL 2019 Dinghan Shen, Pengyu Cheng, Dhanasekar Sundararaman, Xinyuan Zhang, Qian Yang, Meng Tang, Asli Celikyilmaz, Lawrence Carin

Vector representations of sentences, trained on massive text corpora, are widely used as generic sentence embeddings across a variety of NLP problems.

Retrieval Sentence Embeddings

COMET: Commonsense Transformers for Automatic Knowledge Graph Construction

1 code implementation ACL 2019 Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, Yejin Choi

We present the first comprehensive study on automatic knowledge base construction for two prevalent commonsense knowledge graphs: ATOMIC (Sap et al., 2019) and ConceptNet (Speer et al., 2017).

graph construction Knowledge Graphs

Efficient Adaptation of Pretrained Transformers for Abstractive Summarization

2 code implementations1 Jun 2019 Andrew Hoang, Antoine Bosselut, Asli Celikyilmaz, Yejin Choi

Large-scale learning of transformer language models has yielded improvements on a variety of natural language understanding tasks.

Abstractive Text Summarization Natural Language Understanding

Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models

no code implementations ACL 2019 Dinghan Shen, Asli Celikyilmaz, Yizhe Zhang, Liqun Chen, Xin Wang, Jianfeng Gao, Lawrence Carin

Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables.

Text Generation

Hierarchically-Structured Variational Autoencoders for Long Text Generation

no code implementations27 Sep 2018 Dinghan Shen, Asli Celikyilmaz, Yizhe Zhang, Liqun Chen, Xin Wang, Lawrence Carin

Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation.

Text Generation

Hierarchically Structured Reinforcement Learning for Topically Coherent Visual Story Generation

no code implementations21 May 2018 Qiuyuan Huang, Zhe Gan, Asli Celikyilmaz, Dapeng Wu, Jian-Feng Wang, Xiaodong He

We propose a hierarchically structured reinforcement learning approach to address the challenges of planning for generating coherent multi-sentence stories for the visual storytelling task.

reinforcement-learning Reinforcement Learning (RL) +1

Deep Communicating Agents for Abstractive Summarization

no code implementations NAACL 2018 Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, Yejin Choi

We present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization.

Ranked #28 on Abstractive Text Summarization on CNN / Daily Mail (using extra training data)

Abstractive Text Summarization reinforcement-learning +1

Learning and analyzing vector encoding of symbolic representations

no code implementations10 Mar 2018 Roland Fernandez, Asli Celikyilmaz, Rishabh Singh, Paul Smolensky

We present a formal language with expressions denoting general symbol structures and queries which access information in those structures.

End-to-End Task-Completion Neural Dialogue Systems

13 code implementations IJCNLP 2017 Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, Asli Celikyilmaz

One of the major drawbacks of modularized task-completion dialogue systems is that each module is trained individually, which presents several challenges.


Scaffolding Networks: Incremental Learning and Teaching Through Questioning

no code implementations28 Feb 2017 Asli Celikyilmaz, Li Deng, Lihong Li, Chong Wang

We introduce a new paradigm of learning for reasoning, understanding, and prediction, as well as the scaffolding network to implement this paradigm.

Incremental Learning

Associative Adversarial Networks

no code implementations18 Nov 2016 Tarik Arici, Asli Celikyilmaz

In this work, we use Restricted Boltzmann Machines (RBMs) as a higher-level associative memory and learn the probability distribution for the high-level features generated by D. The associative memory samples its underlying probability distribution and G learns how to map these samples to data space.

Cannot find the paper you are looking for? You can Submit a new open access paper.