Search Results for author: Dan Roth

Found 263 papers, 76 papers with code

PerKGQA: Question Answering over Personalized Knowledge Graphs

no code implementations Findings (NAACL) 2022 Ritam Dutt, Kasturi Bhattacharjee, Rashmi Gangadharaiah, Dan Roth, Carolyn Rose

The above concerns motivate our question answer- ing setting over personalized knowledge graphs (PERKGQA) where each user has restricted access to their KG.

Knowledge Graphs Question Answering

There’s a Time and Place for Reasoning Beyond the Image

1 code implementation ACL 2022 Xingyu Fu, Ben Zhou, Ishaan Chandratreya, Carl Vondrick, Dan Roth

Images are often more significant than only the pixels to human eyes, as we can infer, associate, and reason with contextual information from other sources to establish a more complete picture.

Image Clustering

Understanding the Extent to which Content Quality Metrics Measure the Information Quality of Summaries

no code implementations CoNLL (EMNLP) 2021 Daniel Deutsch, Dan Roth

Reference-based metrics such as ROUGE or BERTScore evaluate the content quality of a summary by comparing the summary to a reference.

Question Answering

ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations

no code implementations EMNLP 2021 Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

While these tasks partially evaluate machines’ ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning.

Machine Reading Comprehension Natural Language Queries

Capturing the Content of a Document through Complex Event Identification

no code implementations *SEM (NAACL) 2022 Zheng Qi, Elior Sulem, Haoyu Wang, Xiaodong Yu, Dan Roth

We address this task as a pipeline, first predicting whether two granular events mentioned in the text belong to the same complex event, independently of their position in the text, and then using this to cluster them into complex events.

Representation Learning

Quantifying Clinical Outcome Measures in Patients with Epilepsy Using the Electronic Health Record

no code implementations BioNLP (ACL) 2022 Kevin Xie, Brian Litt, Dan Roth, Colin A. Ellis

A wealth of important clinical information lies untouched in the Electronic Health Record, often in the form of unstructured textual documents.

Text Summarization

Building Low-Resource NER Models Using Non-Speaker Annotations

no code implementations NAACL (DaSH) 2021 Tatiana Tsygankova, Francesca Marini, Stephen Mayhew, Dan Roth

In low-resource natural language processing (NLP), the key problems are a lack of target language training data, and a lack of native speakers to create it.

Low Resource Named Entity Recognition named-entity-recognition +2

What Do Users Care About? Detecting Actionable Insights from User Feedback

no code implementations NAACL (ACL) 2022 Kasturi Bhattacharjee, Rashmi Gangadharaiah, Kathleen McKeown, Dan Roth

Users often leave feedback on a myriad of aspects of a product which, if leveraged successfully, can help yield useful insights that can lead to further improvements down the line.

New Frontiers of Information Extraction

no code implementations NAACL (ACL) 2022 Muhao Chen, Lifu Huang, Manling Li, Ben Zhou, Heng Ji, Dan Roth

This tutorial targets researchers and practitioners who are interested in AI and ML technologies for structural information extraction (IE) from unstructured textual sources.

Yes, No or IDK: The Challenge of Unanswerable Yes/No Questions

no code implementations NAACL 2022 Elior Sulem, Jamaal Hay, Dan Roth

For example, given the context “She married a lawyer from New-York.”, we don’t know whether the answer to the question “Did she marry in New York?” is “Yes” or “No”.

Natural Language Understanding RTE

Few-Shot Novel Concept Learning for Semantic Parsing

no code implementations Findings (EMNLP) 2021 Soham Dan, Osbert Bastani, Dan Roth

This way the concept learning problem is naturally a program synthesis problem and our algorithm learns from a few examples to synthesize a program representing the novel concept.

Novel Concepts Program Synthesis +1

Compositional Data and Task Augmentation for Instruction Following

no code implementations Findings (EMNLP) 2021 Soham Dan, Xinran Han, Dan Roth

Executing natural language instructions in a physically grounded domain requires a model that understands both spatial concepts such as “left of” and “above”, and the compositional language used to identify landmarks and articulate instructions relative to them.

Instruction Following

On the Effects of Transformer Size on In- and Out-of-Domain Calibration

no code implementations Findings (EMNLP) 2021 Soham Dan, Dan Roth

To reduce the cost of training such large models, prior work has developed smaller, more compact models which achieves a significant speedup in training time while maintaining competitive accuracy to the original model on downstream tasks.

Do We Know What We Don’t Know? Studying Unanswerable Questions beyond SQuAD 2.0

no code implementations Findings (EMNLP) 2021 Elior Sulem, Jamaal Hay, Dan Roth

Understanding when a text snippet does not provide a sought after information is an essential part of natural language utnderstanding.

RTE

ExpertQA: Expert-Curated Questions and Attributed Answers

1 code implementation14 Sep 2023 Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, Dan Roth

As language models are adapted by a more sophisticated and diverse set of users, the importance of guaranteeing that they provide factually correct information supported by verifiable sources is critical across fields of study & professions.

Language Modelling

Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

no code implementations10 Aug 2023 Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang

We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data.

Data-to-Text Generation

Building Interpretable and Reliable Open Information Retriever for New Domains Overnight

no code implementations9 Aug 2023 Xiaodong Yu, Ben Zhou, Dan Roth

Information retrieval (IR) or knowledge retrieval, is a critical component for many down-stream tasks such as open-domain question answering (QA).

Information Retrieval Open-Domain Question Answering +3

On Regularization and Inference with Label Constraints

no code implementations8 Jul 2023 Kaifu Wang, Hangfeng He, Tin D. Nguyen, Piyush Kumar, Dan Roth

Prior knowledge and symbolic rules in machine learning are often expressed in the form of label constraints, especially in structured prediction problems.

Structured Prediction

The Integer Linear Programming Inference Cookbook

no code implementations30 Jun 2023 Vivek Srikumar, Dan Roth

At the end, we will see two worked examples to illustrate the use of these recipes.

Large Language Models as Sous Chefs: Revising Recipes with GPT-3

1 code implementation24 Jun 2023 Alyssa Hwang, Bryan Li, Zhaoyi Hou, Dan Roth

With their remarkably improved text generation and prompting capabilities, large language models can adapt existing written information into forms that are easier to use and understand.

Text Generation

On Learning Latent Models with Multi-Instance Weak Supervision

no code implementations23 Jun 2023 Kaifu Wang, Efi Tsamoura, Dan Roth

This condition non-trivially generalizes and relaxes the existing small ambiguity degree in the PLL literature, since we allow the transition to be deterministic.

Partial Label Learning Weakly-supervised Learning

Interpretable by Design Visual Question Answering

no code implementations24 May 2023 Xingyu Fu, Ben Zhou, Sihao Chen, Mark Yatskar, Dan Roth

Model interpretability has long been a hard problem for the AI community especially in the multimodal setting, where vision and language need to be aligned and reasoned at the same time.

Question Answering Visual Question Answering

Taxonomy Expansion for Named Entity Recognition

no code implementations22 May 2023 Karthikeyan K, Yogarshi Vyas, Jie Ma, Giovanni Paolini, Neha Anna John, Shuai Wang, Yassine Benajiba, Vittorio Castelli, Dan Roth, Miguel Ballesteros

We experiment with 6 diverse datasets and show that PLM consistently performs better than most other approaches (0. 5 - 2. 5 F1), including in novel settings for taxonomy expansion not considered in prior work.

named-entity-recognition Named Entity Recognition +2

Open-Domain Event Graph Induction for Mitigating Framing Bias

no code implementations22 May 2023 Siyi Liu, Hongming Zhang, Hongwei Wang, Kaiqiang Song, Dan Roth, Dong Yu

However, none of the existing methods have explicitly addressed the issue of framing bias that is inherent in news articles.

Towards Corpus-Scale Discovery of Selection Biases in News Coverage: Comparing What Sources Say About Entities as a Start

no code implementations6 Apr 2023 Sihao Chen, William Bruno, Dan Roth

To facilitate research in this domain, we propose and study a conceptual framework, where we compare how sources typically mention certain controversial entities, and use such as indicators for the sources' content selection preferences.

Representation Learning

Conversation Style Transfer using Few-Shot Learning

no code implementations16 Feb 2023 Shamik Roy, Raphael Shu, Nikolaos Pappas, Elman Mansimov, Yi Zhang, Saab Mansour, Dan Roth

Conventional text style transfer approaches focus on sentence-level style transfer without considering contextual information, and the style is described with attributes (e. g., formality).

Few-Shot Learning intent-classification +3

GLUECons: A Generic Benchmark for Learning Under Constraints

1 code implementation16 Feb 2023 Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng, Roshanak Mirzaee, Yue Zhang, Andrzej Uszok, Alexander Wan, Tanawan Premsri, Dan Roth, Parisa Kordjamshidi

Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models.

Rethinking with Retrieval: Faithful Large Language Model Inference

1 code implementation31 Dec 2022 Hangfeng He, Hongming Zhang, Dan Roth

To address this issue, we propose a novel post-processing approach, rethinking with retrieval (RR), which retrieves relevant external knowledge based on the decomposed reasoning steps obtained from the chain-of-thought (CoT) prompting.

Language Modelling Large Language Model +2

PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition

no code implementations21 Dec 2022 Sihao Chen, Senaka Buthpitiya, Alex Fabrikant, Dan Roth, Tal Schuster

As these propositions can carry different truth values in the context of a given premise, we argue for the need to recognize the textual entailment relation of each proposition in a sentence individually.

Natural Language Inference

ReCode: Robustness Evaluation of Code Generation Models

2 code implementations20 Dec 2022 Shiqi Wang, Zheng Li, Haifeng Qian, Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, Bing Xiang

Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation.

Code Generation

In and Out-of-Domain Text Adversarial Robustness via Label Smoothing

no code implementations20 Dec 2022 Yahan Yang, Soham Dan, Dan Roth, Insup Lee

Recently it has been shown that state-of-the-art NLP models are vulnerable to adversarial attacks, where the predictions of a model can be drastically altered by slight modifications to the input (such as synonym substitutions).

Adversarial Robustness

CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context

no code implementations20 Dec 2022 Yangruibo Ding, Zijian Wang, Wasi Uddin Ahmad, Murali Krishna Ramanathan, Ramesh Nallapati, Parminder Bhatia, Dan Roth, Bing Xiang

While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i. e., in-file context, but ignore the rich semantics in other files within the same project, i. e., cross-file context, a critical source of information that is especially useful in modern modular software development.

Code Completion

Generic Temporal Reasoning with Differential Analysis and Explanation

no code implementations20 Dec 2022 Yu Feng, Ben Zhou, Haoyu Wang, Helen Jin, Dan Roth

Temporal reasoning is the task of predicting temporal relations of event pairs.

Privacy Adhering Machine Un-learning in NLP

no code implementations19 Dec 2022 Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, Dan Roth

In several real world industry applications that use Machine Learning to build models on user data, such mandates require significant effort both in terms of data cleansing as well as model retraining while ensuring the models do not deteriorate in prediction quality due to removal of data.

QQP

Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale

1 code implementation18 Dec 2022 Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, Sravan Bodapati, Katrin Kirchhoff, Dan Roth

Using a 66 billion parameter language model (OPT-66B) across a diverse set of 14 downstream tasks, we find this is indeed the case: $\sim$70% of attention heads and $\sim$20% of feed forward networks can be removed with minimal decline in task performance.

Language Modelling Large Language Model

Investigating Fairness Disparities in Peer Review: A Language Model Enhanced Approach

1 code implementation7 Nov 2022 Jiayao Zhang, Hongming Zhang, Zhun Deng, Dan Roth

We distill several insights from our analysis on study the peer review process with the help of large LMs.

Fairness Language Modelling +1

Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Texts

no code implementations30 Oct 2022 Ben Zhou, Kyle Richardson, Xiaodong Yu, Dan Roth

Explicit decomposition modeling, which involves breaking down complex tasks into more straightforward and often more interpretable sub-tasks, has long been a central theme in developing robust and interpretable NLU systems.

Language Modelling Semantic Parsing +1

Multi-lingual Evaluation of Code Generation Models

1 code implementation26 Oct 2022 Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang

Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.

Code Completion Code Translation +1

On the Limitations of Reference-Free Evaluations of Generated Text

no code implementations22 Oct 2022 Daniel Deutsch, Rotem Dror, Dan Roth

There is significant interest in developing evaluation metrics which accurately estimate the quality of generated text without the aid of a human-written reference text, which can be time consuming and expensive to collect or entirely unavailable in online applications.

Machine Translation

Zero-Shot On-the-Fly Event Schema Induction

no code implementations12 Oct 2022 Rotem Dror, Haoyu Wang, Dan Roth

The answers to these questions can be found by collecting many documents on the complex event of interest, extracting relevant information, and analyzing it.

CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm

no code implementations12 Oct 2022 Hongming Zhang, Yintong Huo, Yanai Elazar, Yangqiu Song, Yoav Goldberg, Dan Roth

We first align commonsense tasks with relevant knowledge from commonsense knowledge bases and ask humans to annotate whether the knowledge is enough or not.

Question Answering

Instruction Tuning for Few-Shot Aspect-Based Sentiment Analysis

1 code implementation12 Oct 2022 Siddharth Varia, Shuai Wang, Kishaloy Halder, Robert Vacareanu, Miguel Ballesteros, Yassine Benajiba, Neha Anna John, Rishita Anubhai, Smaranda Muresan, Dan Roth

Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task which involves four elements from user-generated texts: aspect term, aspect category, opinion term, and sentiment polarity.

Aspect-Based Sentiment Analysis (ABSA) Few-Shot Learning +1

Cross-Lingual Speaker Identification Using Distant Supervision

1 code implementation11 Oct 2022 Ben Zhou, Dian Yu, Dong Yu, Dan Roth

Speaker identification, determining which character said each utterance in literary text, benefits many downstream tasks.

Language Modelling Speaker Identification

Are All Steps Equally Important? Benchmarking Essentiality Detection of Events

no code implementations8 Oct 2022 Hongming Zhang, Yueguan Wang, Yuqian Deng, Haoyu Wang, Muhao Chen, Dan Roth

In this paper, we seek to fill this gap by studying how well current models can understand the essentiality of different step events towards a goal event.

Benchmarking

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

1 code implementation9 Jun 2022 Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, ZiRui Wang, Ziyi Wu

BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.

Common Sense Reasoning Memorization

Repro: An Open-Source Library for Improving the Reproducibility and Usability of Publicly Available Research Code

1 code implementation29 Apr 2022 Daniel Deutsch, Dan Roth

We introduce Repro, an open-source library which aims at improving the reproducibility and usability of research code.

Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics

no code implementations NAACL 2022 Daniel Deutsch, Rotem Dror, Dan Roth

How reliably an automatic summarization evaluation metric replicates human judgments of summary quality is quantified by system-level correlations.

Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics

no code implementations Findings (ACL) 2022 Daniel Deutsch, Dan Roth

Question answering-based summarization evaluation metrics must automatically determine whether the QA model's prediction is correct or not, a task known as answer verification.

Benchmarking Question Answering

DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization

2 code implementations ACL 2022 Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth

Empirical analyses show that, despite the challenging nature of generative tasks, we were able to achieve a 16. 5x model footprint compression ratio with little performance drop relative to the full-precision counterparts on multiple summarization and QA datasets.

Knowledge Distillation Model Compression +2

There is a Time and Place for Reasoning Beyond the Image

1 code implementation1 Mar 2022 Xingyu Fu, Ben Zhou, Ishaan Preetam Chandratreya, Carl Vondrick, Dan Roth

For example, in Figure 1, we can find a way to identify the news articles related to the picture through segment-wise understandings of the signs, the buildings, the crowds, and more.

Image Clustering

Understanding Robust Generalization in Learning Regular Languages

no code implementations20 Feb 2022 Soham Dan, Osbert Bastani, Dan Roth

Currently, deep neural networks struggle to generalize robustly to such shifts in the data distribution.

ROCK: Causal Inference Principles for Reasoning about Commonsense Causality

1 code implementation31 Jan 2022 Jiayao Zhang, Hongming Zhang, Weijie J. Su, Dan Roth

Commonsense causality reasoning (CCR) aims at identifying plausible causes and effects in natural language descriptions that are deemed reasonable by an average person.

Causal Inference

Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval

2 code implementations28 Jan 2022 Uri Alon, Frank F. Xu, Junxian He, Sudipta Sengupta, Dan Roth, Graham Neubig

Retrieval-based language models (R-LM) model the probability of natural language text by combining a standard language model (LM) with examples retrieved from an external datastore at test time.

Language Modelling Retrieval

Event Linking: Grounding Event Mentions to Wikipedia

1 code implementation15 Dec 2021 Xiaodong Yu, Wenpeng Yin, Nitish Gupta, Dan Roth

Third, we retrain and evaluate two state-of-the-art (SOTA) entity linking models, showing the challenges of event linking, and we propose an event-specific linking system EVELINK to set a competitive result for the new task.

Entity Linking Natural Language Understanding

Learning Constraints and Descriptive Segmentation for Subevent Detection

no code implementations EMNLP 2021 Haoyu Wang, Hongming Zhang, Muhao Chen, Dan Roth

The task of subevent detection aims to resolve this granularity issue, recognizing the membership of multi-granular events in event complexes.

Descriptive Text Segmentation

Zero-shot Event Extraction via Transfer Learning: Challenges and Insights

no code implementations ACL 2021 Qing Lyu, Hongming Zhang, Elior Sulem, Dan Roth

Event extraction has long been a challenging task, addressed mostly with supervised methods that require expensive annotation and are not extensible to new event ontologies.

Event Extraction Natural Language Inference +2

Event-Centric Natural Language Processing

no code implementations ACL 2021 Muhao Chen, Hongming Zhang, Qiang Ning, Manling Li, Heng Ji, Kathleen McKeown, Dan Roth

This tutorial targets researchers and practitioners who are interested in AI technologies that help machines understand natural language text, particularly real-world events described in the text.

What is Your Article Based On? Inferring Fine-grained Provenance

no code implementations ACL 2021 Yi Zhang, Zachary Ives, Dan Roth

We experiment with a newly created evaluation dataset, Politi-Prov, based on fact-checking articles from \url{www. politifact. com}; our experimental results show that our solution leads to a significant improvement over baselines.

Fact Checking

MultiOpEd: A Corpus of Multi-Perspective News Editorials

1 code implementation NAACL 2021 Siyi Liu, Sihao Chen, Xander Uyttendaele, Dan Roth

We propose MultiOpEd, an open-domain news editorial corpus that supports various tasks pertaining to the argumentation structure in news editorials, focusing on automatic perspective discovery.

Multi-Task Learning

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System

1 code implementation NAACL 2021 Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, D{\'\i}dac Sur{\'\i}s, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji

We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video).

coreference-resolution Event Extraction

Event Time Extraction and Propagation via Graph Attention Networks

1 code implementation NAACL 2021 Haoyang Wen, Yanru Qu, Heng Ji, Qiang Ning, Jiawei Han, Avi Sil, Hanghang Tong, Dan Roth

Grounding events into a precise timeline is important for natural language understanding but has received limited attention in recent work.

Graph Attention Natural Language Understanding +2

Learning to Decompose and Organize Complex Tasks

1 code implementation NAACL 2021 Yi Zhang, Sujay Kumar Jauhar, Julia Kiseleva, Ryen White, Dan Roth

Both components of our graph induction solution are evaluated in experiments, demonstrating that our models outperform a state-of-the-art text generator significantly.

Management

Generalization in Instruction Following Systems

no code implementations NAACL 2021 Soham Dan, Michael Zhou, Dan Roth

Understanding and executing natural language instructions in a grounded domain is one of the hallmarks of artificial intelligence.

Data Augmentation Instruction Following

Weighted Training for Cross-Task Learning

1 code implementation ICLR 2022 Shuxiao Chen, Koby Crammer, Hangfeng He, Dan Roth, Weijie J. Su

In this paper, we introduce Target-Aware Weighted Training (TAWT), a weighted training algorithm for cross-task learning based on minimizing a representation-based task distance between the source and target tasks.

Chunking named-entity-recognition +5

Toward Code Generation: A Survey and Lessons from Semantic Parsing

no code implementations26 Apr 2021 Celine Lee, Justin Gottschlich, Dan Roth

With the growth of natural language processing techniques and demand for improved software engineering efficiency, there is an emerging interest in translating intention from human languages to programming languages.

Code Generation Program Synthesis +1

Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selection

no code implementations NAACL 2021 Sihao Chen, Fan Zhang, Kazoo Sone, Dan Roth

Despite significant progress in neural abstractive summarization, recent studies have shown that the current models are prone to generating summaries that are unfaithful to the original context.

Abstractive Text Summarization

ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning

1 code implementation16 Apr 2021 Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

While these tasks partially evaluate machines' ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning.

Machine Reading Comprehension Natural Language Queries +1

Learning to Reason for Text Generation from Scientific Tables

1 code implementation16 Apr 2021 Nafise Sadat Moosavi, Andreas Rücklé, Dan Roth, Iryna Gurevych

In this paper, we introduce SciGen, a new challenge dataset for the task of reasoning-aware data-to-text generation consisting of tables from scientific articles and their corresponding descriptions.

Arithmetic Reasoning Data-to-Text Generation

Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema

no code implementations EMNLP 2021 Yanai Elazar, Hongming Zhang, Yoav Goldberg, Dan Roth

To support this claim, we first show that the current evaluation method of WS is sub-optimal and propose a modification that uses twin sentences for evaluation.

Bias Detection Disentanglement +1

Paired Examples as Indirect Supervision in Latent Decision Models

no code implementations EMNLP 2021 Nitish Gupta, Sameer Singh, Matt Gardner, Dan Roth

Such an objective does not require external supervision for the values of the latent output, or even the end task, yet provides an additional training signal to that provided by individual training examples themselves.

Out-of-Distribution Generalization Question Answering +2

How Good (really) are Grammatical Error Correction Systems?

no code implementations EACL 2021 Alla Rozovskaya, Dan Roth

Standard evaluations of Grammatical Error Correction (GEC) systems make use of a fixed reference text generated relative to the original text; they show, even when using multiple references, that we have a long way to go.

Grammatical Error Correction

A Statistical Analysis of Summarization Evaluation Metrics using Resampling Methods

1 code implementation31 Mar 2021 Daniel Deutsch, Rotem Dror, Dan Roth

After evaluating which of the proposed methods is most appropriate for summarization through two simulation experiments, we analyze the results of applying these methods to several different automatic evaluation metrics across three sets of human annotations.

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies

1 code implementation6 Jan 2021 Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, Jonathan Berant

A key limitation in current datasets for multi-hop reasoning is that the required steps for answering the question are mentioned in it explicitly.

Question Answering StrategyQA

Coreference Reasoning in Machine Reading Comprehension

1 code implementation ACL 2021 Mingzhu Wu, Nafise Sadat Moosavi, Dan Roth, Iryna Gurevych

We propose a methodology for creating MRC datasets that better reflect the challenges of coreference reasoning and use it to create a sample evaluation set.

coreference-resolution Machine Reading Comprehension +2

Unsupervised Label-aware Event Trigger and Argument Classification

no code implementations30 Dec 2020 Hongming Zhang, Haoyu Wang, Dan Roth

Rather than relying on annotated data, our model matches the semantics of identified events with those of event type labels.

Classification Event Extraction +1

Learning Contextual Causality from Time-consecutive Images

1 code implementation13 Dec 2020 Hongming Zhang, Yintong Huo, Xinran Zhao, Yangqiu Song, Dan Roth

Compared with pure text-based approaches, learning causality from the visual signal has the following advantages: (1) Causality knowledge belongs to the commonsense knowledge, which is rarely expressed in the text but rich in videos; (2) Most events in the video are naturally time-ordered, which provides a rich resource for us to mine causality knowledge from; (3) All the objects in the video can be used as context to study the contextual property of causal relations.

QANom: Question-Answer driven SRL for Nominalizations

1 code implementation COLING 2020 Ayal Klein, Jonathan Mamou, Valentina Pyatkin, Daniela Stepanov, Hangfeng He, Dan Roth, Luke Zettlemoyer, Ido Dagan

We propose a new semantic scheme for capturing predicate-argument relations for nominalizations, termed QANom.

What do we expect from Multiple-choice QA Systems?

no code implementations Findings of the Association for Computational Linguistics 2020 Krunal Shah, Nitish Gupta, Dan Roth

The recent success of machine learning systems on various QA datasets could be interpreted as a significant improvement in models' language understanding abilities.

Multiple-choice Multiple Choice Question Answering (MCQA)

What Are You Trying to Do? Semantic Typing of Event Processes

no code implementations CONLL 2020 Muhao Chen, Hongming Zhang, Haoyu Wang, Dan Roth

This paper studies a new cognitively motivated semantic typing task, multi-axis event process typing, that, given anevent process, attempts to infer free-form typelabels describing (i) the type of action made bythe process and (ii) the type of object the pro-cess seeks to affect.

Learning-To-Rank Vocal Bursts Type Prediction

Temporal Reasoning on Implicit Events from Distant Supervision

no code implementations NAACL 2021 Ben Zhou, Kyle Richardson, Qiang Ning, Tushar Khot, Ashish Sabharwal, Dan Roth

We propose TRACIE, a novel temporal reasoning dataset that evaluates the degree to which systems understand implicit events -- events that are not mentioned explicitly in natural language text but can be inferred from it.

Natural Language Inference

Understanding the Extent to which Summarization Evaluation Metrics Measure the Information Quality of Summaries

1 code implementation23 Oct 2020 Daniel Deutsch, Dan Roth

Reference-based metrics such as ROUGE or BERTScore evaluate the content quality of a summary by comparing the summary to a reference.

Analogous Process Structure Induction for Sub-event Sequence Prediction

no code implementations EMNLP 2020 Hongming Zhang, Muhao Chen, Haoyu Wang, Yangqiu Song, Dan Roth

Computational and cognitive studies of event understanding suggest that identifying, comprehending, and predicting events depend on having structured representations of a sequence of events and on conceptualizing (abstracting) its components into (soft) event categories.

"What Are You Trying to Do?" Semantic Typing of Event Processes

no code implementations13 Oct 2020 Muhao Chen, Hongming Zhang, Haoyu Wang, Dan Roth

This paper studies a new cognitively motivated semantic typing task, multi-axis event process typing, that, given an event process, attempts to infer free-form type labels describing (i) the type of action made by the process and (ii) the type of object the process seeks to affect.

Learning-To-Rank Vocal Bursts Type Prediction

Joint Constrained Learning for Event-Event Relation Extraction

no code implementations EMNLP 2020 Haoyu Wang, Muhao Chen, Hongming Zhang, Dan Roth

Understanding natural language involves recognizing how multiple event mentions structurally and temporally interact with each other.

Event Relation Extraction Temporal Relation Extraction

Do Language Embeddings Capture Scales?

no code implementations EMNLP (BlackboxNLP) 2020 Xikun Zhang, Deepak Ramachandran, Ian Tenney, Yanai Elazar, Dan Roth

Pretrained Language Models (LMs) have been shown to possess significant linguistic, common sense, and factual knowledge.

Common Sense Reasoning

"I'd rather just go to bed": Understanding Indirect Answers

no code implementations7 Oct 2020 Annie Louis, Dan Roth, Filip Radlinski

We revisit a pragmatic inference problem in dialog: understanding indirect responses to questions.

Language Modelling Transfer Learning

Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior

1 code implementation Findings of the Association for Computational Linguistics 2020 Zi Lin, Jeremiah Zhe Liu, Zi Yang, Nan Hua, Dan Roth

Traditional (unstructured) pruning methods for a Transformer model focus on regularizing the individual weights by penalizing them toward zero.

Towards Question-Answering as an Automatic Metric for Evaluating the Content Quality of a Summary

2 code implementations1 Oct 2020 Daniel Deutsch, Tania Bedrax-Weiss, Dan Roth

A desirable property of a reference-based evaluation metric that measures the content quality of a summary is that it should estimate how much information that summary has in common with a reference.

Question Answering

Visual Pivoting for (Unsupervised) Entity Alignment

2 code implementations28 Sep 2020 Fangyu Liu, Muhao Chen, Dan Roth, Nigel Collier

This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs).

Ranked #3 on Multi-modal Entity Alignment on UMVM-oea-d-w-v1 (using extra training data)

Knowledge Graphs Multi-modal Entity Alignment

From Spatial Relations to Spatial Configurations

no code implementations LREC 2020 Soham Dan, Parisa Kordjamshidi, Julia Bonn, Archna Bhatia, Jon Cai, Martha Palmer, Dan Roth

To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with the fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.

Natural Language Understanding

Understanding Spatial Relations through Multiple Modalities

no code implementations LREC 2020 Soham Dan, Hangfeng He, Dan Roth

Recognizing spatial relations and reasoning about them is essential in multiple applications including navigation, direction giving and human-computer interaction in general.

Common Sense Reasoning Implicit Relations

SacreROUGE: An Open-Source Library for Using and Developing Summarization Evaluation Metrics

1 code implementation EMNLP (NLPOSS) 2020 Daniel Deutsch, Dan Roth

We present SacreROUGE, an open-source library for using and developing summarization evaluation metrics.

``Who said it, and Why?'' Provenance for Natural Language Claims

no code implementations ACL 2020 Yi Zhang, Zachary Ives, Dan Roth

In an era where generating content and publishing it is so easy, we are bombarded with information and are exposed to all kinds of claims, some of which do not always rank high on the truth scale.

Claim Verification Natural Language Inference

Commonsense Reasoning for Natural Language Processing

no code implementations ACL 2020 Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi, Dan Roth

We organize this tutorial to provide researchers with the critical foundations and recent advances in commonsense representation and reasoning, in the hopes of casting a brighter light on this promising area of future research.

Navigate

Building Low-Resource NER Models Using Non-Speaker Annotation

no code implementations17 Jun 2020 Tatiana Tsygankova, Francesca Marini, Stephen Mayhew, Dan Roth

In low-resource natural language processing (NLP), the key problems are a lack of target language training data, and a lack of native speakers to create it.

Low Resource Named Entity Recognition named-entity-recognition +2

Learnability with Indirect Supervision Signals

no code implementations NeurIPS 2020 Kaifu Wang, Qiang Ning, Dan Roth

Learning from indirect supervision signals is important in real-world AI applications when, often, gold labels are missing or too costly.

Generalization Bounds Multi-class Classification

Foreseeing the Benefits of Incidental Supervision

2 code implementations EMNLP 2021 Hangfeng He, Mingyuan Zhang, Qiang Ning, Dan Roth

Real-world applications often require improved models by leveraging a range of cheap incidental supervision signals.

Informativeness Learning Theory +4

Incidental Supervision: Moving beyond Supervised Learning

no code implementations25 May 2020 Dan Roth

Machine Learning and Inference methods have become ubiquitous in our attempt to induce more abstract representations of natural language text, visual scenes, and other messy, naturally occurring data, and support decisions that depend on it.

BIG-bench Machine Learning

Text Classification with Few Examples using Controlled Generalization

no code implementations NAACL 2019 Abhijit Mahabal, Jason Baldridge, Burcu Karagol Ayan, Vincent Perot, Dan Roth

Training data for text classification is often limited in practice, especially for applications with many output classes or involving many related classification problems.

General Classification text-classification +2

Context-Based Quotation Recommendation

no code implementations17 May 2020 Ansel MacLaughlin, Tao Chen, Burcu Karagol Ayan, Dan Roth

Our experiments confirm the strong performance of BERT-based methods on this task, which outperform bag-of-words and neural ranking baselines by more than 30% relative across all ranking metrics.

Open-Domain Question Answering

Design Challenges in Low-resource Cross-lingual Entity Linking

1 code implementation EMNLP 2020 Xingyu Fu, Weijia Shi, Xiaodong Yu, Zian Zhao, Dan Roth

Cross-lingual Entity Linking (XEL), the problem of grounding mentions of entities in a foreign language text into an English knowledge base such as Wikipedia, has seen a lot of research in recent years, with a range of promising techniques.

Cross-Lingual Entity Linking Entity Linking

TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions

no code implementations EMNLP 2020 Qiang Ning, Hao Wu, Rujun Han, Nanyun Peng, Matt Gardner, Dan Roth

A critical part of reading is being able to understand the temporal relationships between events described in a passage of text, even when those relationships are not explicitly stated.

Machine Reading Comprehension Question Answering

Cross-lingual Entity Alignment with Incidental Supervision

1 code implementation EACL 2021 Muhao Chen, Weijia Shi, Ben Zhou, Dan Roth

Much research effort has been put to multilingual knowledge graph (KG) embedding methods to address the entity alignment task, which seeks to match entities in different languagespecific KGs that refer to the same real-world object.

Entity Alignment Knowledge Graphs

TransOMCS: From Linguistic Graphs to Commonsense Knowledge

1 code implementation1 May 2020 Hongming Zhang, Daniel Khashabi, Yangqiu Song, Dan Roth

Commonsense knowledge acquisition is a key problem for artificial intelligence.

Cross-Lingual Ability of Multilingual BERT: An Empirical Study

no code implementations ICLR 2020 Karthikeyan K, Zihan Wang, Stephen Mayhew, Dan Roth

Recent work has exhibited the surprising cross-lingual abilities of multilingual BERT (M-BERT) -- surprising since it is trained without any cross-lingual objective and with no aligned data.

named-entity-recognition Named Entity Recognition +2

Robust Named Entity Recognition with Truecasing Pretraining

no code implementations15 Dec 2019 Stephen Mayhew, Nitish Gupta, Dan Roth

Although modern named entity recognition (NER) systems show impressive performance on standard datasets, they perform poorly when presented with noisy data.

named-entity-recognition Named Entity Recognition +1

Neural Module Networks for Reasoning over Text

2 code implementations ICLR 2020 Nitish Gupta, Kevin Lin, Dan Roth, Sameer Singh, Matt Gardner

Answering compositional questions that require multiple steps of reasoning against text is challenging, especially when they involve discrete, symbolic operations.

Inductive Bias

Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks

no code implementations CL 2019 Mrinmaya Sachan, Avinava Dubey, Eduard H. Hovy, Tom M. Mitchell, Dan Roth, Eric P. Xing

At the same time, these help the readers pick up the structure of the discourse and comprehend the conveyed information.

Not All Claims are Created Equal: Choosing the Right Statistical Approach to Assess Hypotheses

1 code implementation ACL 2020 Erfan Sadeqi Azer, Daniel Khashabi, Ashish Sabharwal, Dan Roth

Empirical research in Natural Language Processing (NLP) has adopted a narrow set of principles for assessing hypotheses, relying mainly on p-value computation, which suffers from several known issues.

Bayesian Inference Misconceptions

Summary Cloze: A New Task for Content Selection in Topic-Focused Summarization

no code implementations IJCNLP 2019 Daniel Deutsch, Dan Roth

A key challenge in topic-focused summarization is determining what information should be included in the summary, a problem known as content selection.

Named Entity Recognition with Partially Annotated Training Data

no code implementations CONLL 2019 Stephen Mayhew, Snigdha Chaturvedi, Chen-Tse Tsai, Dan Roth

Supervised machine learning assumes the availability of fully-labeled data, but in many cases, such as low-resource languages, the only data available is partially annotated.

named-entity-recognition Named Entity Recognition +1