Search Results for author:
Found 188 papers, 88 papers with code
Exposing the Limits of Video-Text Models through Contrast Sets
We test the robustness of recent methods on the proposed automatic contrast sets, and compare them to additionally collected human-generated counterparts, to assess their effectiveness.
Reframing Instructional Prompts to GPTk’s Language
Our experiments compare the zero-shot and few-shot performance of LMs prompted with reframed instructions on 12 NLP tasks across 6 categories.
Misinfo Reaction Frames: Reasoning about Readers’ Reactions to News Headlines
Even to a simple and short news headline, readers react in a multitude of ways: cognitively (e. g. inferring the writer’s intent), emotionally (e. g. feeling distrust), and behaviorally (e. g. sharing the news with their friends).
Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right
Large language models have shown promising results in zero-shot settings.
proScript: Partially Ordered Scripts Generation
Scripts – prototypical event sequences describing everyday activities – have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information.
Do Embodied Agents Dream of Pixelated Sheep?: Embodied Decision Making using Language Guided World Modelling
Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world, which makes learning complex tasks with sparse rewards difficult.
MAUVE Scores for Generative Models: Theory and Practice
We present MAUVE, a family of comparison measures between pairs of distributions such as those encountered in the generative modeling of text or images.
Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts
Text detoxification has the potential to mitigate the harms of toxicity by rephrasing text to remove offensive meaning, but subtle toxicity remains challenging to tackle.
Reinforced Clarification Question Generation with Defeasibility Rewards for Disambiguating Social and Moral Situations
Context is vital for commonsense moral reasoning.
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization
We present SODA: the first publicly available, million-scale high-quality social dialogue dataset.
An AI Dungeon Master's Guide: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons
We propose a novel task, G4C (Goal-driven Guidance Generation in Grounded Communication), for studying goal-driven and grounded natural language interactions.
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
The key intellectual question we ask here is whether it is possible, if at all, to design a learning algorithm that does not benefit from scale, yet leads to a competitive level of commonsense acquisition.
Statistical and Computational Guarantees for Influence Diagnostics
Influence diagnostics such as influence functions and approximate maximum influence perturbations are popular in machine learning and in AI domain applications.
NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?
While a substantial body of prior work has explored adversarial example generation for natural language understanding tasks, these examples are often unrealistic and diverge from the real-world data distributions.
Generating Sequences by Learning to Self-Correct
Sequence generation applications require satisfying semantic constraints, such as ensuring that programs are correct, using certain keywords, or avoiding undesirable content.
Referee: Reference-Free Sentence Summarization with Sharper Controllability through Symbolic Knowledge Distillation
Moreover, we uniquely propose iterative distillation of knowledge, where student models from the previous iteration of distillation serve as teacher models in the next iteration.
Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs
We show that one of today's largest language models (GPT-3; Brown et al., 2020) lacks this kind of social intelligence out-of-the box, using two tasks: SocialIQa (Sap et al., 2019), which measures models' ability to understand intents and reactions of participants of social interactions, and ToMi (Le et al., 2019), which measures whether models can infer mental states and realities of participants of situations.
NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentation
While counterfactual data augmentation offers a promising step towards robust generalization in natural language processing, producing a set of counterfactuals that offer valuable inductive bias for models remains a challenge.
REV: Information-Theoretic Evaluation of Free-Text Rationales
While existing metrics have mostly focused on measuring the direct association between the rationale and a given label, we argue that an ideal metric should also be able to focus on the new information uniquely provided in the rationale that is otherwise not provided in the input or the label.
Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering
Our work is the first to report that knowledge generated by models that are orders of magnitude smaller than GPT-3, even without direct supervision on the knowledge itself, can exceed the quality of commonsense knowledge elicited from GPT-3.
Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
To help answer this, we first introduce an open-source modular library, RL4LMs (Reinforcement Learning for Language Models), for optimizing language generators with RL.
Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest
We challenge AI models to "demonstrate understanding" of the sophisticated multimodal humor of The New Yorker Caption Contest.
RealTime QA: What's the Answer Right Now?
We introduce RealTime QA, a dynamic question answering (QA) platform that announces questions and evaluates systems on a regular basis (weekly in this version).
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
Quark: Controllable Text Generation with Reinforced Unlearning
Large-scale language models often learn behaviors that are misaligned with user expectations.
NaturalProver: Grounded Mathematical Proof Generation with Language Models
Theorem proving in natural mathematical language - the mixture of symbolic and natural language used by humans - plays a central role in mathematical advances and education, and tests aspects of reasoning that are core to intelligence.
Multimodal Knowledge Alignment with Reinforcement Learning
Large language models readily adapt to novel settings, even without task-specific training data.
ProsocialDialog: A Prosocial Backbone for Conversational Agents
With this dataset, we introduce a dialogue safety detection module, Canary, capable of generating RoTs given conversational context, and a socially-informed dialogue agent, Prost.
Ranked #1 on
Dialogue Safety Prediction
on ProsocialDialog
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations
Despite their impressive capabilities, large pre-trained language models (LMs) struggle with consistent reasoning; recently, prompting LMs to generate explanations that self-guide the inference has emerged as a promising direction to amend this.
Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions
Generics express generalizations about the world (e. g., birds can fly) that are not universally true (e. g., newborn birds and penguins cannot fly).
Twist Decoding: Diverse Generators Guide Each Other
Our extensive evaluations on machine translation and scientific paper summarization demonstrate that Twist decoding substantially outperforms each model decoded in isolation over various scenarios, including cases where domain-specific and general-purpose models are both available.
Aligning to Social Norms and Values in Interactive Narratives
We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games -- environments wherein an agent perceives and interacts with a world through natural language.
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the remaining unseen ones.
Beam Decoding with Controlled Patience
Text generation with beam search has proven successful in a wide range of applications.
Probing Factually Grounded Content Transfer with Factual Ablation
Particularly, this domain allows us to introduce the notion of factual ablation for automatically measuring factual consistency: this captures the intuition that the model should be less likely to produce an output given a less relevant grounding document.
Faking Fake News for Real Fake News Detection: Propaganda-loaded Training Data Generation
While there has been a lot of research and many recent advances in neural fake news detection, defending against human-written disinformation remains underexplored.
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics
Many applications of text generation require incorporating different constraints to control the semantics or style of generated text.
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning
We present Sherlock, an annotated corpus of 103K images for testing machine capacity for abductive reasoning beyond literal image contents.
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
Starting with an existing dataset, MultiNLI for natural language inference (NLI), our approach uses dataset cartography to automatically identify examples that demonstrate challenging reasoning patterns, and instructs GPT-3 to compose new examples with similar patterns.
CommonsenseQA 2.0: Exposing the Limits of AI through Gamification
Constructing benchmarks that test the abilities of modern natural language understanding models is difficult - pre-trained language models exploit artifacts in benchmarks to achieve human parity, but still fail on adversarial examples and make errors that demonstrate a lack of common sense.
Imagined versus Remembered Stories: Quantifying Differences in Narrative Flow
We quantify the differences between autobiographical and imagined stories by introducing sequentiality, a measure of narrative flow of events, drawing probabilistic inferences from a cutting-edge large language model (GPT-3).
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound
Given a video, we replace snippets of text and audio with a MASK token; the model learns by choosing the correct masked-out snippet.
Ranked #4 on
Action Classification
on Kinetics-600
(using extra training data)
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer
In a difficult zero-shot setting with no paired audio-text data, our model demonstrates state-of-the-art zero-shot performance on the ESC50 and US8K audio classification tasks, and even surpasses the supervised state of the art for Clotho caption retrieval (with audio queries) by 2. 2\% R@1.
Reframing Human-AI Collaboration for Generating Free-Text Explanations
We create a pipeline that combines GPT-3 with a supervised filter that incorporates binary acceptability judgments from humans in the loop.
NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics
To enable constrained generation, we build on NeuroLogic decoding (Lu et al., 2021), combining its flexibility in incorporating logical constraints with A*esque estimates of future constraint satisfaction.
Ranked #1 on
Text Generation
on ROCStories
Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts
Fine-tuning continuous prompts for target tasks has recently emerged as a compact alternative to full model fine-tuning.
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand
We therefore propose a generalization of leaderboards, bidimensional leaderboards (Billboards), that simultaneously tracks progress in language generation models and metrics for their evaluation.
Transparent Human Evaluation for Image Captioning
We establish THumB, a rubric-based human evaluation protocol for image captioning models.
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection
The perceived toxicity of language can vary based on someone's identity and beliefs, but this variation is often ignored when collecting toxic language datasets, resulting in dataset and model biases.
Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information
However, this comparison provides little understanding of how difficult each instance in a given distribution is, or what attributes make the dataset difficult for a given model.
Generated Knowledge Prompting for Commonsense Reasoning
It remains an open question whether incorporating external knowledge benefits commonsense reasoning while maintaining the flexibility of pretrained sequence models.
Can Machines Learn Morality? The Delphi Experiment
As AI systems become increasingly powerful and pervasive, there are growing concerns about machines' morality or a lack thereof.
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
We apply this to the ATOMIC resource, and share our new symbolic knowledge graph and commonsense models.
Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematics
Neural sequence models trained with maximum likelihood estimation have led to breakthroughs in many tasks, where success is defined by the gap between training and test performance.
Out-of-Distribution Generalization
Systematic Generalization
Conversational Multi-Hop Reasoning with Neural Commonsense Knowledge and Symbolic Logic Rules
Our reasoner uses a state-of-the-art transformer-based generative commonsense knowledge base (KB) as its source of background knowledge for reasoning.
Reframing Instructional Prompts to GPTk's Language
Our experiments compare the zero-shot and few-shot performance of LMs prompted with reframed instructions on 12 NLP tasks across 6 categories.
It's not Rocket Science : Interpreting Figurative Language in Narratives
Figurative language is ubiquitous in English.
Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformation
Understanding manipulated media, from automatically generated {`}deepfakes{'} to manually edited ones, raises novel research challenges.
Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text
To support the broad range of real machine errors that can be identified by laypeople, the ten error categories of Scarecrow -- such as redundancy, commonsense errors, and incoherence -- are identified through several rounds of crowd annotation experiments without a predefined ontology.
Divergence Frontiers for Generative Models: Sample Complexity, Quantization Effects, and Frontier Integrals
The spectacular success of deep generative models calls for quantitative tools to measure their statistical performance.
TIMEDIAL: Temporal Commonsense Reasoning in Dialog
In this paper, we present the first study to investigate pre-trained LMs for their temporal reasoning capabilities in dialogs by introducing a new task and a crowd-sourced English challenge set, TIMEDIAL.
MERLOT: Multimodal Neural Script Knowledge Models
As humans, we understand events in the visual world contextually, performing multimodal reasoning across time to make inferences about the past, present, and future.
``I'm Not Mad'': Commonsense Implications of Negation and Contradiction
In this paper, we present the first comprehensive study focusing on commonsense implications of negated statements and contradictions.
PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World
We propose PIGLeT: a model that learns physical commonsense knowledge through interaction, and then uses this knowledge to ground language.
DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts
Despite recent advances in natural language generation, it remains challenging to control attributes of generated text.
CLIPScore: A Reference-free Evaluation Metric for Image Captioning
Image captioning has conventionally relied on reference-based automatic evaluations, where machine captions are compared against captions written by humans.
Ranked #1 on
Hallucination Pair-wise Detection (4-ref)
on FOIL
Hallucination Pair-wise Detection (1-ref)
Hallucination Pair-wise Detection (4-ref)
+3
Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlines
We propose Misinfo Reaction Frames (MRF), a pragmatic formalism for modeling how readers might react to a news headline.
Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
Large language models have shown promising results in zero-shot settings (Brown et al., 2020; Radford et al., 2019).
proScript: Partially Ordered Scripts Generation via Pre-trained Language Models
Scripts - standardized event sequences describing typical everyday activities - have been shown to help understand narratives by providing expectations, resolving ambiguity, and filling in unstated information.
"I'm Not Mad": Commonsense Implications of Negation and Contradiction
In this paper, we present the first comprehensive study focusing on commonsense implications of negated statements and contradictions.
NaturalProofs: Mathematical Theorem Proving in Natural Language
Understanding and creating mathematics using natural mathematical language - the mixture of symbolic and natural language used by humans - is a challenging and important problem for driving progress in machine learning.
UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark
First, we propose a new multitask benchmark, RAINBOW, to promote research on commonsense models that generalize well over multiple tasks and datasets.
Contrastive Explanations for Model Interpretability
Our method is based on projecting model representation to a latent space that captures only the features that are useful (to the model) to differentiate two potential decisions.
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
As major progress is made in open-ended text generation, measuring how close machine-generated text is to human language remains a critical open problem.
MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversations
We study conversational dialog in which there are many possible responses to a given history.
Challenges in Automated Debiasing for Toxic Language Detection
Overall, our findings show that debiasing a model trained on biased toxic language data is not as effective as simply relabeling the data to remove existing biases.
GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation
While often assumed a gold standard, effective human evaluation of text generation remains an important, open area for research.
On-the-Fly Attention Modulation for Neural Generation
Despite considerable advancements with deep neural language models (LMs), neural text generation still suffers from degeneration: the generated text is repetitive, generic, self-contradictory, and often lacks commonsense.
VinVL: Revisiting Visual Representations in Vision-Language Models
In our experiments we feed the visual features generated by the new object detection model into a Transformer-based VL fusion model \oscar \cite{li2020oscar}, and utilize an improved approach \short\ to pre-train the VL model and fine-tune it on a wide range of downstream VL tasks.
Ranked #9 on
Visual Question Answering
on VQA v2 test-std
Analyzing Commonsense Emergence in Few-shot Knowledge Models
Our results show that commonsense knowledge models can rapidly adapt from limited examples, indicating that KG fine-tuning serves to learn an interface to encoded knowledge learned during pretraining.
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences
In social settings, much of human behavior is governed by unspoken rules of conduct.
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
In this paper, we investigate the extent to which neural models can reason about natural language rationales that explain model predictions, relying only on distant supervision with no additional annotation cost for human-written rationales.
Edited Media Understanding: Reasoning About Implications of Manipulated Images
The difference between this example, and harmful edits that spread disinformation, is one of intent.
Do Neural Language Models Overcome Reporting Bias?
Mining commonsense knowledge from corpora suffers from reporting bias, over-representing the rare at the expense of the trivial (Gordon and Van Durme, 2013).
Thinking Like a Skeptic: Defeasible Inference in Natural Language
Defeasible inference is a mode of reasoning in which an inference (X is a bird, therefore X flies) may be weakened or overturned in light of new evidence (X is a penguin).
Social Chemistry 101: Learning to Reason about Social and Moral Norms
We present Social Chemistry, a new conceptual formalism to study people's everyday social norms and moral judgments over a rich spectrum of real life situations described in natural language.
PowerTransformer: Unsupervised Controllable Revision for Biased Language Correction
Unconscious biases continue to be prevalent in modern text and media, calling for algorithms that can assist writers with bias correction.
GO FIGURE: A Meta Evaluation of Factuality in Summarization
While neural language models can generate text with remarkable fluency and coherence, controlling for factual correctness in generation remains an open research question.
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
While the dominant recipe for conditional text generation has been large-scale pretrained language models that are finetuned on the task-specific training data, such models do not learn to follow the underlying constraints reliably, even when supervised with large amounts of task-specific examples.
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models
In this paper, we present Reflective Decoding, a novel unsupervised algorithm that allows for direct application of unidirectional LMs to non-sequential tasks.
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs
Natural language rationales could provide intuitive, higher-level explanations that are easily understandable by humans, complementing the more broadly studied lower-level explanations based on gradients or attention weights.
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning
Abductive and counterfactual reasoning, core abilities of everyday human cognition, require reasoning about what might have happened at time t, while conditioning on multiple contexts from the relative past and future.
COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs
Next, we show that ATOMIC 2020 is better suited for training knowledge models that can generate accurate, representative knowledge for new, unseen entities and events.
Paragraph-level Commonsense Transformers with Recurrent Memory
Human understanding of narrative texts requires making commonsense inferences beyond what is stated explicitly in the text.
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
We investigate the extent to which pretrained LMs can be prompted to generate toxic language, and the effectiveness of controllable text generation algorithms at preventing such toxic degeneration.
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
Experiments across four datasets show that these model-dependent measures reveal three distinct regions in the data map, each with pronounced characteristics.
Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes
As AI systems become an increasing part of people's everyday lives, it becomes ever more important that they understand people's ethical norms.
Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models
We introduce a measure of narrative flow and use this to examine the narratives for imagined and recalled events.
Commonsense Reasoning for Natural Language Processing
We organize this tutorial to provide researchers with the critical foundations and recent advances in commonsense representation and reasoning, in the hopes of casting a brighter light on this promising area of future research.
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
We propose the task of outline-conditioned story generation: given an outline as a set of phrases that describe key characters and events to appear in a story, the task is to generate a coherent narrative that is consistent with the provided outline.
Generative Data Augmentation for Commonsense Reasoning
Recent advances in commonsense reasoning depend on large-scale human-annotated training data to achieve peak performance.
Ranked #1 on
Question Answering
on CODAH
VisualCOMET: Reasoning about the Dynamic Context of a Still Image
In addition, we provide person-grounding (i. e., co-reference links) between people appearing in the image and people mentioned in the textual commonsense descriptions, allowing for tighter integration between images and text.
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks.
Ranked #1 on
Image Retrieval
on COCO
(Recall@10 metric)
Unsupervised Commonsense Question Answering with Self-Talk
Natural language understanding involves reading between the lines with implicit background knowledge.
TuringAdvice: A Generative and Dynamic Evaluation of Language Use
We propose TuringAdvice, a new challenge task and dataset for language understanding models.
Procedural Reading Comprehension with Attribute-Aware Context Flow
Procedural texts often describe processes (e. g., photosynthesis and cooking) that happen over entities (e. g., light, food).
Multi-View Learning for Vision-and-Language Navigation
Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified.
Adversarial Filters of Dataset Biases
Large neural models have demonstrated human-level performance on language and vision benchmarks, while their performance degrades considerably on adversarial or out-of-distribution samples.
PIQA: Reasoning about Physical Commonsense in Natural Language
Questions requiring this kind of physical commonsense pose a challenge to today's natural language understanding systems.
Natural Language Understanding
Physical Commonsense Reasoning
+2
Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
Understanding narratives requires reasoning about implicit world knowledge related to the causes, effects, and states of situations described in text.
Social Bias Frames: Reasoning about Social and Power Implications of Language
We introduce Social Bias Frames, a new conceptual formalism that aims to model the pragmatic frames in which people project social biases and stereotypes onto others.
CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
In this paper, we present a constrained text generation task, CommonGen associated with a benchmark dataset, to explicitly test machines for the ability of generative commonsense reasoning.
Ranked #1 on
Text Generation
on CommonGen
Social IQa: Commonsense Reasoning about Social Interactions
We introduce Social IQa, the first large-scale benchmark for commonsense reasoning about social situations.
Commonsense Knowledge Base Completion with Structural and Semantic Context
Our results demonstrate the effectiveness of language model representations in boosting link prediction performance and the advantages of learning from local graph structure (+1. 5 points in MRR for ConceptNet) when training on subgraphs for computational efficiency.
BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
In this paper, we propose a novel approach to unsupervised sentence summarization by mapping the Information Bottleneck principle to a conditional language modelling objective: given a sentence, our approach seeks a compressed sentence that can best predict the next sentence.
Counterfactual Story Reasoning and Generation
Counterfactual reasoning requires predicting how alternative events, contrary to what actually happened, might have resulted in different outcomes.
Robust Navigation with Language Pretraining and Stochastic Sampling
Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments.
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
In this paper, we introduce Cosmos QA, a large-scale dataset of 35, 600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions.
Do Neural Language Representations Learn Physical Commonsense?
Humans understand language based on the rich background knowledge about how the physical world works, which in turn allows us to reason about the physical world through language.
WinoGrande: An Adversarial Winograd Schema Challenge at Scale
The key steps of the dataset construction consist of (1) a carefully designed crowdsourcing procedure, followed by (2) systematic bias reduction using a novel AfLite algorithm that generalizes human-detectable word associations to machine-detectable embedding associations.
Discourse Understanding and Factual Consistency in Abstractive Summarization
We introduce a general framework for abstractive summarization with factual consistency and distinct modeling of the narrative flow in an output summary.
The Risk of Racial Bias in Hate Speech Detection
We investigate how annotators{'} insensitivity to differences in dialect can lead to racial bias in automatic hate speech detection models, potentially amplifying harm against minority populations.
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction
We present the first comprehensive study on automatic knowledge base construction for two prevalent commonsense knowledge graphs: ATOMIC (Sap et al., 2019) and ConceptNet (Speer et al., 2017).
Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading
Although neural conversation models are effective in learning how to produce fluent responses, their primary challenge lies in knowing what to say to make the conversation contentful and non-vacuous.
Benchmarking Hierarchical Script Knowledge
Understanding procedural language requires reasoning about both hierarchical and temporal relations between events.
Efficient Adaptation of Pretrained Transformers for Abstractive Summarization
Large-scale learning of transformer language models has yielded improvements on a variety of natural language understanding tasks.
Abstractive Text Summarization
Natural Language Understanding
MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms
We introduce a new representation language to model precise operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models.
Defending Against Neural Fake News
We find that best current discriminators can classify neural fake news from real, human-written, news with 73% accuracy, assuming access to a moderate level of training data.
Ranked #2 on
Fake News Detection
on Grover-Mega
HellaSwag: Can a Machine Really Finish Your Sentence?
In this paper, we show that commonsense inference still proves difficult for even state-of-the-art models, by presenting HellaSwag, a new challenge dataset.
SocialIQA: Commonsense Reasoning about Social Interactions
We introduce Social IQa, the first largescale benchmark for commonsense reasoning about social situations.
The Curious Case of Neural Text Degeneration
Despite considerable advancements with deep neural language models, the enigma of neural text degeneration persists when these models are tested as text generators.
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
We present the Frontier Aware Search with backTracking (FAST) Navigator, a general framework for action decoding, that achieves state-of-the-art results on the Room-to-Room (R2R) Vision-and-Language navigation challenge of Anderson et.
Ranked #3 on
Vision-Language Navigation
on Room2Room
DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension
We present DREAM, the first dialogue-based multiple-choice reading comprehension data set.
DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension
DREAM is likely to present significant challenges for existing reading comprehension systems: 84% of answers are non-extractive, 85% of questions require reasoning beyond a single sentence, and 34% of questions also involve commonsense knowledge.
From Recognition to Cognition: Visual Commonsense Reasoning
While this task is easy for humans, it is tremendously difficult for today's vision systems, requiring higher-order cognition and commonsense reasoning about the world.
Multiple-choice
Multiple Choice Question Answering (MCQA)
+1
Early Fusion for Goal Directed Robotic Vision
Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline.
ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning
We present ATOMIC, an atlas of everyday commonsense reasoning, organized through 877k textual descriptions of inferential knowledge.
QuAC: Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
Neural Metaphor Detection in Context
We present end-to-end neural models for detecting metaphorical word use in context.