Search Results for author:
Found 55 papers, 32 papers with code
Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection
Even though it is such a fundamental tool in NLP, crowdsourcing use is largely guided by common practices and the personal experience of researchers.
Retouchdown: Releasing Touchdown on StreetLearn as a Public Resource for Language Grounding Tasks in Street View
These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.
Continually Improving Extractive QA via Human Feedback
We study continually improving an extractive question answering (QA) system via human user feedback.
Semantic uncertainty guides the extension of conventions to new referents
A long tradition of studies in psycholinguistics has examined the formation and generalization of ad hoc conventions in reference games, showing how newly acquired conventions for a given target transfer to new referential contexts.
CB2: Collaborative Natural Language Interaction Research Platform
CB2 is a multi-agent platform to study collaborative natural language interaction in a grounded task-oriented scenario.
Continual Learning for Instruction Following from Realtime Feedback
We study the problem of continually training an instruction-following agent through feedback provided by users during collaborative interactions.
Abstract Visual Reasoning with Tangram Shapes
We introduce KiloGram, a resource for studying abstract visual reasoning in humans and machines.
lilGym: Natural Language Visual Reasoning with Reinforcement Learning
We present lilGym, a new benchmark for language-conditioned reinforcement learning in visual environments.
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
We introduce Wav2Seq, the first self-supervised approach to pre-train both parts of encoder-decoder models for speech data.
Ranked #3 on
Named Entity Recognition (NER)
on SLUE
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Simulating Bandit Learning from User Feedback for Extractive Question Answering
We study learning from user feedback for extractive question answering by simulating feedback using supervised data.
SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speech
Historically these have focused on automatic speech recognition (ASR), speaker identification, or other lower-level tasks.
Ranked #1 on
Named Entity Recognition (NER)
on SLUE
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
When in Doubt: Improving Classification Performance with Alternating Normalization
We introduce Classification with Alternating Normalization (CAN), a non-parametric post-processing step for classification.
Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition
This paper is a study of performance-efficiency trade-offs in pre-trained models for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Analysis of Language Change in Collaborative Instruction Following
We analyze language change over time in a collaborative, goal-oriented instructional task, where utility-maximizing participants form conventions and increase their expertise.
Who's Waldo? Linking People Across Text and Images
We present a task and benchmark dataset for person-centric visual grounding, the problem of linking between people named in a caption and people pictured in an image.
Ranked #1 on
Person-centric Visual Grounding
on Who’s Waldo
(using extra training data)
Continual Learning for Grounded Instruction Generation by Observing Human Following Behavior
We study continual learning for natural language instruction generation, by observing human users' instruction execution.
A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution
Natural language provides an accessible and expressive interface to specify long-term tasks for robotic agents.
Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following
We study the problem of learning a robot policy to follow natural language instructions that can be easily extended to reason about new objects.
Evaluating NLP Models via Contrast Sets
Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.
Revisiting Few-sample BERT Fine-tuning
We empirically test the impact of these factors, and identify alternative practices that resolve the commonly observed instability of the process.
What is Learned in Visually Grounded Neural Syntax Acquisition
Visual features are a promising signal for learning bootstrap textual models.
Evaluating Models' Local Decision Boundaries via Contrast Sets
Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.
Retouchdown: Adding Touchdown to StreetLearn as a Shareable Resource for Language Grounding Tasks in Street View
These have been added to the StreetLearn dataset and can be obtained via the same process as used previously for StreetLearn.
Ranked #7 on
Vision and Language Navigation
on Touchdown Dataset
Interactive Classification by Asking Informative Questions
We study the potential for interaction in natural language classification.
Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight
Learning uses both simulation and real environments without requiring autonomous flight in the physical environment during training, and combines supervised learning for predicting positions to visit and reinforcement learning for continuous control.
Executing Instructions in Situated Collaborative Interactions
We study a collaborative scenario where a user not only instructs a system to complete tasks, but also acts alongside it.
NLVR2 Visual Bias Analysis
We show that the performance of existing models (Li et al., 2019; Tan and Bansal 2019) is relatively robust to this potential bias.
BERTScore: Evaluating Text Generation with BERT
We propose BERTScore, an automatic evaluation metric for text generation.
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task.
Ranked #10 on
Vision and Language Navigation
on Touchdown Dataset
Early Fusion for Goal Directed Robotic Vision
Building perceptual systems for robotics which perform well under tight computational budgets requires novel architectures which rethink the traditional computer vision pipeline.
Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction
We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone.
A Corpus for Reasoning About Natural Language Grounded in Photographs
We crowdsource the data using sets of visually rich images and a compare-and-contrast task to elicit linguistically diverse language.
Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction
We propose to decompose instruction execution to goal prediction and action generation.
Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning
We introduce a method for following high-level navigation instructions by mapping directly from images, instructions and pose estimates to continuous low-level velocity commands for real-time control.
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation
We propose a learning approach for mapping context-dependent sequential instructions to actions.
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies
We present NEWSROOM, a summarization dataset of 1. 3 million articles and summaries written by authors and editors in newsrooms of 38 major news publications.
Learning to Map Context-Dependent Sentences to Executable Formal Queries
We propose a context-dependent model to map utterances within an interaction to executable formal queries.
CHALET: Cornell House Agent Learning Environment
We present CHALET, a 3D house simulator with support for navigation and manipulation.
Training RNNs as Fast as CNNs
Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
Visual Reasoning with Natural Language
To understand language in complex environments, agents must reason about the full range of language inputs and their correspondence to the world.
Simple Recurrent Units for Highly Parallelizable Recurrence
Common recurrent neural architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
Ranked #32 on
Question Answering
on SQuAD1.1 dev
A Corpus of Natural Language for Visual Reasoning
We present a new visual reasoning language dataset, containing 92, 244 pairs of examples of natural statements grounded in synthetic images with 3, 962 unique sentences.
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning
We propose to directly map raw visual observations and text input to actions for instruction execution.
Cornell SPF: Cornell Semantic Parsing Framework
The Cornell Semantic Parsing Framework (SPF) is a learning and inference framework for mapping natural language to formal representation of its meaning.
Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions
The context in which language is used provides a strong signal for learning to recover its meaning.
