Search Results for author:
Found 48 papers, 25 papers with code
Demonstrating EMMA: Embodied MultiModal Agent for Language-guided Action Execution in 3D Simulated Environments
We demonstrate EMMA, an embodied multimodal agent which has been developed for the Alexa Prize SimBot challenge.
Learning to Read Maps: Understanding Natural Language Instructions from Unseen Maps
Robust situated dialog requires the ability to process instructions based on spatial information, which may or may not be available.
Visually Grounded Language Learning: a review of language games, datasets, tasks, and models
Our analysis of the literature provides evidence that future work should be focusing on interactive games where communication in Natural Language is important to resolve ambiguities about object referents and action plans and that physical embodiment is essential to understand the semantics of situations and events.
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion
Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and observations, and 2) referential disambiguation.
No that's not what I meant: Handling Third Position Repair in Conversational Question Answering
We demonstrate the usefulness of the data by training and evaluating strong baseline models for executing TPRs.
Neuron to Graph: Interpreting Language Model Neurons at Scale
Conventional methods require examination of examples with strong neuron activation and manual identification of patterns to decipher the concepts a neuron responds to.
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark
We use this improved benchmark to evaluate recent model editing techniques and find that they suffer from low specificity.
The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering
Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e. g. "unfaithful" with respect to a rationale as retrieved from a knowledge base.
Conversational Question Answering
Open-Domain Question Answering
The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python
Large Language Models (LLMs) have successfully been applied to code generation tasks, raising the question of how well these models understand programming.
iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?
There are two competing approaches for modelling annotator disagreement: distributional soft-labelling approaches (which aim to capture the level of disagreement) or modelling perspectives of individual annotators or groups thereof.
Going for GOAL: A Resource for Grounded Football Commentaries
As the course of a game is unpredictable, so are commentaries, which makes them a unique resource to investigate dynamic language grounding.
Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models
Pretrained language models (PLMs) for data-to-text (D2T) generation can use human-readable data labels such as column headings, keys, or relation names to generalize to out-of-domain examples.
Task Formulation Matters When Learning Continually: A Case Study in Visual Question Answering
We first propose three plausible task formulations and demonstrate their impact on the performance of continual learning algorithms.
SPaR.txt, a cheap Shallow Parsing approach for Regulatory texts
Automated Compliance Checking (ACC) systems aim to semantically parse building regulations to a set of rules.
MiRANews: Dataset and Benchmarks for Multi-Resource-Assisted News Summarization
We show via data analysis that it's not only the models which are to blame: more than 27% of facts mentioned in the gold summaries of MiRANews are better grounded on assisting documents than in the main source articles.
AGGGEN: Ordering and Aggregating while Generating
We present AGGGEN (pronounced 'again'), a data-to-text model which re-introduces two explicit sentence planning stages into neural data-to-text systems: input ordering and input aggregation.
OTTers: One-turn Topic Transitions for Open-Domain Dialogue
Mixed initiative in open-domain dialogue requires a system to pro-actively introduce new topics.
An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games
Guessing games are a prototypical instance of the "learning by interacting" paradigm.
Imagining Grounded Conceptual Representations from Perceptual Information in Situated Guessing Games
However, as shown by Suglia et al. (2020), existing models fail to learn truly multi-modal representations, relying instead on gold category labels for objects in the scene both at training and inference time.
Findings of the Fourth Workshop on Neural Generation and Translation
We describe the finding of the Fourth Workshop on Neural Generation and Translation, held in concert with the annual conference of the Association for Computational Linguistics (ACL 2020).
Fact-based Content Weighting for Evaluating Abstractive Summarisation
Abstractive summarisation is notoriously hard to evaluate since standard word-overlap-based metrics are insufficient.
CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learning
To remedy this, we present GROLLA, an evaluation framework for Grounded Language Learning with Attributes with three sub-tasks: 1) Goal-oriented evaluation; 2) Object attribute prediction evaluation; and 3) Zero-shot evaluation.
A Scientific Information Extraction Dataset for Nature Inspired Engineering
Nature has inspired various ground-breaking technological developments in applications ranging from robotics to aerospace engineering and the manufacturing of medical devices.
In Layman's Terms: Semi-Open Relation Extraction from Scientific Texts
First, we present the Focused Open Biological Information Extraction (FOBIE) dataset and use FOBIE to train a state-of-the-art narrow scientific IE system to extract trade-off relations and arguments that are central to biology texts.
History for Visual Dialog: Do we really need it?
Visual Dialog involves "understanding" the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response.
Findings of the Third Workshop on Neural Generation and Translation
This document describes the findings of the Third Workshop on Neural Generation and Translation, held in concert with the annual conference of the Empirical Methods in Natural Language Processing (EMNLP 2019).
Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)
We present a recurrent neural network based system for automatic quality estimation of natural language generation (NLG) outputs, which jointly learns to assign numerical ratings to individual outputs and to provide pairwise rankings of two different outputs.
Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia
Dialogue technologies such as Amazon's Alexa have the potential to transform the healthcare industry.
Corpus of Multimodal Interaction for Collaborative Planning
As autonomous systems become more commonplace, we need a way to easily and naturally communicate to them our goals and collaboratively come up with a plan on how to achieve these goals.
SEQ\^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression
The proposed model does not require parallel text-summary pairs, achieving promising results in unsupervised sentence compression on benchmark datasets.
SEQ^3: Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression
The proposed model does not require parallel text-summary pairs, achieving promising results in unsupervised sentence compression on benchmark datasets.
Improving Context Modelling in Multimodal Dialogue Generation
In this work, we investigate the task of textual response generation in a multimodal task-oriented dialogue system.
A Knowledge-Grounded Multimodal Search-Based Conversational Agent
Multimodal search-based dialogue is a challenging new task: It extends visually grounded question answering systems into multi-turn conversations with access to an external database.
Better Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity
We present three enhancements to existing encoder-decoder models for open-domain conversational agents, aimed at effectively modeling coherence and promoting output diversity: (1) We introduce a measure of coherence as the GloVe embedding similarity between the dialogue context and the generated response, (2) we filter our training corpora based on the measure of coherence to obtain topically coherent and lexically diverse context-response pairs, (3) we then train a response generator using a conditional variational autoencoder model that incorporates the measure of coherence as a latent variable and uses a context gate to guarantee topical consistency with the context and promote lexical diversity.
Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity
We present three enhancements to existing encoder-decoder models for open-domain conversational agents, aimed at effectively modeling coherence and promoting output diversity: (1) We introduce a measure of coherence as the GloVe embedding similarity between the dialogue context and the generated response, (2) we filter our training corpora based on the measure of coherence to obtain topically coherent and lexically diverse context-response pairs, (3) we then train a response generator using a conditional variational autoencoder model that incorporates the measure of coherence as a latent variable and uses a context gate to guarantee topical consistency with the context and promote lexical diversity.
Mapping Language to Code in Programmatic Context
To study this phenomenon, we introduce the task of generating class member functions given English documentation and the programmatic context provided by the rest of the class.
Learning a Neural Semantic Parser from User Feedback
We present an approach to rapidly and easily build natural language interfaces to databases for new domains, whose performance improves over time based on user feedback, and requires minimal intervention.
Ranked #1 on
SQL Parsing
on Restaurants
Neural AMR: Sequence-to-Sequence Models for Parsing and Generation
Sequence-to-sequence models have shown strong performance across a broad range of applications.
Ranked #6 on
AMR Parsing
on LDC2015E86
Story Cloze Task: UW NLP System
This paper describes University of Washington NLP{'}s submission for the Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem 2017) shared task{---}the Story Cloze Task.
The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
A writer's style depends not just on personal traits but also on her intent and mental state.
A Theme-Rewriting Approach for Generating Algebra Word Problems
Texts present coherent stories that have a particular theme or overall setting, for example science fiction or western.
