Search Results for author:
Found 32 papers, 9 papers with code
Demonstrating EMMA: Embodied MultiModal Agent for Language-guided Action Execution in 3D Simulated Environments
We demonstrate EMMA, an embodied multimodal agent which has been developed for the Alexa Prize SimBot challenge.
Dialogue Act and Slot Recognition in Italian Complex Dialogues
Since the advent of Transformer-based, pretrained language models (LM) such as BERT, Natural Language Understanding (NLU) components in the form of Dialogue Act Recognition (DAR) and Slot Recognition (SR) for dialogue systems have become both more accurate and easier to create for specific application domains.
Combine to Describe: Evaluating Compositional Generalization in Image Captioning
Compositionality – the ability to combine simpler concepts to understand & generate arbitrarily more complex conceptual structures – has long been thought to be the cornerstone of human language capacity.
Incremental Graph-Based Semantics and Reasoning for Conversational AI
The next generation of conversational AI systems need to: (1) process language incrementally, token-by-token to be more responsive and enable handling of conversational phenomena such as pauses, restarts and self-corrections; (2) reason incrementally allowing meaning to be established beyond what is said; (3) be transparent and controllable, allowing designers as well as the system itself to easily establish reasons for particular behaviour and tailor to particular user groups, or domains.
Lost in Space: Probing Fine-grained Spatial Understanding in Vision and Language Resamplers
In this paper, we use \textit{diagnostic classifiers} to measure the extent to which the visual prompt produced by the resampler encodes spatial information.
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion
Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and observations, and 2) referential disambiguation.
Learning to generate and corr- uh I mean repair language in real-time
We further do a zero-shot evaluation of the ability of the same model to generate self-repairs when the generation goal changes mid-utterance.
No that's not what I meant: Handling Third Position Repair in Conversational Question Answering
We demonstrate the usefulness of the data by training and evaluating strong baseline models for executing TPRs.
'What are you referring to?' Evaluating the Ability of Multi-Modal Dialogue Models to Process Clarificational Exchanges
Referential ambiguities arise in dialogue when a referring expression does not uniquely identify the intended referent for the addressee.
The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering
Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e. g. "unfaithful" with respect to a rationale as retrieved from a knowledge base.
Conversational Question Answering
Open-Domain Question Answering
Exploring Multi-Modal Representations for Ambiguity Detection & Coreference Resolution in the SIMMC 2.0 Challenge
Anaphoric expressions, such as pronouns and referential descriptions, are situated with respect to the linguistic context of prior turns, as well as, the immediate visual environment.
A Study of Automatic Metrics for the Evaluation of Natural Language Explanations
As transparency becomes key for robotics and AI, it will be necessary to evaluate the methods through which transparency is provided, including automatically generated natural language (NL) explanations.
A Comprehensive Evaluation of Incremental Speech Recognition and Diarization for Conversational AI
Automatic Speech Recognition (ASR) systems are increasingly powerful and more accurate, but also more numerous with several options existing currently as a service (e. g. Google, IBM, and Microsoft).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Data-Efficient Goal-Oriented Conversation with Dialogue Knowledge Transfer Networks
Our main dataset is the Stanford Multi-Domain dialogue corpus.
Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia
Dialogue technologies such as Amazon's Alexa have the potential to transform the healthcare industry.
Few-Shot Dialogue Generation Without Annotated Data: A Transfer Learning Approach
Learning with minimal data is one of the key challenges in the development of practical, production-ready goal-oriented dialogue systems.
Benchmarking Natural Language Understanding Services for building Conversational Agents
We have recently seen the emergence of several publicly available Natural Language Understanding (NLU) toolkits, which map user utterances to structured, but more abstract, Dialogue Act (DA) or Intent specifications, while making this process accessible to the lay developer.
Multi-Task Learning for Domain-General Spoken Disfluency Detection in Dialogue Systems
To test the model's generalisation potential, we evaluate the same model on the bAbI+ dataset, without any additional training.
Learning how to learn: an adaptive dialogue agent for incrementally learning visually grounded word meanings
We present an optimised multi-modal dialogue agent for interactive learning of visually grounded word meanings from a human tutor, trained on real human-human tutoring data.
The BURCHAK corpus: a Challenge Data Set for Interactive Learning of Visually Grounded Word Meanings
We motivate and describe a new freely available human-human dialogue dataset for interactive learning of visually grounded word meanings through ostensive definition by a tutor to a learner.
Training an adaptive dialogue policy for interactive learning of visually grounded word meanings
We present a multi-modal dialogue system for interactive learning of perceptually grounded word meanings from a human tutor.
Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars
Our experiments show that our model can process 74% of the Facebook AI bAbI dataset even when trained on only 0. 13% of the data (5 dialogues).
Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena
Results show that the semantic accuracy of the MemN2N model drops drastically; and that although it is in principle able to learn to process the constructions in bAbI+, it needs an impractical amount of training data to do so.
VOILA: An Optimised Dialogue System for Interactively Learning Visually-Grounded Word Meanings (Demonstration System)
We present VOILA: an optimised, multi-modal dialogue agent for interactive learning of visually grounded word meanings from a human user.
Bootstrapping incremental dialogue systems: using linguistic knowledge to learn from minimal data
We present a method for inducing new dialogue systems from very small amounts of unannotated dialogue data, showing how word-level exploration using Reinforcement Learning (RL), combined with an incremental and semantic grammar - Dynamic Syntax (DS) - allows systems to discover, generate, and understand many new dialogue variants.
