Search Results for author:
Found 23 papers, 18 papers with code
DialogStitch: Synthetic Deeper and Multi-Context Task-Oriented Dialogs
Real-world conversational agents must effectively handle long conversations that span multiple contexts.
An Analysis of State-of-the-Art Models for Situated Interactive MultiModal Conversations (SIMMC)
There is a growing interest in virtual assistants with multimodal capabilities, e. g., inferring the context of a conversation through scene understanding.
Continual Dialogue State Tracking via Example-Guided Question Answering
Dialogue systems are frequently updated to accommodate new services, but naively updating them by continually training with data for new services in diminishing performance on previously learnt services.
Hierarchical Video-Moment Retrieval and Step-Captioning
Our hierarchical benchmark consists of video retrieval, moment retrieval, and two novel moment segmentation and step captioning tasks.
Navigating Connected Memories with a Task-oriented Dialog System
Recent years have seen an increasing trend in the volume of personal media captured by users, thanks to the advent of smartphones and smart glasses, resulting in large media collections.
Tell Your Story: Task-Oriented Dialogs for Interactive Content Creation
We collect a new dataset C3 (Conversational Content Creation), comprising 10k dialogs conditioned on media montages simulated from a large media collection.
Database Search Results Disambiguation for Task-Oriented Dialog Systems
We find that training on our augmented dialog data improves the model's ability to deal with ambiguous scenarios, without sacrificing performance on unmodified turns.
Robustness through Data Augmentation Loss Consistency
Our experiments show that DAIR consistently outperforms ERM and DA-ERM with little marginal computational cost and sets new state-of-the-art results in several benchmarks involving covariant data augmentation.
Ranked #1 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.2
Multi-domain Dialogue State Tracking
Visual Question Answering
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations
Next generation task-oriented dialog systems need to understand conversational contexts with their perceived surroundings, to effectively help users in the real-world multimodal environment.
DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue
A video-grounded dialogue system is required to understand both dialogue, which contains semantic dependencies from turn to turn, and video, which contains visual cues of spatial and temporal scene variations.
Overview of the Ninth Dialog System Technology Challenge: DSTC9
Interactive evaluation of dialog, and 4.
Situated and Interactive Multimodal Conversations
Next generation virtual assistants are envisioned to handle multimodal inputs (e. g., vision, memories of previous interactions, in addition to the user's utterances), and perform multimodal actions (e. g., displaying a route in addition to generating the system's utterance).
On Emergent Communication in Competitive Multi-Agent Teams
Several recent works have found the emergence of grounded compositional language in the communication protocols developed by mostly cooperative multi-agent systems when learned end-to-end to maximize performance on a downstream task.
CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialog
Specifically, we construct a dialog grammar that is grounded in the scene graphs of the images from the CLEVR dataset.
Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context.
Ranked #1 on
Common Sense Reasoning
on Visual Dialog v0.9
Natural Language Does Not Emerge `Naturally' in Multi-Agent Dialog
A number of recent works have proposed techniques for end-to-end learning of communication protocols among cooperative multi-agent populations, and have simultaneously found the emergence of grounded human-interpretable language in the protocols developed by the agents, learned without any human supervision!
Canopy --- Fast Sampling with Cover Trees
In this work, we propose Canopy, a sampler based on Cover Trees that is exact, has guaranteed runtime logarithmic in the number of atoms, and is provably polynomial in the inherent dimensionality of the underlying parameter space.
Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog
A number of recent works have proposed techniques for end-to-end learning of communication protocols among cooperative multi-agent populations, and have simultaneously found the emergence of grounded human-interpretable language in the protocols developed by the agents, all learned without any human supervision!
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
Specifically, we pose a cooperative 'image guessing' game between two agents -- Qbot and Abot -- who communicate in natural language dialog so that Qbot can select an unseen image from a lineup of images.
Deep Sets
Our main theorem characterizes the permutation invariant functions and provides a family of functions to which any permutation invariant objective function must belong.
Visual Dialog
We introduce the task of Visual Dialog, which requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content.
Ranked #15 on
Visual Dialog
on VisDial v0.9 val
Visual Word2Vec (vis-w2v): Learning Visually Grounded Word Embeddings Using Abstract Scenes
While word embeddings trained using text have been extremely successful, they cannot uncover notions of semantic relatedness implicit in our visual world.
