Search Results for author:
Found 25 papers, 9 papers with code
cTBL: Augmenting Large Language Models for Conversational Tables
An open challenge in multimodal conversational AI requires augmenting large language models with information from textual and non-textual sources for multi-turn dialogue.
Commonsense Reasoning for Conversational AI: A Survey of the State of the Art
Large, transformer-based pretrained language models like BERT, GPT, and T5 have demonstrated a deep understanding of contextual semantics and language syntax.
Multimodal Conversational AI: A Survey of Datasets and Approaches
As humans, we experience the world with all our senses or modalities (sound, sight, touch, smell, and taste).
Grounding Open-Domain Instructions to Automate Web Support Tasks
RUSS consists of two models: First, a BERT-LSTM with pointers parses instructions to ThingTalk, a domain-specific language we design for grounding natural language on the web.
Zero-Shot Visual Slot Filling as Question Answering
This paper presents a new approach to slot filling by reformulating the slot filling task as Question Answering, and replacing slot tags with rich natural language questions that capture the semantics of visual information and lexical text often displayed on device screens.
end-to-end training of a large vocabulary end-to-end speech recognition system
Our end-to-end speech recognition system built using this training infrastructure showed a 2. 44 % WER on test-clean of the LibriSpeech test set after applying shallow fusion with a Transformer language model (LM).
RILOD: Near Real-Time Incremental Learning for Object Detection at the Edge
The key component of RILOD is a novel incremental learning algorithm that trains end-to-end for one-stage deep object detection models only using training data of new object classes.
Class-incremental Learning via Deep Model Consolidation
The idea is to first train a separate model only for the new classes, and then combine the two individual models trained on data of two distinct set of classes (old classes and new classes) via a novel double distillation training objective.
Generative Visual Dialogue System via Adaptive Reasoning and Weighted Likelihood Estimation
The key challenge of generative Visual Dialogue (VD) systems is to respond to human queries with informative answers in natural and contiguous conversation flow.
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded
Many vision and language models suffer from poor visual grounding - often falling back on easy-to-learn language priors rather than basing their decisions on visual concepts in the image.
Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems
To address this challenge, we propose a hybrid imitation and reinforcement learning method, with which a dialogue agent can effectively learn from its interaction with users by learning from human teaching and feedback.
Building a Conversational Agent Overnight with Dialogue Self-Play
We propose Machines Talking To Machines (M2M), a framework combining automation and crowdsourcing to rapidly bootstrap end-to-end dialogue agents for goal-oriented dialogues in arbitrary domains.
Scalable Multi-Domain Dialogue State Tracking
We introduce a novel framework for state tracking which is independent of the slot value set, and represent the dialogue state as a distribution over a set of values of interest (candidate set) derived from the dialogue history or knowledge.
Dialogue State Tracking
Multi-domain Dialogue State Tracking
+2
Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning
We present a framework combining hierarchical and multi-agent deep reinforcement learning approaches to solve coordination problems among a multitude of agents using a semi-decentralized model.
End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning
We show that deep RL based optimization leads to significant improvement on task success rate and reduction in dialogue length comparing to supervised training model.
Towards Zero-Shot Frame Semantic Parsing for Domain Scaling
While multi-task training of such models alleviates the need for large in-domain annotated datasets, bootstrapping a semantic parsing model for a new domain using only the semantic frame, such as the back-end API or knowledge graph schema, is still one of the holy grail tasks of language understanding for dialogue systems.
Learning and Evaluating Musical Features with Deep Autoencoders
In this work we describe and evaluate methods to learn musical embeddings.
Sequential Dialogue Context Modeling for Spoken Language Understanding
We compare the performance of our proposed architecture with two context models, one that uses just the previous turn context and another that encodes dialogue context in a memory network, but loses the order of utterances in the dialogue history.
Goal-Oriented Dialogue Systems
Spoken Language Understanding
A Unit Selection Methodology for Music Generation Using Deep Neural Networks
We explore the use of unit selection and concatenation as a means of generating music using a procedure based on ranking, where, we consider a unit to be a variable length number of measures of music.
Leveraging Semantic Web Search and Browse Sessions for Multi-Turn Spoken Dialog Systems
Our session-based models outperform the state-of-the-art method for entity extraction task in SDS.
Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding
The goal of this paper is to use multi-task learning to efficiently scale slot filling models for natural language understanding to handle multiple target tasks or domains.
Contextual LSTM (CLSTM) models for Large scale NLP tasks
We evaluate CLSTM on three specific NLP tasks: word prediction, next sentence selection, and sentence topic prediction.
Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation
Entity Disambiguation aims to link mentions of ambiguous entities to a knowledge base (e. g., Wikipedia).
Zero-Shot Learning for Semantic Utterance Classification
We propose a novel zero-shot learning method for semantic utterance classification (SUC).
Learning deep structured semantic models for web search using clickthrough data
The proposed deep structured semantic models are discriminatively trained by maximizing the conditional likelihood of the clicked documents given a query using the clickthrough data.
