Search Results for author: Larry Heck

Found 25 papers, 9 papers with code

cTBL: Augmenting Large Language Models for Conversational Tables

no code implementations21 Mar 2023 Anirudh S Sundar, Larry Heck

An open challenge in multimodal conversational AI requires augmenting large language models with information from textual and non-textual sources for multi-turn dialogue.

Response Generation Retrieval

Commonsense Reasoning for Conversational AI: A Survey of the State of the Art

no code implementations15 Feb 2023 Christopher Richardson, Larry Heck

Large, transformer-based pretrained language models like BERT, GPT, and T5 have demonstrated a deep understanding of contextual semantics and language syntax.

Multimodal Conversational AI: A Survey of Datasets and Approaches

no code implementations NLP4ConvAI (ACL) 2022 Anirudh Sundar, Larry Heck

As humans, we experience the world with all our senses or modalities (sound, sight, touch, smell, and taste).


Grounding Open-Domain Instructions to Automate Web Support Tasks

1 code implementation NAACL 2021 Nancy Xu, Sam Masling, Michael Du, Giovanni Campagna, Larry Heck, James Landay, Monica S Lam

RUSS consists of two models: First, a BERT-LSTM with pointers parses instructions to ThingTalk, a domain-specific language we design for grounding natural language on the web.

Zero-Shot Visual Slot Filling as Question Answering

no code implementations24 Nov 2020 Larry Heck, Simon Heck

This paper presents a new approach to slot filling by reformulating the slot filling task as Question Answering, and replacing slot tags with rich natural language questions that capture the semantics of visual information and lexical text often displayed on device screens.

Question Answering slot-filling +2

end-to-end training of a large vocabulary end-to-end speech recognition system

no code implementations22 Dec 2019 Chanwoo Kim, Sungsoo Kim, Kwangyoun Kim, Mehul Kumar, Jiyeon Kim, Kyungmin Lee, Changwoo Han, Abhinav Garg, Eunhyang Kim, Minkyoo Shin, Shatrughan Singh, Larry Heck, Dhananjaya Gowda

Our end-to-end speech recognition system built using this training infrastructure showed a 2. 44 % WER on test-clean of the LibriSpeech test set after applying shallow fusion with a Transformer language model (LM).

Data Augmentation Language Modelling +2

RILOD: Near Real-Time Incremental Learning for Object Detection at the Edge

no code implementations26 Mar 2019 Dawei Li, Serafettin Tasci, Shalini Ghosh, Jingwen Zhu, Junting Zhang, Larry Heck

The key component of RILOD is a novel incremental learning algorithm that trains end-to-end for one-stage deep object detection models only using training data of new object classes.

Incremental Learning object-detection +1

Class-incremental Learning via Deep Model Consolidation

2 code implementations19 Mar 2019 Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, C. -C. Jay Kuo

The idea is to first train a separate model only for the new classes, and then combine the two individual models trained on data of two distinct set of classes (old classes and new classes) via a novel double distillation training objective.

Class Incremental Learning Image Classification +3

Generative Visual Dialogue System via Adaptive Reasoning and Weighted Likelihood Estimation

no code implementations26 Feb 2019 Heming Zhang, Shalini Ghosh, Larry Heck, Stephen Walsh, Junting Zhang, Jie Zhang, C. -C. Jay Kuo

The key challenge of generative Visual Dialogue (VD) systems is to respond to human queries with informative answers in natural and contiguous conversation flow.

Visual Dialog

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

no code implementations ICCV 2019 Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, Devi Parikh

Many vision and language models suffer from poor visual grounding - often falling back on easy-to-learn language priors rather than basing their decisions on visual concepts in the image.

Image Captioning Question Answering +3

Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems

1 code implementation NAACL 2018 Bing Liu, Gokhan Tur, Dilek Hakkani-Tur, Pararth Shah, Larry Heck

To address this challenge, we propose a hybrid imitation and reinforcement learning method, with which a dialogue agent can effectively learn from its interaction with users by learning from human teaching and feedback.

Dialogue State Tracking Imitation Learning +3

Building a Conversational Agent Overnight with Dialogue Self-Play

3 code implementations15 Jan 2018 Pararth Shah, Dilek Hakkani-Tür, Gokhan Tür, Abhinav Rastogi, Ankur Bapna, Neha Nayak, Larry Heck

We propose Machines Talking To Machines (M2M), a framework combining automation and crowdsourcing to rapidly bootstrap end-to-end dialogue agents for goal-oriented dialogues in arbitrary domains.

Scalable Multi-Domain Dialogue State Tracking

1 code implementation29 Dec 2017 Abhinav Rastogi, Dilek Hakkani-Tur, Larry Heck

We introduce a novel framework for state tracking which is independent of the slot value set, and represent the dialogue state as a distribution over a set of values of interest (candidate set) derived from the dialogue history or knowledge.

Dialogue State Tracking Multi-domain Dialogue State Tracking +2

Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning

1 code implementation22 Dec 2017 Saurabh Kumar, Pararth Shah, Dilek Hakkani-Tur, Larry Heck

We present a framework combining hierarchical and multi-agent deep reinforcement learning approaches to solve coordination problems among a multitude of agents using a semi-decentralized model.

Efficient Exploration reinforcement-learning +2

End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning

no code implementations29 Nov 2017 Bing Liu, Gokhan Tur, Dilek Hakkani-Tur, Pararth Shah, Larry Heck

We show that deep RL based optimization leads to significant improvement on task success rate and reduction in dialogue length comparing to supervised training model.

reinforcement-learning Reinforcement Learning (RL)

Towards Zero-Shot Frame Semantic Parsing for Domain Scaling

1 code implementation7 Jul 2017 Ankur Bapna, Gokhan Tur, Dilek Hakkani-Tur, Larry Heck

While multi-task training of such models alleviates the need for large in-domain annotated datasets, bootstrapping a semantic parsing model for a new domain using only the semantic frame, such as the back-end API or knowledge graph schema, is still one of the holy grail tasks of language understanding for dialogue systems.

Semantic Parsing slot-filling +1

Sequential Dialogue Context Modeling for Spoken Language Understanding

1 code implementation WS 2017 Ankur Bapna, Gokhan Tur, Dilek Hakkani-Tur, Larry Heck

We compare the performance of our proposed architecture with two context models, one that uses just the previous turn context and another that encodes dialogue context in a memory network, but loses the order of utterances in the dialogue history.

Goal-Oriented Dialogue Systems Spoken Language Understanding

A Unit Selection Methodology for Music Generation Using Deep Neural Networks

no code implementations12 Dec 2016 Mason Bretan, Gil Weinberg, Larry Heck

We explore the use of unit selection and concatenation as a means of generating music using a procedure based on ranking, where, we consider a unit to be a variable length number of measures of music.

Music Generation

Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding

no code implementations1 Apr 2016 Aaron Jaech, Larry Heck, Mari Ostendorf

The goal of this paper is to use multi-task learning to efficiently scale slot filling models for natural language understanding to handle multiple target tasks or domains.

Domain Adaptation Multi-Task Learning +3

Contextual LSTM (CLSTM) models for Large scale NLP tasks

no code implementations19 Feb 2016 Shalini Ghosh, Oriol Vinyals, Brian Strope, Scott Roy, Tom Dean, Larry Heck

We evaluate CLSTM on three specific NLP tasks: word prediction, next sentence selection, and sentence topic prediction.

Paraphrase Generation Question Answering +1

Leveraging Deep Neural Networks and Knowledge Graphs for Entity Disambiguation

no code implementations28 Apr 2015 Hongzhao Huang, Larry Heck, Heng Ji

Entity Disambiguation aims to link mentions of ambiguous entities to a knowledge base (e. g., Wikipedia).

Entity Disambiguation Knowledge Graphs

Learning deep structured semantic models for web search using clickthrough data

3 code implementations CIKM 2013 Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, Larry Heck

The proposed deep structured semantic models are discriminatively trained by maximizing the conditional likelihood of the clicked documents given a query using the clickthrough data.

Document Ranking

Cannot find the paper you are looking for? You can Submit a new open access paper.