A Multi-level Alignment Training Scheme for Video-and-Language Grounding

no code implementations • 22 Apr 2022 • Yubo Zhang, Feiyang Niu, Qing Ping, Govind Thattai

To solve video-and-language grounding tasks, the key is for the network to understand the connection between the two modalities.

Retrieval Semantic Similarity +1

Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning

no code implementations • 21 Jan 2022 • Tongzhou Mu, Kaixiang Lin, Feiyang Niu, Govind Thattai

We present a two-step hybrid reinforcement learning (RL) policy that is designed to generate interpretable and robust hierarchical policies on the RL problem with graph-based input.

Decision Making Graph Neural Network +4

Interactive Teaching for Conversational AI

no code implementations • 2 Dec 2020 • Qing Ping, Feiyang Niu, Govind Thattai, Joel Chengottusseriyil, Qiaozi Gao, Aishwarya Reganti, Prashanth Rajagopal, Gokhan Tur, Dilek Hakkani-Tur, Prem Nataraja

Current conversational AI systems aim to understand a set of pre-designed requests and execute related actions, which limits them to evolve naturally and adapt based on human interactions.

LRTA: A Transparent Neural-Symbolic Reasoning Framework with Modular Supervision for Visual Question Answering

3 code implementations • 21 Nov 2020 • Weixin Liang, Feiyang Niu, Aishwarya Reganti, Govind Thattai, Gokhan Tur

We show that LRTA makes a step towards truly understanding the question while the state-of-the-art model tends to learn superficial correlations from the training data.

Answer Generation Question Answering +1