Search Results for author: Hung Le

Found 46 papers, 18 papers with code

VGNMN: Video-grounded Neural Module Networks for Video-Grounded Dialogue Systems

no code implementations NAACL 2022 Hung Le, Nancy Chen, Steven Hoi

Neural module networks (NMN) have achieved success in image-grounded tasks such as Visual Question Answering (VQA) on synthetic images.

Information Retrieval Question Answering +2

Improving Document Image Understanding with Reinforcement Finetuning

no code implementations26 Sep 2022 Bao-Sinh Nguyen, Dung Tien Le, Hieu M. Vu, Tuan Anh D. Nguyen, Minh-Tien Nguyen, Hung Le

In this paper, we investigate the problem of improving the performance of Artificial Intelligence systems in understanding document images, especially in cases where training data is limited.

reinforcement-learning

Momentum Adversarial Distillation: Handling Large Distribution Shifts in Data-Free Knowledge Distillation

no code implementations21 Sep 2022 Kien Do, Hung Le, Dung Nguyen, Dang Nguyen, Haripriya Harikumar, Truyen Tran, Santu Rana, Svetha Venkatesh

Since the EMA generator can be considered as an ensemble of the generator's old versions and often undergoes a smaller change in updates compared to the generator, training on its synthetic samples can help the student recall the past knowledge and prevent the student from adapting too quickly to new updates of the generator.

Knowledge Distillation

LAVIS: A Library for Language-Vision Intelligence

1 code implementation15 Sep 2022 Dongxu Li, Junnan Li, Hung Le, Guangsen Wang, Silvio Savarese, Steven C. H. Hoi

We introduce LAVIS, an open-source deep learning library for LAnguage-VISion research and applications.

Image Captioning Image Retrieval +6

CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning

2 code implementations5 Jul 2022 Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, Steven C. H. Hoi

To address the limitations, we propose "CodeRL", a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning (RL).

Code Generation Pretrained Language Models +2

Multimodal Dialogue State Tracking

1 code implementation NAACL 2022 Hung Le, Nancy F. Chen, Steven C. H. Hoi

Specifically, we introduce a novel dialogue state tracking task to track the information of visual objects that are mentioned in video-grounded dialogues.

Dialogue State Tracking Video Understanding

HYCEDIS: HYbrid Confidence Engine for Deep Document Intelligence System

no code implementations1 Jun 2022 Bao-Sinh Nguyen, Quang-Bach Tran, Tuan-Anh Nguyen Dang, Duc Nguyen, Hung Le

Measuring the confidence of AI models is critical for safely deploying AI in real-world industrial systems.

OmniXAI: A Library for Explainable AI

1 code implementation1 Jun 2022 Wenzhuo Yang, Hung Le, Tanmay Laud, Silvio Savarese, Steven C. H. Hoi

We introduce OmniXAI (short for Omni eXplainable AI), an open-source Python library of eXplainable AI (XAI), which offers omni-way explainable AI capabilities and various interpretable machine learning techniques to address the pain points of understanding and interpreting the decisions made by machine learning (ML) in practice.

Counterfactual Explanation Decision Making +3

Learning to Constrain Policy Optimization with Virtual Trust Region

no code implementations20 Apr 2022 Hung Le, Thommen Karimpanal George, Majid Abdolshah, Dung Nguyen, Kien Do, Sunil Gupta, Svetha Venkatesh

We introduce a constrained optimization method for policy gradient reinforcement learning, which uses a virtual trust region to regulate each policy update.

Atari Games Policy Gradient Methods

Learning Theory of Mind via Dynamic Traits Attribution

no code implementations17 Apr 2022 Dung Nguyen, Phuoc Nguyen, Hung Le, Kien Do, Svetha Venkatesh, Truyen Tran

Inspired by the observation that humans often infer the character traits of others, then use it to explain behaviour, we propose a new neural ToM architecture that learns to generate a latent trait vector of an actor from the past trajectories.

Future prediction Inductive Bias +1

Episodic Policy Gradient Training

1 code implementation3 Dec 2021 Hung Le, Majid Abdolshah, Thommen K. George, Kien Do, Dung Nguyen, Svetha Venkatesh

We introduce a novel training procedure for policy gradient methods wherein episodic memory is used to optimize the hyperparameters of reinforcement learning algorithms on-the-fly.

Policy Gradient Methods

Robust Deep Reinforcement Learning for Extractive Legal Summarization

no code implementations13 Nov 2021 Duy-Hung Nguyen, Bao-Sinh Nguyen, Nguyen Viet Dung Nghiem, Dung Tien Le, Mim Amina Khatun, Minh-Tien Nguyen, Hung Le

Automatic summarization of legal texts is an important and still a challenging task since legal documents are often long and complicated with unusual structures and styles.

reinforcement-learning

Balanced Q-learning: Combining the Influence of Optimistic and Pessimistic Targets

no code implementations3 Nov 2021 Thommen George Karimpanal, Hung Le, Majid Abdolshah, Santu Rana, Sunil Gupta, Truyen Tran, Svetha Venkatesh

The optimistic nature of the Q-learning target leads to an overestimation bias, which is an inherent problem associated with standard $Q-$learning.

Q-Learning

Model-Based Episodic Memory Induces Dynamic Hybrid Controls

no code implementations NeurIPS 2021 Hung Le, Thommen Karimpanal George, Majid Abdolshah, Truyen Tran, Svetha Venkatesh

Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory.

reinforcement-learning

Generative Pseudo-Inverse Memory

no code implementations ICLR 2022 Kha Pham, Hung Le, Man Ngo, Truyen Tran, Bao Ho, Svetha Venkatesh

We propose Generative Pseudo-Inverse Memory (GPM), a class of deep generative memory models that are fast to write in and read out.

Denoising

Reachability Traces for Curriculum Design in Reinforcement Learning

no code implementations29 Sep 2021 Thommen Karimpanal George, Majid Abdolshah, Hung Le, Santu Rana, Sunil Gupta, Truyen Tran, Svetha Venkatesh

The objective in goal-based reinforcement learning is to learn a policy to reach a particular goal state within the environment.

reinforcement-learning

Neural Latent Traversal with Semantic Constraints

no code implementations29 Sep 2021 Majid Abdolshah, Hung Le, Thommen Karimpanal George, Vuong Le, Sunil Gupta, Santu Rana, Svetha Venkatesh

Whilst Generative Adversarial Networks (GANs) generate visually appealing high resolution images, the latent representations (or codes) of these models do not allow controllable changes on the semantic attributes of the generated images.

Plug and Play, Model-Based Reinforcement Learning

no code implementations20 Aug 2021 Majid Abdolshah, Hung Le, Thommen Karimpanal George, Sunil Gupta, Santu Rana, Svetha Venkatesh

This is achieved by representing the global transition dynamics as a union of local transition functions, each with respect to one active object in the scene.

Model-based Reinforcement Learning reinforcement-learning +1

Memory and attention in deep learning

1 code implementation3 Jul 2021 Hung Le

Artificial neural networks model neurons and synapses in the brain by interconnecting computational units via weights, which is a typical class of machine learning algorithms that resembles memory structure.

$C^3$: Compositional Counterfactual Constrastive Learning for Video-grounded Dialogues

no code implementations16 Jun 2021 Hung Le, Nancy F. Chen, Steven C. H. Hoi

Video-grounded dialogue systems aim to integrate video understanding and dialogue understanding to generate responses that are relevant to both the dialogue and video context.

Contrastive Learning Dialogue Understanding +1

VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks

no code implementations16 Apr 2021 Hung Le, Nancy F. Chen, Steven C. H. Hoi

Neural module networks (NMN) have achieved success in image-grounded tasks such as Visual Question Answering (VQA) on synthetic images.

Information Retrieval Question Answering +2

VilNMN: A Neural Module Network approach to Video-Grounded Language Tasks

no code implementations1 Jan 2021 Hung Le, Nancy F. Chen, Steven Hoi

Neural module networks (NMN) have achieved success in image-grounded tasks such as question answering (QA) on synthetic images.

Information Retrieval Question Answering

DVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue

1 code implementation ACL 2021 Hung Le, Chinnadhurai Sankar, Seungwhan Moon, Ahmad Beirami, Alborz Geramifard, Satwik Kottur

A video-grounded dialogue system is required to understand both dialogue, which contains semantic dependencies from turn to turn, and video, which contains visual cues of spatial and temporal scene variations.

Object Tracking Visual Reasoning

BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues

1 code implementation EMNLP 2020 Hung Le, Doyen Sahoo, Nancy F. Chen, Steven C. H. Hoi

Video-grounded dialogues are very challenging due to (i) the complexity of videos which contain both spatial and temporal variations, and (ii) the complexity of user utterances which query different segments and/or different objects in videos over multiple dialogue turns.

Neurocoder: Learning General-Purpose Computation Using Stored Neural Programs

no code implementations NeurIPS 2021 Hung Le, Svetha Venkatesh

For the first time a Neural Program is treated as a datum in memory, paving the ways for modular, recursive and procedural neural programming.

Continual Learning Object Recognition

Video-Grounded Dialogues with Pretrained Generation Language Models

no code implementations ACL 2020 Hung Le, Steven C. H. Hoi

Pre-trained language models have shown remarkable success in improving various downstream NLP tasks due to their ability to capture dependencies in textual data and generate natural responses.

UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues

1 code implementation EMNLP 2020 Hung Le, Doyen Sahoo, Chenghao Liu, Nancy F. Chen, Steven C. H. Hoi

Building an end-to-end conversational agent for multi-domain task-oriented dialogues has been an open challenge for two main reasons.

Dialogue State Tracking

Multimodal Transformer with Pointer Network for the DSTC8 AVSD Challenge

no code implementations25 Feb 2020 Hung Le, Nancy F. Chen

Audio-Visual Scene-Aware Dialog (AVSD) is an extension from Video Question Answering (QA) whereby the dialogue agent is required to generate natural language responses to address user queries and carry on conversations.

Question Answering Video Question Answering

Non-Autoregressive Dialog State Tracking

1 code implementation ICLR 2020 Hung Le, Richard Socher, Steven C. H. Hoi

Recent efforts in Dialogue State Tracking (DST) for task-oriented dialogues have progressed toward open-vocabulary or generation-based approaches where the models can generate slot value candidates from the dialogue history itself.

Dialogue State Tracking Multi-domain Dialogue State Tracking +1

Self-Attentive Associative Memory

1 code implementation ICML 2020 Hung Le, Truyen Tran, Svetha Venkatesh

Heretofore, neural networks with external memory are restricted to single memory with lossy representations of memory interactions.

Question Answering Relational Reasoning

Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems

1 code implementation ACL 2019 Hung Le, Doyen Sahoo, Nancy F. Chen, Steven C. H. Hoi

Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is conducted based on visual and audio aspects of a given video, is significantly more challenging than traditional image or text-grounded dialogue systems because (1) feature space of videos span across multiple picture frames, making it difficult to obtain semantic information; and (2) a dialogue agent must perceive and process information from different modalities (audio, video, caption, etc.)

Neural Stored-program Memory

1 code implementation ICLR 2020 Hung Le, Truyen Tran, Svetha Venkatesh

Neural networks powered with external memory simulate computer behaviors.

Ranked #5 on Question Answering on bAbi (Mean Error Rate metric)

Few-Shot Learning Question Answering

Meta-Learning with Domain Adaptation for Few-Shot Learning under Domain Shift

no code implementations ICLR 2019 Doyen Sahoo, Hung Le, Chenghao Liu, Steven C. H. Hoi

Most existing work assumes that both training and test tasks are drawn from the same distribution, and a large amount of labeled data is available in the training tasks.

Domain Adaptation Few-Shot Learning

Learning to Remember More with Less Memorization

1 code implementation ICLR 2019 Hung Le, Truyen Tran, Svetha Venkatesh

Memory-augmented neural networks consisting of a neural controller and an external memory have shown potentials in long-term sequential learning.

Sentiment Analysis Sequential Image Classification +1

Variational Memory Encoder-Decoder

1 code implementation NeurIPS 2018 Hung Le, Truyen Tran, Thin Nguyen, Svetha Venkatesh

Introducing variability while maintaining coherence is a core task in learning to generate utterances in conversation.

Dual Control Memory Augmented Neural Networks for Treatment Recommendations

no code implementations11 Feb 2018 Hung Le, Truyen Tran, Svetha Venkatesh

The decoding controller generates a treatment sequence, one treatment option at a time.

URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection

4 code implementations9 Feb 2018 Hung Le, Quang Pham, Doyen Sahoo, Steven C. H. Hoi

This approach allows the model to capture several types of semantic information, which was not possible by the existing models.

BIG-bench Machine Learning Feature Engineering +1

DeepProcess: Supporting business process execution using a MANN-based recommender system

1 code implementation3 Feb 2018 Asjad Khan, Hung Le, Kien Do, Truyen Tran, Aditya Ghose, Hoa Dam, Renuka Sindhgatta

Process-aware Recommender systems can provide critical decision support functionality to aid business process execution by recommending what actions to take next.

Activity Prediction Recommendation Systems

What are the Receptive, Effective Receptive, and Projective Fields of Neurons in Convolutional Neural Networks?

no code implementations19 May 2017 Hung Le, Ali Borji

In this work, we explain in detail how receptive fields, effective receptive fields, and projective fields of neurons in different layers, convolution or pooling, of a Convolutional Neural Network (CNN) are calculated.

Cannot find the paper you are looking for? You can Submit a new open access paper.