Search Results for author:
Found 33 papers, 19 papers with code
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks.
Ranked #1 on
Image Retrieval
on MS COCO
(Recall@10 metric)
VinVL: Revisiting Visual Representations in Vision-Language Models
In our experiments we feed the visual features generated by the new object detection model into a Transformer-based VL fusion model \oscar \cite{li2020oscar}, and utilize an improved approach \short\ to pre-train the VL model and fine-tune it on a wide range of downstream VL tasks.
Ranked #2 on
Image-text matching
on CommercialAdsDataset
ConvLab: Multi-Domain End-to-End Dialog System Platform
We present ConvLab, an open-source multi-domain end-to-end dialog system platform, that enables researchers to quickly set up experiments with reusable components and compare a large set of different approaches, ranging from conventional pipeline systems to end-to-end neural models, in common environments.
End-to-End Task-Completion Neural Dialogue Systems
One of the major drawbacks of modularized task-completion dialogue systems is that each module is trained individually, which presents several challenges.
A User Simulator for Task-Completion Dialogues
Then, one can train reinforcement learning agents in an online fashion as they interact with the simulator.
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space
We hope that our first pre-trained big VAE language model itself and results can help the NLP community renew the interests of deep generative models in the era of large-scale pre-training, and make these principled methods more practical.
Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems
This proposal introduces a Dialogue Challenge for building end-to-end task-completion dialogue systems, with the goal of encouraging the dialogue research community to collaborate and benchmark on standard datasets and unified experimental environment.
Few-shot Natural Language Generation for Task-Oriented Dialog
It is pre-trained on a large set of annotated NLG corpus to acquire the controllable generation ability, and fine-tuned with only a few domain-specific labels to adapt to new domains.
Ranked #4 on
Data-to-Text Generation
on MULTIWOZ 2.1
Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access
In this paper, we address this limitation by replacing symbolic queries with an induced "soft" posterior distribution over the KB that indicates which entities the user is interested in.
Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning
During dialogue policy learning, the world model is constantly updated with real user experience to approach real user behavior, and in turn, the dialogue agent is optimized using both real experience and simulated experience.
Reinforcement Learning (RL)
Task-Completion Dialogue Policy Learning
LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation
Existing automatic evaluation on text-to-image synthesis can only provide an image-text matching score, without considering the object-level compositionality, which results in poor correlation with human judgments.
Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training
By training on a large amount of image-text-action triplets in a self-supervised learning manner, the pre-trained model provides generic representations of visual environments and language instructions.
Ranked #1 on
Visual Navigation
on Help, Anna! (HANNA)
End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager
Natural language understanding and dialogue policy learning are both essential in conversational systems that predict the next system actions in response to a current user utterance.
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation
We present the Frontier Aware Search with backTracking (FAST) Navigator, a general framework for action decoding, that achieves state-of-the-art results on the Room-to-Room (R2R) Vision-and-Language navigation challenge of Anderson et.
Ranked #3 on
Vision-Language Navigation
on Room2Room
Interactive Semantic Parsing for If-Then Recipes via Hierarchical Reinforcement Learning
Given a text description, most existing semantic parsers synthesize a program in one shot.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning
This paper presents a Discriminative Deep Dyna-Q (D3Q) approach to improving the effectiveness and robustness of Deep Dyna-Q (DDQ), a recently proposed framework that extends the Dyna-Q algorithm to integrate planning for task-completion dialogue policy learning.
Generating Informative and Diverse Conversational Responses via Adversarial Information Maximization
Responses generated by neural conversational models tend to lack informativeness and diversity.
Switch-based Active Deep Dyna-Q: Efficient Adaptive Planning for Task-Completion Dialogue Policy Learning
Training task-completion dialogue agents with reinforcement learning usually requires a large number of real user experiences.
Robust Navigation with Language Pretraining and Stochastic Sampling
Core to the vision-and-language navigation (VLN) challenge is building robust instruction representations and action decoding schemes, which can generalize well to previously unseen instructions and environments.
Subgoal Discovery for Hierarchical Dialogue Policy Learning
Experiments with simulated and real users show that our approach performs competitively against a state-of-the-art method that requires human-defined subgoals.
Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning
This paper presents a new method --- adversarial advantage actor-critic (Adversarial A2C), which significantly improves the efficiency of dialogue policy learning in task-completion dialogue systems.
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems.
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
We present a new algorithm that significantly improves the efficiency of exploration for deep Q-learning agents in dialogue systems.
Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning
Building a dialogue agent to fulfill complex tasks, such as travel planning, is challenging because the agent has to learn to collectively complete multiple subtasks.
Investigation of Language Understanding Impact for Reinforcement Learning Based Dialogue Systems
Language understanding is a key component in a spoken dialogue system.
Recurrent Reinforcement Learning: A Hybrid Approach
Successful applications of reinforcement learning in real-world problems often require dealing with partially observable states.
Towards Coherent and Cohesive Long-form Text Generation
Generating coherent and cohesive long-form texts is a challenging task.
Budgeted Policy Learning for Task-Oriented Dialogue Systems
This paper presents a new approach that extends Deep Dyna-Q (DDQ) by incorporating a Budget-Conscious Scheduling (BCS) to best utilize a fixed, small amount of user interactions (budget) for learning task-oriented dialogue agents.
Multi-View Learning for Vision-and-Language Navigation
Learning to navigate in a visual environment following natural language instructions is a challenging task because natural language instructions are highly variable, ambiguous, and under-specified.
MiniVLM: A Smaller and Faster Vision-Language Model
We design a Two-stage Efficient feature Extractor (TEE), inspired by the one-stage EfficientDet network, to significantly reduce the time cost of visual feature extraction by $95\%$, compared to a baseline model.
Self-supervised Pre-training with Hard Examples Improves Visual Representations
Self-supervised pre-training (SSP) employs random image transformations to generate training data for visual representation learning.
Ranked #69 on
Self-Supervised Image Classification
on ImageNet
A bird's eye view on coherence, and a worm's eye view on cohesion
Generating coherent and cohesive long-form texts is a challenging problem in natural language generation.
VIM: Probing Multimodal Large Language Models for Visual Embedded Instruction Following
We introduce VISUAL EMBEDDED INSTRUCTION (VIM), a new framework designed to evaluate the visual instruction following capability of Multimodal Large Language Models (MLLMs).
