Gated-Attention Architectures for Task-Oriented Language Grounding

22 Jun 2017Devendra Singh Chaplot • Kanthashree Mysore Sathyendra • Rama Kumar Pasumarthi • Dheeraj Rajagopal • Ruslan Salakhutdinov

To perform tasks specified by natural language instructions, autonomous agents need to extract semantically meaningful representations of language and map it to visual elements and actions in the environment. This problem is called task-oriented language grounding. We propose an end-to-end trainable neural architecture for task-oriented language grounding in 3D environments which assumes no prior linguistic or perceptual knowledge and requires only raw pixels from the environment and the natural language instruction as input.

Full paper


No evaluation results yet. Help compare this paper to other papers by submitting the tasks and evaluation metrics from the paper.