Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructions

2 code implementations • 18 May 2023 • Shrestha Mohanty, Negar Arabzadeh, Julia Kiseleva, Artem Zholus, Milagro Teruel, Ahmed Awadallah, Yuxuan Sun, Kavya Srinet, Arthur Szlam

Human intelligence's adaptability is remarkable, allowing us to adjust to new tasks and multi-modal environments swiftly.

Collecting Interactive Multi-modal Datasets for Grounded Language Understanding

2 code implementations • 12 Nov 2022 • Shrestha Mohanty, Negar Arabzadeh, Milagro Teruel, Yuxuan Sun, Artem Zholus, Alexey Skrynnik, Mikhail Burtsev, Kavya Srinet, Aleksandr Panov, Arthur Szlam, Marc-Alexandre Côté, Julia Kiseleva

Human intelligence can remarkably adapt quickly to new tasks and environments.

Task 2

Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions

1 code implementation • 1 Nov 2022 • Alexey Skrynnik, Zoya Volovikova, Marc-Alexandre Côté, Anton Voronov, Artem Zholus, Negar Arabzadeh, Shrestha Mohanty, Milagro Teruel, Ahmed Awadallah, Aleksandr Panov, Mikhail Burtsev, Julia Kiseleva

The adoption of pre-trained language models to generate action plans for embodied agents is a promising research strategy.

Language Modelling reinforcement-learning +1

IGLU Gridworld: Simple and Fast Environment for Embodied Dialog Agents

1 code implementation • 31 May 2022 • Artem Zholus, Alexey Skrynnik, Shrestha Mohanty, Zoya Volovikova, Julia Kiseleva, Artur Szlam, Marc-Alexandre Coté, Aleksandr I. Panov

We present the IGLU Gridworld: a reinforcement learning environment for building and evaluating language conditioned embodied agents in a scalable way.

reinforcement-learning Reinforcement Learning (RL)

IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022

1 code implementation • 27 May 2022 • Julia Kiseleva, Alexey Skrynnik, Artem Zholus, Shrestha Mohanty, Negar Arabzadeh, Marc-Alexandre Côté, Mohammad Aliannejadi, Milagro Teruel, Ziming Li, Mikhail Burtsev, Maartje ter Hoeve, Zoya Volovikova, Aleksandr Panov, Yuxuan Sun, Kavya Srinet, Arthur Szlam, Ahmed Awadallah

Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions.

Natural Language Understanding Reinforcement Learning (RL)

Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021

no code implementations • 5 May 2022 • Julia Kiseleva, Ziming Li, Mohammad Aliannejadi, Shrestha Mohanty, Maartje ter Hoeve, Mikhail Burtsev, Alexey Skrynnik, Artem Zholus, Aleksandr Panov, Kavya Srinet, Arthur Szlam, Yuxuan Sun, Marc-Alexandre Côté, Katja Hofmann, Ahmed Awadallah, Linar Abdrazakov, Igor Churin, Putra Manggala, Kata Naszadi, Michiel van der Meer, Taewoon Kim

The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment.

Multitask Adaptation by Retrospective Exploration with Learned World Models

no code implementations • 25 Oct 2021 • Artem Zholus, Aleksandr I. Panov

Model-based reinforcement learning (MBRL) allows solving complex tasks in a sample-efficient manner.

Model-based Reinforcement Learning Test

NeurIPS 2021 Competition IGLU: Interactive Grounded Language Understanding in a Collaborative Environment

no code implementations • 13 Oct 2021 • Julia Kiseleva, Ziming Li, Mohammad Aliannejadi, Shrestha Mohanty, Maartje ter Hoeve, Mikhail Burtsev, Alexey Skrynnik, Artem Zholus, Aleksandr Panov, Kavya Srinet, Arthur Szlam, Yuxuan Sun, Katja Hofmann, Michel Galley, Ahmed Awadallah

Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions.

Natural Language Understanding Reinforcement Learning (RL)

Continuous Histogram Loss: Beyond Neural Similarity

no code implementations • 6 Apr 2020 • Artem Zholus, Evgeny Putin

Similarity learning has gained a lot of attention from researches in recent years and tons of successful approaches have been recently proposed.

Data Visualization Representation Learning