Search Results for author:
Found 25 papers, 6 papers with code
MineAnyBuild: Benchmarking Spatial Planning for Open-world AI Agents
In this work, we take a step further to build a comprehensive benchmark called MineAnyBuild, aiming to evaluate the spatial planning ability of open-world AI agents in the Minecraft game.
Structured Preference Optimization for Vision-Language Long-Horizon Task Planning
Existing methods for vision-language task planning excel in short-horizon tasks but often fall short in complex, long-horizon planning within dynamic environments.
Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation
Recent advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have made them powerful tools in embodied navigation, enabling agents to leverage commonsense and spatial reasoning for efficient exploration in unfamiliar environments.
3D-MoE: A Mixture-of-Experts Multi-modal LLM for 3D Vision and Pose Diffusion via Rectified Flow
3D vision and spatial reasoning have long been recognized as preferable for accurately perceiving our three-dimensional world, especially when compared with traditional visual reasoning based on 2D images.
SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning
To address these limitations, we propose a novel approach named SpatialCoT, specifically designed to bolster the spatial reasoning capabilities of VLMs.
ET-Plan-Bench: Embodied Task-level Planning Benchmark Towards Spatial-Temporal Cognition with Foundation Models
To further explore this area, we introduce a new embodied task planning benchmark, ET-Plan-Bench, which specifically targets embodied task planning using LLMs.
ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning
Key features of the framework include: integration of ROS with an AI agent connected to a plethora of open-source and commercial LLMs, automatic extraction of a behavior from the LLM output and execution of ROS actions/services, support for three behavior modes (sequence, behavior tree, state machine), imitation learning for adding new robot actions to the library of possible actions, and LLM reflection via human and environment feedback.
A Survey on Vision-Language-Action Models for Embodied AI
To this end, we present the first survey on VLAs for embodied AI.
Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts
Inspired by the great success of diffusion model in image generation tasks we propose a novel hierarchical framework named as CoTDiffusion that incorporates diffusion model as a high-level planner to convert the general and multi-modal prompts into coherent visual subgoal plans which further guide the low-level policy model before action execution.
VOLTA: Improving Generative Diversity by Variational Mutual Information Maximizing Autoencoder
The natural language generation domain has witnessed great success thanks to Transformer models.
Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection
The exploration problem is one of the main challenges in deep reinforcement learning (RL).
Ranked #1 on
Atari Games
on Atari-57
Planning Immediate Landmarks of Targets for Model-Free Skill Transfer across Agents
In reinforcement learning applications like robotics, agents usually need to deal with various input/output features when specified with different state/action spaces by their developers or physical restrictions.
Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning
The latent world model provides a promising way to learn policies in a compact latent space for tasks with high-dimensional observations, however, its generalization across diverse environments with unseen dynamics remains challenging.
Model-based Reinforcement Learning
reinforcement-learning
+1
NeurIPS 2022 Competition: Driving SMARTS
The proposed competition supports methodologically diverse solutions, such as reinforcement learning (RL) and offline learning methods, trained on a combination of naturalistic AD data and open-source simulation platform SMARTS.
RITA: Boost Driving Simulators with Realistic Interactive Traffic Flow
High-quality traffic flow generation is the core module in building simulators for autonomous driving.
Decomposed Mutual Information Optimization for Generalized Context in Meta-Reinforcement Learning
This paper addresses such a challenge by Decomposed Mutual INformation Optimization (DOMINO) for context learning, which explicitly learns a disentangled context to maximize the mutual information between the context and historical trajectories, while minimizing the state transition prediction error.
Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization
Recent progress in state-only imitation learning extends the scope of applicability of imitation learning to real-world settings by relieving the need for observing expert actions.
Model-Based Reinforcement Learning via Imagination with Derived Memory
Model-based reinforcement learning aims to improve the sample efficiency of policy learning by modeling the dynamics of the environment.
Model-based Reinforcement Learning
reinforcement-learning
+2
Plan Your Target and Learn Your Skills: State-Only Imitation Learning via Decoupled Policy Optimization
State-only imitation learning (SOIL) enables agents to learn from massive demonstrations without explicit action or reward information.
Learning Symbolic Rules for Interpretable Deep Reinforcement Learning
To address this challenge and improve the transparency, we propose a Neural Symbolic Reinforcement Learning framework by introducing symbolic logic into DRL.
Robust Memory Augmentation by Constrained Latent Imagination
The latent dynamics model summarizes an agent’s high dimensional experiences in a compact way.
Interpretable Reinforcement Learning With Neural Symbolic Logic
Recent progress in deep reinforcement learning (DRL) can be largely attributed to the use of neural networks.
Deep Reinforcement Learning
Hierarchical Reinforcement Learning
+3
Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets
Generative adversarial imitation learning (GAIL) has shown promising results by taking advantage of generative adversarial nets, especially in the field of robot learning.
Multi-Agent Interactions Modeling with Correlated Policies
In this paper, we cast the multi-agent interactions modeling problem into a multi-agent imitation learning framework with explicit modeling of correlated policies by approximating opponents' policies, which can recover agents' policies that can regenerate similar interactions.
Graph Attention Memory for Visual Navigation
To address the long-term memory issue, this paper proposes a graph attention memory (GAM) architecture consisting of memory construction module, graph attention module and control module.
