Search Results for author: Longtao Zheng

Found 6 papers, 3 papers with code

AgentStudio: A Toolkit for Building General Virtual Agents

no code implementations26 Mar 2024 Longtao Zheng, Zhiyuan Huang, Zhenghai Xue, Xinrun Wang, Bo An, Shuicheng Yan

We have open-sourced the environments, datasets, benchmarks, and interfaces to promote research towards developing general virtual agents for the future.

Visual Grounding

Towards General Computer Control: A Multimodal Agent for Red Dead Redemption II as a Case Study

2 code implementations5 Mar 2024 Weihao Tan, Ziluo Ding, Wentao Zhang, Boyu Li, Bohan Zhou, Junpeng Yue, Haochong Xia, Jiechuan Jiang, Longtao Zheng, Xinrun Xu, Yifei Bi, Pengjie Gu, Xinrun Wang, Börje F. Karlsson, Bo An, Zongqing Lu

Despite the success in specific tasks and scenarios, existing foundation agents, empowered by large models (LMs) and advanced tools, still cannot generalize to different scenarios, mainly due to dramatic differences in the observations and actions across scenarios.

Efficient Exploration

True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning

1 code implementation25 Jan 2024 Weihao Tan, Wentao Zhang, Shanqi Liu, Longtao Zheng, Xinrun Wang, Bo An

Despite the impressive performance across numerous tasks, large language models (LLMs) often fail in solving simple decision-making tasks due to the misalignment of the knowledge in LLMs with environments.

Decision Making Reinforcement Learning (RL)

Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control

1 code implementation13 Jun 2023 Longtao Zheng, Rundong Wang, Xinrun Wang, Bo An

To address these challenges, we introduce Synapse, a computer agent featuring three key components: i) state abstraction, which filters out task-irrelevant information from raw states, allowing more exemplars within the limited context, ii) trajectory-as-exemplar prompting, which prompts the LLM with complete trajectories of the abstracted states and actions to improve multi-step decision-making, and iii) exemplar memory, which stores the embeddings of exemplars and retrieves them via similarity search for generalization to novel tasks.

Decision Making In-Context Learning +1

Towards Skilled Population Curriculum for Multi-Agent Reinforcement Learning

no code implementations7 Feb 2023 Rundong Wang, Longtao Zheng, Wei Qiu, Bowei He, Bo An, Zinovi Rabinovich, Yujing Hu, Yingfeng Chen, Tangjie Lv, Changjie Fan

Despite its success, ACL's applicability is limited by (1) the lack of a general student framework for dealing with the varying number of agents across tasks and the sparse reward problem, and (2) the non-stationarity of the teacher's task due to ever-changing student strategies.

Multi-agent Reinforcement Learning reinforcement-learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.