SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning

1 code implementation • 22 Oct 2024 • Yizhou Chi, Yizhang Lin, Sirui Hong, Duyi Pan, Yaying Fei, Guanghao Mei, Bangbang Liu, Tianqi Pang, Jacky Kwok, Ceyao Zhang, Bang Liu, Chenglin Wu

Automated Machine Learning (AutoML) approaches encompass traditional methods that optimize fixed pipelines for model selection and ensembling, as well as newer LLM-based frameworks that autonomously build pipelines.

AutoML Diversity +1

OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents

no code implementations • 27 Jun 2024 • ZiHao Wang, Shaofei Cai, Zhancun Mu, Haowei Lin, Ceyao Zhang, Xuejie Liu, Qing Li, Anji Liu, Xiaojian Ma, Yitao Liang

First, we introduce a self-supervised approach to learn a behavior encoder that produces discretized tokens for behavior trajectories $\tau = \{o_0, a_0, \dots\}$ and an imitation learning policy decoder conditioned on these tokens.

Decoder Imitation Learning +2

Inverse Design of Photonic Crystal Surface Emitting Lasers is a Sequence Modeling Problem

no code implementations • 8 Mar 2024 • Ceyao Zhang, Renjie Li, Cheng Zhang, Zhaoyu Zhang, Feng Yin

By modeling the inverse design of PCSEL as a sequential decision-making problem, RL approaches can construct a satisfactory PCSEL structure from scratch.

Decision Making Reinforcement Learning (RL) +1

Cradle: Empowering Foundation Agents Towards General Computer Control

1 code implementation • 5 Mar 2024 • Weihao Tan, Wentao Zhang, Xinrun Xu, Haochong Xia, Ziluo Ding, Boyu Li, Bohan Zhou, Junpeng Yue, Jiechuan Jiang, Yewen Li, Ruyi An, Molei Qin, Chuqiao Zong, Longtao Zheng, Yujie Wu, Xiaoqiang Chai, Yifei Bi, Tianbao Xie, Pengjie Gu, Xiyun Li, Ceyao Zhang, Long Tian, Chaojie Wang, Xinrun Wang, Börje F. Karlsson, Bo An, Shuicheng Yan, Zongqing Lu

To handle this issue, we propose the General Computer Control (GCC) setting to restrict foundation agents to interact with software through the most unified and standardized interface, i. e., using screenshots as input and keyboard and mouse actions as output.

Efficient Exploration

Data Interpreter: An LLM Agent For Data Science

1 code implementation • 28 Feb 2024 • Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Chenxing Wei, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Xiangru Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Zhibin Gou, Zongze Xu, Chenglin Wu

On InfiAgent-DABench, it achieves a 25% performance boost, raising accuracy from 75. 9% to 94. 9%.

Code Generation Language Modelling +2

Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects

1 code implementation • 7 Jan 2024 • Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, ZiHao Wang, Zekai Wang, Feng Yin, Junhua Zhao, Xiuqiang He

Intelligent agents stand out as a potential path toward artificial general intelligence (AGI).

Language Modelling Large Language Model

ProAgent: Building Proactive Cooperative Agents with Large Language Models

no code implementations • 22 Aug 2023 • Ceyao Zhang, Kaijie Yang, Siyi Hu, ZiHao Wang, Guanghe Li, Yihang Sun, Cheng Zhang, Zhaowei Zhang, Anji Liu, Song-Chun Zhu, Xiaojun Chang, Junge Zhang, Feng Yin, Yitao Liang, Yaodong Yang

Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems.

MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

1 code implementation • 1 Aug 2023 • Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, Jürgen Schmidhuber

Remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs).

Code Generation

Heterogeneous Value Alignment Evaluation for Large Language Models

2 code implementations • 26 May 2023 • Zhaowei Zhang, Ceyao Zhang, Nian Liu, Siyuan Qi, Ziqi Rong, Song-Chun Zhu, Shuguang Cui, Yaodong Yang

We conduct evaluations with new auto-metric \textit{value rationality} to represent the ability of LLMs to align with specific values.

Attribute