MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache

no code implementations • 27 Nov 2024 • Akshat Sharma, Hangliang Ding, Jianping Li, Neel Dani, Minjia Zhang

How to efficiently serve LLMs in practice has become exceptionally challenging due to their prohibitive memory and computation requirements.

AgentBench: Evaluating LLMs as Agents

1 code implementation • 7 Aug 2023 • Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, Jie Tang

We present AgentBench, a multi-dimensional evolving benchmark that currently consists of 8 distinct environments to assess LLM-as-Agent's reasoning and decision-making abilities in a multi-turn open-ended generation setting.

Decision Making Instruction Following