Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation

1 code implementation • 23 Feb 2025 • Simin Chen, Yiming Chen, Zexin Li, Yifan Jiang, Zhongwei Wan, Yixin He, Dezhi Ran, Tianle Gu, Haizhou Li, Tao Xie, Baishakhi Ray

To mitigate the risk of potential data contamination, LLM benchmarking has undergone a transformation from static to dynamic benchmarking.

Benchmarking

Data and System Perspectives of Sustainable Artificial Intelligence

no code implementations • 13 Jan 2025 • Tao Xie, David Harel, Dezhi Ran, Zhenwen Li, Maoliang Li, Zhi Yang, Leye Wang, Xiang Chen, Ying Zhang, Wentao Zhang, Meng Li, Chen Zhang, Linyi Li, Assaf Marron

Sustainable AI is a subfield of AI for concerning developing and using AI systems in ways of aiming to reduce environmental impact and achieve sustainability.

SWE-bench-java: A GitHub Issue Resolving Benchmark for Java

2 code implementations • 26 Aug 2024 • Daoguang Zan, Zhirong Huang, Ailun Yu, Shaoxin Lin, Yifan Shi, Wei Liu, Dong Chen, Zongshuai Qi, Hao Yu, Lei Yu, Dezhi Ran, Muhan Zeng, Bo Shen, Pan Bian, Guangtai Liang, Bei guan, Pengjie Huang, Tao Xie, Yongji Wang, Qianxiang Wang

GitHub issue resolving is a critical task in software engineering, recently gaining significant attention in both industry and academia.

Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software

no code implementations • 11 Jul 2024 • Dezhi Ran, Mengzhou Wu, Wei Yang, Tao Xie

By treating data and models as the source code, Foundation Models (FMs) become a new type of software.

Management