CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark

1 code implementation • 22 Jan 2024 • Ge Zhang, Xinrun Du, Bei Chen, Yiming Liang, Tongxu Luo, Tianyu Zheng, Kang Zhu, Yuyang Cheng, Chunpu Xu, Shuyue Guo, Haoran Zhang, Xingwei Qu, Junjie Wang, Ruibin Yuan, Yizhi Li, Zekun Wang, Yudong Liu, Yu-Hsuan Tsai, Fengji Zhang, Chenghua Lin, Wenhao Huang, Wenhu Chen, Jie Fu

We introduce CMMMU, a new Chinese Massive Multi-discipline Multimodal Understanding benchmark designed to evaluate LMMs on tasks demanding college-level subject knowledge and deliberate reasoning in a Chinese context.

SoTaNa: The Open-Source Software Development Assistant

1 code implementation • 25 Aug 2023 • Ensheng Shi, Fengji Zhang, Yanlin Wang, Bei Chen, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, Hongbin Sun

To meet the demands of this dynamic field, there is a growing need for an effective software development assistant.

Code Summarization

RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation

1 code implementation • 22 Mar 2023 • Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, Weizhu Chen

The task of repository-level code completion is to continue writing the unfinished code based on a broader context of the repository.

Code Completion Language Modelling +1

Large Language Models Meet NL2Code: A Survey

no code implementations • 19 Dec 2022 • Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei guan, Yongji Wang, Jian-Guang Lou

The task of generating code from a natural language description, or NL2Code, is considered a pressing and significant challenge in code intelligence.

Diverse Title Generation for Stack Overflow Posts with Multiple Sampling Enhanced Transformer

1 code implementation • 24 Aug 2022 • Fengji Zhang, Jin Liu, Yao Wan, Xiao Yu, Xiao Liu, Jacky Keung

Stack Overflow is one of the most popular programming communities where developers can seek help for their encountered problems.

CodeT: Code Generation with Generated Tests

1 code implementation • 21 Jul 2022 • Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, Weizhu Chen

A natural way to evaluate the quality and correctness of a code solution is to run it against a set of test cases, but the manual creation of such test cases is often costly and time-consuming.

Code Generation

Improving Stack Overflow question title generation with copying enhanced CodeBERT model and bi-modal information

1 code implementation • 27 Sep 2021 • Fengji Zhang, Xiao Yu, Jacky Keung, Fuyang Li, Zhiwen Xie, Zhen Yang, Caoyuan Ma, Zhimin Zhang

However, only using the code snippets in the question body cannot provide sufficient information for title generation, and LSTMs cannot capture the long-range dependencies between tokens.