1 code implementation • 12 Mar 2025 • Chenyuan Yang, Zijie Zhao, Zichen Xie, Haoyu Li, Lingming Zhang
Our evaluation on the Linux kernel demonstrates that KNighter generates high-precision checkers capable of detecting diverse bug patterns overlooked by existing human-written analyzers.
no code implementations • 25 Feb 2025 • Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, Sida I. Wang
The recent DeepSeek-R1 release has demonstrated the immense potential of reinforcement learning (RL) in enhancing the general reasoning capabilities of large language models (LLMs).
2 code implementations • 31 Oct 2024 • Yuxiang Wei, Federico Cassano, Jiawei Liu, Yifeng Ding, Naman jain, Zachary Mueller, Harm de Vries, Leandro von Werra, Arjun Guha, Lingming Zhang
In our primary experiments, we use SelfCodeAlign with CodeQwen1. 5-7B to generate a dataset of 74k instruction-response pairs.
1 code implementation • 4 Sep 2024 • Junwei Liu, Kaixin Wang, Yixuan Chen, Xin Peng, Zhenpeng Chen, Lingming Zhang, Yiling Lou
The recent advance in Large Language Models (LLMs) has shaped a new paradigm of AI agents, i. e., LLM-based agents.
no code implementations • 12 Aug 2024 • Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, Lingming Zhang
We introduce Differential Performance Evaluation (DPE), a framework designed to reliably evaluate Large Language Models (LLMs) for efficient code generation.
1 code implementation • 1 Jul 2024 • Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, Lingming Zhang
However, the complexity of these agent-based approaches, together with the limited abilities of current LLMs, raises the following question: Do we really have to employ complex autonomous software agents?
1 code implementation • 10 Jun 2024 • Jiawei Liu, Jia Le Tian, Vijay Daita, Yuxiang Wei, Yifeng Ding, Yuhan Katherine Wang, Jun Yang, Lingming Zhang
Recent advances have been improving the context windows of Large Language Models (LLMs).
1 code implementation • 23 Apr 2024 • Yifeng Ding, Jiawei Liu, Yuxiang Wei, Terry Yue Zhuo, Lingming Zhang
We introduce XFT, a simple yet powerful training scheme, by simply merging upcycled Mixture-of-Experts (MoE) to unleash the performance limit of instruction-tuned code Large Language Models (LLMs).
no code implementations • 14 Apr 2024 • Siyuan Feng, Jiawei Liu, Ruihang Lai, Charlie F. Ruan, Yong Yu, Lingming Zhang, Tianqi Chen
However, this traditional development approach fails to meet the productivity requirements when deploying emerging ML applications, with the testing and debugging part as a bottleneck.
1 code implementation • 28 Mar 2024 • Chunqiu Steven Xia, Yinlin Deng, Lingming Zhang
Such limitations inevitably lead us to inquire: Is the leaderboard performance on existing benchmarks reliable and comprehensive enough to measure the program synthesis ability of LLMs?
4 code implementations • 29 Feb 2024 • Anton Lozhkov, Raymond Li, Loubna Ben allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Jian Zhu, Terry Yue Zhuo, Evgenii Zheltonozhskii, Nii Osae Osae Dade, Wenhao Yu, Lucas Krauß, Naman jain, Yixuan Su, Xuanli He, Manan Dey, Edoardo Abati, Yekun Chai, Niklas Muennighoff, Xiangru Tang, Muhtasham Oblokulov, Christopher Akiki, Marc Marone, Chenghao Mou, Mayank Mishra, Alex Gu, Binyuan Hui, Tri Dao, Armel Zebaze, Olivier Dehaene, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sebastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf, Arjun Guha, Leandro von Werra, Harm de Vries
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #35 on
Code Generation
on MBPP
1 code implementation • 31 Dec 2023 • Chenyuan Yang, Zijie Zhao, Lingming Zhang
As a result, a large body of research has been focused on kernel fuzzing, i. e., automatically generating syscall (system call) sequences to detect potential kernel bugs or vulnerabilities.
3 code implementations • 4 Dec 2023 • Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, Lingming Zhang
Magicoder models are trained on 75K synthetic instruction data using OSS-Instruct, a novel approach to enlightening LLMs with open-source code snippets to generate diverse instruction data for code.
1 code implementation • 24 Oct 2023 • Chenyuan Yang, Yinlin Deng, Runyu Lu, Jiayi Yao, Jiawei Liu, Reyhaneh Jabbarvand, Lingming Zhang
To this end, we propose WhiteFox, the first white-box compiler fuzzer using LLMs with source-code information to test compiler optimization, with a spotlight on detecting deep logic bugs in the deep learning (DL) compilers.
1 code implementation • 1 Sep 2023 • Yuxiang Wei, Chunqiu Steven Xia, Lingming Zhang
Therefore, we propose Repilot, a general code generation framework to further copilot the AI "copilots" (i. e., LLMs) by synthesizing more valid patches during the repair process.
1 code implementation • 9 Aug 2023 • Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, Lingming Zhang
Moreover, the inputs generated by existing fuzzers are often limited to specific features of the input language, and thus can hardly reveal bugs related to other or new features.
1 code implementation • NeurIPS 2023 • Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, Lingming Zhang
While EvalPlus is general, we extend the test-cases of the popular HumanEval benchmark by 80x to build HumanEval+.
no code implementations • 1 Apr 2023 • Chunqiu Steven Xia, Lingming Zhang
For earlier patches that failed to pass all tests, we combine the incorrect patches with their corresponding relevant test failure information to construct a new prompt for the LLM to generate the next patch.
no code implementations • 18 Mar 2023 • Chunqiu Steven Xia, Yifeng Ding, Lingming Zhang
Traditional APR tools have largely leveraged the plastic surgery hypothesis by designing manual or heuristic-based approaches to exploit such existing code ingredients.
1 code implementation • 4 Feb 2023 • Jiawei Liu, Jinjun Peng, Yuyao Wang, Lingming Zhang
NeuRI finds 100 new bugs for PyTorch and TensorFlow in four months, with 81 already fixed or confirmed.
no code implementations • 30 Jan 2023 • Chunqiu Steven Xia, Lingming Zhang
As such, we leverage the long-term context window of LLMs to not only avoid generating previously incorrect patches but also incorporate validation feedback to help the model understand the semantic meaning of the program under test.
1 code implementation • 26 Jul 2022 • Jiawei Liu, JinKun Lin, Fabian Ruffy, Cheng Tan, Jinyang Li, Aurojit Panda, Lingming Zhang
In this work, we propose a new fuzz testing approach for finding bugs in deep-learning compilers.
1 code implementation • 19 Apr 2022 • Yue Zhao, Lingming Zhang, Yang Liu, Deyu Meng, Zhiming Cui, Chenqiang Gao, Xinbo Gao, Chunfeng Lian, Dinggang Shen
The state-of-the-art deep learning-based methods often simply concatenate the raw geometric attributes (i. e., coordinates and normal vectors) of mesh cells to train a single-stream network for automatic intra-oral scanner image segmentation.
1 code implementation • 21 Feb 2022 • Jiawei Liu, Yuxiang Wei, Sen yang, Yinlin Deng, Lingming Zhang
Our results show that Tzer substantially outperforms existing fuzzing techniques on tensor compiler testing, with 75% higher coverage and 50% more valuable tests than the 2nd-best technique.
no code implementations • Pattern Recognition Letters 2021 • Yue Zhao, Lingming Zhang, Chongshi Yang, Yingyun Tan, Yang Liu, Pengcheng Li, Tianhao Huang, Chenqiang Gao
We have evaluated our network on a real-patient dataset of dental models acquired through 3D intraoral scanners, and experimental results show that our method outperforms state-of-the-art deep learning methods for 3D shape segmentation.
no code implementations • CVPR 2021 • Lingming Zhang, Yue Zhao, Deyu Meng, Zhiming Cui, Chenqiang Gao, Xinbo Gao, Chunfeng Lian, Dinggang Shen
State-of-the-art methods directly concatenate the raw attributes of 3D inputs, namely coordinates and normal vectors of mesh cells, to train a single-stream network for fully-automated tooth segmentation.
1 code implementation • 26 Dec 2020 • Lingming Zhang, Yue Zhao, Deyu Meng, Zhiming Cui, Chenqiang Gao, Xinbo Gao, Chunfeng Lian, Dinggang Shen
State-of-the-art methods directly concatenate the raw attributes of 3D inputs, namely coordinates and normal vectors of mesh cells, to train a single-stream network for fully-automated tooth segmentation.
no code implementations • 27 Dec 2018 • Husheng Zhou, Wei Li, Yuankun Zhu, Yuqun Zhang, Bei Yu, Lingming Zhang, Cong Liu
Furthermore, DeepBillboard is sufficiently robust and resilient for generating physical-world adversarial billboard tests for real-world driving under various weather conditions.
1 code implementation • 7 Feb 2018 • Mengshi Zhang, Yuqun Zhang, Lingming Zhang, Cong Liu, Sarfraz Khurshid
In this paper, we propose DeepRoad, an unsupervised framework to automatically generate large amounts of accurate driving scenes to test the consistency of DNN-based autonomous driving systems across different scenes.
Software Engineering