no code implementations • 3 Feb 2024 • Dong Huang, Jie M. Zhang, Yuhao QING, Heming Cui
This paper presents EffiBench, a benchmark with 1, 000 efficiency-critical coding problems for assessing the efficiency of code generated by code generation models.
no code implementations • 17 Aug 2023 • Dong Huang, Qingwen Bu, Yuhao QING, Heming Cui
However, its application in code generation faces a distinct challenge, i. e., although the code generated with CoT reasoning is logically correct, it faces the problem of syntax error (e. g., invalid syntax error report) during code execution, which causes the CoT result's pass@1 in HumanEval even lower than the zero-shot result.
no code implementations • 20 Jul 2023 • Dong Huang, Qingwen Bu, Yichao Fu, Yuhao QING, Bocheng Xiao, Heming Cui
To address the above-mentioned problem, we propose NSS, Neuron Sensitivity guided test case Selection, which can reduce the labeling time by selecting valuable test cases from unlabeled datasets.
1 code implementation • 17 Aug 2022 • Dong Huang, Qingwen Bu, Yuhao QING, Haowen Pi, Sen Wang, Heming Cui
Compared to all methods that do not use additional data for training, our models achieve 67. 3% and 41. 5% robust accuracy on CIFAR-10 and CIFAR-100 (improving upon the state-of-the-art by +7. 23% and +9. 07%).
no code implementations • 29 Sep 2021 • Fanxin Li, Shixiong Zhao, Haowen Pi, Yuhao QING, Yichao Fu, Sen Wang, Heming Cui
Neural Architecture Search (NAS) automatically searches for well-performed network architectures from a given search space.