Search Results for author:
Found 6 papers, 2 papers with code
Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models
In this paper, we empirically analyze the differences in coding style between the code generated by mainstream Code LLMs and the code written by human developers, and summarize coding style inconsistency taxonomy.
Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG
In addition, our user study shows that the vulnerability knowledge generated by Vul-RAG can serve as high-quality explanations which can improve the manual detection accuracy from 0. 60 to 0. 77.
Resolving Crash Bugs via Large Language Models: An Empirical Study
Evaluating IntDiagSolver on multiple LLMs reveals consistent enhancement in the accuracy of crash bug resolution, including ChatGPT, Claude, and CodeLlama.
CodeGen4Libs: A Two-Stage Approach for Library-Oriented Code Generation
To evaluate the effectiveness of our approach, we conduct extensive experiments on a dataset of 403, 780 data items.
ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation
Third, we find that generating the entire class all at once (i. e. holistic generation strategy) is the best generation strategy only for GPT-4 and GPT-3. 5, while method-by-method generation (i. e. incremental and compositional) is better strategies for the other models with limited ability of understanding long instructions and utilizing the middle information.
Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation
First, for the zero-shot setting, instructed LLMs are very competitive on code comprehension and generation tasks and sometimes even better than small SOTA models specifically fine-tuned on each downstream task.
