1 code implementation • 27 Mar 2024 • Yangruibo Ding, Marcus J. Min, Gail Kaiser, Baishakhi Ray
Pre-trained code language models have achieved promising performance in code generation and improved the programming efficiency of human developers.
1 code implementation • 21 Oct 2023 • Marcus J. Min, Yangruibo Ding, Luca Buratti, Saurabh Pujar, Gail Kaiser, Suman Jana, Baishakhi Ray
In this paper, we first formally define the self-consistency of Code LLMs and then design a framework, IdentityChain, which effectively and efficiently evaluates the self-consistency and conventional accuracy of a model at the same time.