Search Results for author:
Found 6 papers, 2 papers with code
Quality Assessment of Prompts Used in Code Generation
We found that code generation evaluation benchmarks mainly focused on Python and coding exercises and had very limited contextual dependencies to challenge the model.
Generate and Pray: Using SALLMS to Evaluate the Security of LLM Generated Code
This framework has three major components: a novel dataset of security-centric Python prompts, an evaluation environment to test the generated code, and novel metrics to evaluate the models' performance from the perspective of secure code generation.
A Lightweight Framework for High-Quality Code Generation
FRANC includes a static filter to make the generated code compilable with heuristics and a quality-aware ranker to sort the code snippets based on a quality score.
Using Large Language Models to Generate JUnit Tests: An Empirical Study
A code generation model generates code by taking a prompt from a code comment, existing code, or a combination of both.
Enhancing Automated Program Repair through Fine-tuning and Prompt Engineering
We applied PLBART and CodeT5, two state-of-the-art language models that are pre-trained with both PL and NL, on two such natural language-based program repair datasets and found that the pre-trained language models fine-tuned with datasets containing both code review and subsequent code changes notably outperformed each of the previous models.
SecurityEval Dataset: Mining Vulnerability Examples to Evaluate Machine Learning-Based Code Generation Techniques
For this reason, evaluating the security of a code generation model is a must.
