Search Results for author:
Found 4 papers, 4 papers with code
Measuring The Impact Of Programming Language Distribution
Training a model on a balanced corpus results in, on average, 12. 34% higher $pass@k$ across all tasks and languages compared to the baseline.
Evaluating How Fine-tuning on Bimodal Data Effects Code Generation
Despite the increase in popularity of language models for code generation, it is still unknown how training on bimodal coding forums affects a model's code generation performance and reliability.
Evaluating Prompts Across Multiple Choice Tasks In a Zero-Shot Setting
That \textit{prompts} the question: how do the qualities of a prompt effects its performance?
Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation
We evaluate prior state-of-the-art CoNaLa models with this additional data and find that our proposed method of using the body and mined data beats the BLEU score of the prior state-of-the-art by $71. 96\%$.
Ranked #1 on
Code Generation
on CoNaLa-Ext
