no code implementations • 21 Mar 2025 • David N. Palacio, Dipin Khati, Daniel Rodriguez-Cardenas, Alejandro Velasco, Denys Poshyvanyk
Moreover, participants of this study highlighted Code$Q$'s ability to show a causal relationship between the input and output of the model with readable and informative explanations on code completion and test generation tasks.
no code implementations • 18 Mar 2025 • Dipin Khati, Yijin Liu, David N. Palacio, Yixuan Zhang, Denys Poshyvanyk
To bring clarity to the current research status and identify opportunities for future work, we conducted a comprehensive review of $88$ papers: a systematic literature review of $18$ papers focused on LLMs in SE, complemented by an analysis of 70 papers from broader trust literature.
1 code implementation • 18 Feb 2025 • Liangying Shao, Yanfu Yan, Denys Poshyvanyk, Jinsong Su
Deep learning-based code generation has completely transformed the way developers write programs today.
no code implementations • 10 Feb 2025 • Daniel Rodriguez-Cardenas, Alejandro Velasco, Denys Poshyvanyk
SnipGen aims to mitigate data contamination by generating robust testbeds and crafting tailored data points to assist researchers and practitioners in evaluating LLMs for code-related tasks.
no code implementations • 6 Feb 2025 • Trevor Stalnaker, Nathan Wintersgill, Oscar Chaparro, Laura A. Heymann, Massimiliano Di Penta, Daniel M German, Denys Poshyvanyk
First, we evaluate the current state of documentation in the Hugging Face supply chain, report real-world examples of shortcomings, and offer actionable suggestions for improvement.
no code implementations • 3 Feb 2025 • Alejandro Velasco, Aya Garryyeva, David N. Palacio, Antonio Mastropaolo, Denys Poshyvanyk
Recent advancements in Large Language Models (LLMs) have paved the way for Large Code Models (LCMs), enabling automation in complex software engineering tasks, such as code generation, software testing, and program comprehension, among others.
no code implementations • 25 Dec 2024 • Alejandro Velasco, Daniel Rodriguez-Cardenas, Luftar Rahman Alif, David N. Palacio, Denys Poshyvanyk
These findings highlight the effectiveness of our benchmark in evaluating LLMs, providing valuable insights into their reliability and their propensity to introduce code smells in code generation tasks.
no code implementations • 6 Dec 2024 • David N. Palacio, Daniel Rodriguez-Cardenas, Denys Poshyvanyk, Kevin Moran
In this paper, we propose an approach, TraceXplainer, for using information theory metrics to evaluate and better understand the performance (limits) of unsupervised traceability techniques.
1 code implementation • 28 Nov 2024 • Anamaria Mojica-Hanke, David Nader Palacio, Denys Poshyvanyk, Mario Linares-Vásquez, Steffen Herbold
Objective: This study aims to contribute to the knowledge, about the synergy between ML and SE from the perspective of SE researchers, by providing insights into the practices followed when researching, teaching, and reviewing SE studies that apply ML.
no code implementations • 16 Nov 2024 • Trevor Stalnaker, Nathan Wintersgill, Oscar Chaparro, Laura A. Heymann, Massimiliano Di Penta, Daniel M German, Denys Poshyvanyk
Despite the utility that Generative AI (GenAI) tools provide for tasks such as writing code, the use of these tools raises important legal questions and potential risks, particularly those associated with copyright law.
no code implementations • 12 Jul 2024 • David N. Palacio, Daniel Rodriguez-Cardenas, Alejandro Velasco, Dipin Khati, Kevin Moran, Denys Poshyvanyk
Trustworthiness and interpretability are inextricably linked concepts for LLMs.
no code implementations • 11 Jul 2024 • Yanfu Yan, Nathan Cooper, Oscar Chaparro, Kevin Moran, Denys Poshyvanyk
Video-based bug reports are increasingly being used to document bugs for programs centered around a graphical user interface (GUI).
no code implementations • 23 Aug 2023 • Daniel Rodriguez-Cardenas, David N. Palacio, Dipin Khati, Henry Burke, Denys Poshyvanyk
We illustrate the insights of our benchmarking strategy by conducting a case study on the performance of ChatGPT under distinct prompt engineering methods.
no code implementations • 7 Aug 2023 • David N Palacio, Alejandro Velasco, Daniel Rodriguez-Cardenas, Kevin Moran, Denys Poshyvanyk
To this end, this paper introduces ASTxplainer, an explainability method specific to LLMs for code that enables both new methods for LLM evaluation and visualizations of LLM predictions that aid end-users in understanding model predictions.
no code implementations • 7 Feb 2023 • David N. Palacio, Alejandro Velasco, Nathan Cooper, Alvaro Rodriguez, Kevin Moran, Denys Poshyvanyk
To demonstrate the practical benefit of $do_{code}$, we illustrate the insights that our framework can provide by performing a case study on two popular deep learning architectures and ten NCMs.
no code implementations • 3 Jan 2023 • Kevin Moran, Ali Yachnes, George Purnell, Junayed Mahmud, Michele Tufano, Carlos Bernal-Cárdenas, Denys Poshyvanyk, Zach H'Doubler
This paper offers one of the first comprehensive empirical investigations into the connection between GUIs and functional, natural language descriptions of software.
1 code implementation • 22 Jan 2021 • Nathan Cooper, Carlos Bernal-Cárdenas, Oscar Chaparro, Kevin Moran, Denys Poshyvanyk
Given the importance of visual information to the process of identifying and understanding such bugs, users are increasingly making use of screenshots and screen-recordings as a means to report issues to developers.
Optical Character Recognition
Optical Character Recognition (OCR)
+1
1 code implementation • 17 Sep 2020 • Prem Devanbu, Matthew Dwyer, Sebastian Elbaum, Michael Lowry, Kevin Moran, Denys Poshyvanyk, Baishakhi Ray, Rishabh Singh, Xiangyu Zhang
The intent of this report is to serve as a potential roadmap to guide future work that sits at the intersection of SE & DL.
no code implementations • 14 Sep 2020 • Cody Watson, Nathan Cooper, David Nader Palacio, Kevin Moran, Denys Poshyvanyk
An increasingly popular set of techniques adopted by software engineering (SE) researchers to automate development tasks are those rooted in the concept of Deep Learning (DL).
no code implementations • 18 May 2020 • Carlos Bernal-Cárdenas, Nathan Cooper, Kevin Moran, Oscar Chaparro, Andrian Marcus, Denys Poshyvanyk
In light of unique mobile development constraints, including swift release cycles and rapidly evolving platforms, automated techniques for analyzing all types of rich software artifacts provide benefit to mobile developers.
no code implementations • 18 May 2020 • Kevin Moran, David N. Palacio, Carlos Bernal-Cárdenas, Daniel McCrystal, Denys Poshyvanyk, Chris Shenefiel, Jeff Johnson
To this end, we design and implement a HierarchiCal PrObabilistic Model for SoftwarE Traceability (Comet) that is able to infer candidate trace links.
no code implementations • 12 Feb 2020 • Michele Tufano, Jason Kimko, Shiya Wang, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Denys Poshyvanyk
To this aim, two characteristics of mutation testing frameworks are of paramount importance: (i) they should generate mutants that are representative of real faults; and (ii) they should provide a complete tool chain able to automatically generate, inject, and test the mutants.
no code implementations • 25 Jan 2019 • Michele Tufano, Jevgenija Pantiuchina, Cody Watson, Gabriele Bavota, Denys Poshyvanyk
We show that, when applied in a narrow enough context (i. e., small/medium-sized pairs of methods before/after the pull request changes), NMT can automatically replicate the changes implemented by developers during pull requests in up to 36% of the cases.
no code implementations • 27 Dec 2018 • Michele Tufano, Cody Watson, Gabriele Bavota, Massimiliano Di Penta, Martin White, Denys Poshyvanyk
Starting from code fixed by developers in the context of a bug-fix, our empirical evaluation showed that our models are able to predict mutants that resemble original fixed bugs in between 9% and 45% of the cases (depending on the model).
Software Engineering
2 code implementations • 24 Dec 2018 • Zimin Chen, Steve Kommrusch, Michele Tufano, Louis-Noël Pouchet, Denys Poshyvanyk, Martin Monperrus
This paper presents a novel end-to-end approach to program repair based on sequence-to-sequence learning.
no code implementations • 7 Feb 2018 • Kevin Moran, Carlos Bernal-Cárdenas, Michael Curcio, Richard Bonett, Denys Poshyvanyk
It is common practice for developers of user-facing software to transform a mock-up of a graphical user interface (GUI) into code.
1 code implementation • 15 Jul 2017 • Martin White, Michele Tufano, Matias Martinez, Martin Monperrus, Denys Poshyvanyk
We aim to reason about the repair ingredients by using code similarities to prioritize and transform statements in a codebase for patch generation.
Software Engineering