Search Results for author:
Found 16 papers, 4 papers with code
PapersPlease: A Benchmark for Evaluating Motivational Values of Large Language Models Based on ERG Theory
Evaluating the performance and biases of large language models (LLMs) through role-playing scenarios is becoming increasingly common, as LLMs often exhibit biased behaviors in these contexts.
Capturing Semantic Flow of ML-based Systems
We propose the idea of semantic flow, introduce two examples using a DNN and an LLM agent, and finally sketch its properties and how it can be used to adapt existing dynamic analysis techniques for use in ML-based software systems.
Challenges in Testing Large Language Model Based Software: A Faceted Taxonomy
This paper presents a taxonomy for LLM test case design, informed by both the research literature, our experience, and open-source tools that represent the state of practice.
DANDI: Diffusion as Normative Distribution for Deep Neural Network Input
While SA has been widely adopted as a test prioritization method, its major weakness is the fact that the computation of the metric requires access to the training dataset, which is often not allowed in real-world use cases.
COSMosFL: Ensemble of Small Language Models for Fault Localisation
LLMs are rapidly being adopted to build powerful tools and agents for software engineering, but most of them rely heavily on extremely large closed-source models.
Predictive Prompt Analysis
LLMs, due to their training, are sensitive to how exactly a question is presented, also known as prompting.
Adaptive Testing for LLM-Based Applications: A Diversity-based Approach
The recent surge of building software systems powered by Large Language Models (LLMs) has led to the development of various testing frameworks, primarily focused on treating prompt templates as the unit of testing.
Real Faults in Deep Learning Fault Benchmarks: How Real Are They?
As the adoption of Deep Learning (DL) systems continues to rise, an increasing number of approaches are being proposed to test these systems, localise faults within them, and repair those faults.
An Empirical Study of Fault Localisation Techniques for Deep Learning
Results indicate that \dfd is the most effective tool, achieving an average recall of 0. 61 and precision of 0. 41 on our benchmark.
CSA-Trans: Code Structure Aware Transformer for AST
When applying the Transformer architecture to source code, designing a good self-attention mechanism is critical as it affects how node relationship is extracted from the Abstract Syntax Trees (ASTs) of the source code.
Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing
On average, DroidAgent achieved 61% activity coverage, compared to 51% for current state-of-the-art GUI testing techniques.
Genetic Improvement @ ICSE 2020
Following Prof. Mark Harman of Facebook's keynote and formal presentations (which are recorded in the proceedings) there was a wide ranging discussion at the eighth international Genetic Improvement workshop, GI-2020 @ ICSE (held as part of the 42nd ACM/IEEE International Conference on Software Engineering on Friday 3rd July 2020).
Reducing DNN Labelling Cost using Surprise Adequacy: An Industrial Case Study for Autonomous Driving
The development process in use consists of multiple iterations of data collection, labelling, training, and evaluation.
SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation
The testing of Deep Neural Networks (DNNs) has become increasingly important as DNNs are widely adopted by safety critical systems.
Arachne: Search Based Repair of Deep Neural Networks
The rapid and widespread adoption of Deep Neural Networks (DNNs) has called for ways to test their behaviour, and many testing approaches have successfully revealed misbehaviour of DNNs.
Guiding Deep Learning System Testing using Surprise Adequacy
Recently, a number of coverage criteria based on neuron activation values have been proposed.
