Search Results for author:
Found 19 papers, 10 papers with code
Aligning LLMs for FL-free Program Repair
Our core insight is that LLM's APR capability can be greatly improved by simply aligning the output to their training objective and allowing them to refine the whole program without first performing fault localization.
A & B == B & A: Triggering Logical Reasoning Failures in Large Language Models
In addition, the test cases of LogicAsker can be further used to design demonstration examples for in-context learning, which effectively improves the logical reasoning ability of LLMs, e. g., 10\% for GPT-4.
Retromorphic Testing: A New Approach to the Test Oracle Problem
Inspired by the mathematical concept of inverse function, we present Retromorphic Testing, a novel black-box testing methodology.
An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software
Moreover, through retraining the models with the test cases generated by OASIS, the robustness of the moderation model can be improved without performance degradation.
Automated Testing and Improvement of Named Entity Recognition Systems
For automated repairing, TIN achieves a high error reduction rate (26. 8%-50. 6%) over the four systems under test, which successfully repairs 1, 056 out of the 1, 877 reported NER errors.
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
We propose a novel framework CipherChat to systematically examine the generalizability of safety alignment to non-natural languages -- ciphers.
Validating Multimedia Content Moderation Software via Semantic Fusion
To this end, content moderation software has been widely deployed on these platforms to detect and blocks toxic content.
BiasAsker: Measuring the Bias in Conversational AI System
Particularly, it is hard to generate inputs that can comprehensively trigger potential bias due to the lack of data containing both social groups as well as biased properties.
MTTM: Metamorphic Testing for Textual Content Moderation Software
In addition, we leverage the test cases generated by MTTM to retrain the model we explored, which largely improves model robustness (0% to 5. 9% EFR) while maintaining the accuracy on the original test set.
AEON: A Method for Automatic Evaluation of NLP Test Cases
However, in practice, many of the generated test cases fail to preserve similar semantic meaning and are unnatural (e. g., grammar errors), which leads to a high false alarm rate and unnatural test cases.
Empirical Standards for Software Engineering Research
Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e. g. a questionnaire survey).
Software Engineering General Literature
Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics
To fill this significant gap and facilitate more research on AI-driven log analytics, we have collected and released loghub, a large collection of system log datasets.
Software Engineering
Testing Machine Translation via Referential Transparency
Machine translation software has seen rapid progress in recent years due to the advancement of deep neural networks.
Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression
Data compression is essential to reduce the cost of log storage.
Databases Software Engineering
Structure-Invariant Testing for Machine Translation
Despite its apparent importance, validating the robustness of machine translation systems is very difficult and has, therefore, been much under-explored.
Tools and Benchmarks for Automated Log Parsing
Logs are imperative in the development and maintenance process of many software systems.
Software Engineering
Testing Untestable Neural Machine Translation: An Industrial Case
To fill the gap of lacking test oracle for in-vivo testing of an NMT system, in this paper, we propose a new approach for automatically identifying translation failures, without requiring reference translations for a translation task; our approach can directly serve as a test oracle for in-vivo testing.
A Directed Acyclic Graph Approach to Online Log Parsing
A typical log-based system reliability management procedure is to first parse log messages because of their unstructured format; and apply data mining techniques on the parsed logs to obtain critical system behavior information.
Software Engineering
Semantically Consistent Image Completion with Fine-grained Details
Image completion has achieved significant progress due to advances in generative adversarial networks (GANs).
