DNN Testing
9 papers with code • 0 benchmarks • 0 datasets
Testing the reliability of DNNs.
Benchmarks
These leaderboards are used to track progress in DNN Testing
Most implemented papers
DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems
SullyChen/Autopilot-TensorFlow
•
•
Furthermore, DeepBillboard is sufficiently robust and resilient for generating physical-world adversarial billboard tests for real-world driving under various weather conditions.
Testing DNN Image Classifiers for Confusion & Bias Errors
We found that many of the reported erroneous cases in popular DNN image classifiers occur because the trained models confuse one class with another or show biases towards some classes over others.
Boosting Operational DNN Testing Efficiency through Conditioning
With the increasing adoption of Deep Neural Network (DNN) models as integral parts of software systems, efficient operational testing of DNNs is much in demand to ensure these models' actual performance in field conditions.
Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning
cas-lrj/deeppac
•
•
It is shown that DeepPAC outperforms the state-of-the-art statistical method PROVERO, and it achieves more practical robustness analysis than the formal verification tool ERAN.
Distribution-Aware Testing of Neural Networks Using Generative Models
swa112003/DistributionAwareDNNTesting
•
•
Using deep generative model based input validation, we show that all the three techniques generate significant number of invalid test inputs.
Perception Matters: Detecting Perception Failures of VQA Models Using Metamorphic Testing
MetaVQA checks whether the answer to (i, q) satisfies metamorphic relationships (MRs), denoting perception consistency, with the composed answers of transformed questions and images.
Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterion
yuanyuan-yuan/neural-coverage
•
•
We demonstrate that NLC is significantly correlated with the diversity of a test suite across a number of tasks (classification and generation) and data formats (image and text).
Black-Box Testing of Deep Neural Networks Through Test Case Diversity
In this paper, we investigate black-box input diversity metrics as an alternative to white-box coverage criteria.
An Overview of Structural Coverage Metrics for Testing Neural Networks
dnncov/dnncov
•
•
Deep neural network (DNN) models, including those used in safety-critical domains, need to be thoroughly tested to ensure that they can reliably perform well in different scenarios.
