DNN Testing

8 papers with code • 0 benchmarks • 0 datasets

Testing the reliability of DNNs.

Most implemented papers

Testing DNN Image Classifiers for Confusion & Bias Errors

ARiSE-Lab/DeepInspect 20 May 2019

We found that many of the reported erroneous cases in popular DNN image classifiers occur because the trained models confuse one class with another or show biases towards some classes over others.

Boosting Operational DNN Testing Efficiency through Conditioning

Lizenan1995/DNNOpAcc 6 Jun 2019

With the increasing adoption of Deep Neural Network (DNN) models as integral parts of software systems, efficient operational testing of DNNs is much in demand to ensure these models' actual performance in field conditions.

Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning

cas-lrj/deeppac 25 Jan 2021

It is shown that DeepPAC outperforms the state-of-the-art statistical method PROVERO, and it achieves more practical robustness analysis than the formal verification tool ERAN.

Distribution-Aware Testing of Neural Networks Using Generative Models

swa112003/DistributionAwareDNNTesting 26 Feb 2021

Using deep generative model based input validation, we show that all the three techniques generate significant number of invalid test inputs.

Perception Matters: Detecting Perception Failures of VQA Models Using Metamorphic Testing

MetaVQA/MetaVQA CVPR 2021

MetaVQA checks whether the answer to (i, q) satisfies metamorphic relationships (MRs), denoting perception consistency, with the composed answers of transformed questions and images.

Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterion

yuanyuan-yuan/neural-coverage 3 Dec 2021

We demonstrate that NLC is significantly correlated with the diversity of a test suite across a number of tasks (classification and generation) and data formats (image and text).

Black-Box Testing of Deep Neural Networks Through Test Case Diversity

zohreh-aaa/dnn-testing 20 Dec 2021

In this paper, we investigate black-box input diversity metrics as an alternative to white-box coverage criteria.

An Overview of Structural Coverage Metrics for Testing Neural Networks

dnncov/dnncov 5 Aug 2022

Deep neural network (DNN) models, including those used in safety-critical domains, need to be thoroughly tested to ensure that they can reliably perform well in different scenarios.