We use the axis-aligned bounding box with a class ID for each defect in the tested images. As illustrated in above, we annotate six common types of PCB defects: open, short, mousebite, spur, pin hole, and spurious copper. Since there are only a few defects in the real tested image, we manually arguement some artificial defects on each tested image according to the PCB defect patterns, which leads to around 3 to 12 defects in each 640 x 640 image. The number of PCB defects is shown in the following figure. We separate 1,000 images as a training set and the remains as a test set. Each annotated image owns an annotation file with the same filename, e.g.00041000_test.jpg, 00041000_temp.jpg, and 00041000.txt are the tested image, template image, and the corresponding annotation file. Each defect on the tested image is annotated as the format:x1,y1,x2,y2, type, where (x1,y1) and (x2,y2) is the top left and the bottom right corner of the bounding box of the defect. type is an integer ID that follows the matches: 0-background (not used), 1-open, 2-short, 3-mousebite, 4-spur, 5-copper, 6-pin-hole.
The annotation tool is now available with the source code in the ./tools directory.
The average precision rate and F-score are used for evaluation. A detection is correct only if the intersection of unit (IoU) between the detected bounding box and any of the ground truth box with the same class is larger than 0.33. F-score is calculated as: F-score=2PR/(P+R), where P and R is the precision and recall rate. Notice that F-score is threshold-sensitive, which means you could adjust your score threshold to obtain a better result. Although F-score is not as fair as the mAP criteria but more practical since a threshold should always be given when deploying the model and not all of the algorithms have a score evaluation for the target. Thus, F-score and mAP are both under consideration in the benchmarks.
The evaluation script for mAP and F-score are borrowed from Icdar2015 evaluation scripts with a small modification (You may first register an account.). Here, we give the modified evaluation scripts and the ground truth gt.zip file of the test set in evaluation/ directory. You can evaluate your own method by following the instructions: * run your algorithm and save the detected result for each image named as image_name.txt, where the image_name should be the same as in the gt.zip. You should follow the format of evaluation/gt.zip except that the output description of each defect from your algorithm should be: x1,y1,x2,y2, confidence, type, where (x1,y1) and (x2,y2) is the top left and the bottom right corner of the bounding box of the defect. confidence is a float number to show how confident you believe such detection results. type is a string and should be one of the following: open, short,mousebite,spur,copper,pin-hole. Notice there is no space except the comma. * zip your .txt file to res.zip. (You should not contain any sub-directory in the res.zip file) * run the evaluation script: python script.py -s=res.zip -g=gt.zip
This section with the source code will be public after the acceptance of the paper.
Here we show some results of our model based on deep neural networks. Our model achieves 98.6% mAp, 98.2% F-score @ 62FPS. More statistical analysis will be public after the acceptance of the paper. The green bounding box is the predicted location of the PCB defect with the confidence on the top of each.
This work is contributed by the paper On-line PCB Defect Detector On A New PCB Defect Dataset. You can only use this dataset for research purposes.
Contributor for paper with code:
* Allena Venkata Sai Abhishek
Paper | Code | Results | Date | Stars |
---|