Domain-Adaptive Self-Supervised Pre-Training for Face & Body Detection in Drawings

19 Nov 2022  ยท  BarฤฑลŸ Batuhan Topal, Deniz Yuret, Tevfik Metin Sezgin ยท

Drawings are powerful means of pictorial abstraction and communication. Understanding diverse forms of drawings, including digital arts, cartoons, and comics, has been a major problem of interest for the computer vision and computer graphics communities. Although there are large amounts of digitized drawings from comic books and cartoons, they contain vast stylistic variations, which necessitate expensive manual labeling for training domain-specific recognizers. In this work, we show how self-supervised learning, based on a teacher-student network with a modified student network update design, can be used to build face and body detectors. Our setup allows exploiting large amounts of unlabeled data from the target domain when labels are provided for only a small subset of it. We further demonstrate that style transfer can be incorporated into our learning pipeline to bootstrap detectors using a vast amount of out-of-domain labeled images from natural images (i.e., images from the real world). Our combined architecture yields detectors with state-of-the-art (SOTA) and near-SOTA performance using minimal annotation effort. Our code can be accessed from https://github.com/barisbatuhan/DASS_Detector.

PDF Abstract

Results from the Paper


 Ranked #1 on Weakly Supervised Object Detection on Comic2k (using extra training data)

     Get a GitHub badge
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Body Detection Clipart1k DASS-Detector (YOLOX XL) MAP 83.59 # 1
Weakly Supervised Object Detection Clipart1k DASS-Detector (YOLOX Tiny) MAP 64.25 # 2
Body Detection Comic2k DASS-Detector (YOLOX XL) MAP 73.65 # 1
Weakly Supervised Object Detection Comic2k DASS-Detector (YOLOX Tiny) MAP 67.41 # 1
Body Detection DCM DASS-Detector (YOLOX Tiny) Average Precision 87.06 # 2
Face Detection DCM DASS-Detector (YOLOX Tiny) Average Precision 77.40 # 1
Body Detection DCM DASS-Detector (YOLOX XL) Average Precision 86.14 # 1
Face Detection DCM DASS-Detector (YOLOX XL) Average Precision 77.40 # 1
Face Detection iCartoonFace DASS-Detector (YOLOX Tiny) Average Precision 87.75 # 2
Face Detection iCartoonFace DASS-Detector (YOLOX XL) Average Precision 90.01 # 1
Body Detection Manga109 DASS-Detector (YOLOX XL) Average Precision 87.98 # 1
Face Detection Manga109 DASS-Detector (YOLOX XL) Average Precision 87.88 # 1
Object Detection Manga109 DASS-Detector (YOLOX XL) Average Precision 87.93 # 1
Object Detection Manga109 DASS-Detector (YOLOX Tiny) Average Precision 87.46 # 2
Weakly Supervised Object Detection Watercolor2k DASS-Detector (YOLOX Tiny) MAP 71.53 # 2
Body Detection Watercolor2k DASS-Detector (YOLOX XL) MAP 89.81 # 1

Methods


No methods listed for this paper. Add relevant methods here