Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having highly similar looking animals that interact more closely than in typical multi-human scenarios. To take up this challenge, we build on DeepLabCut, an open-source pose estimation toolbox, and provide high-performance animal assembly and tracking—features required for multi-animal scenarios. Furthermore, we integrate the ability to predict an animal’s identity to assist tracking (in case of occlusions). We illustrate the power of this framework with four datasets varying in complexity, which we release to serve as a benchmark for future algorithm development.

PDF Abstract


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Animal Pose Estimation Fish-100 DLCRNet_ms4graph9 mAP 71.6 # 4
Animal Pose Estimation Marmoset-8K DLCRNet_ms-graph34 mAP 89 # 4
Animal Pose Estimation TriMouse-161 ResNet50_s4graph11 mAP 93 # 3
Animal Pose Estimation TriMouse-161 DLCRNet_ms4graph11 mAP 92 # 4


No methods listed for this paper. Add relevant methods here