Cross-Iteration Batch Normalization

13 Feb 2020Zhuliang YaoYue CaoShuxin ZhengGao HuangStephen Lin

A well-known issue of Batch Normalization is its significantly reduced effectiveness in the case of small mini-batch sizes. When a mini-batch contains few examples, the statistics upon which the normalization is defined cannot be reliably estimated from it during a training iteration... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Object Detection COCO test-dev Mask R-CNN (ResNet-101-FPN, CBN) box AP 40.1 # 55
AP50 60.5 # 48
AP75 44.1 # 49
APS 35.8 # 3
APM 57.3 # 1
APL 38.5 # 65