Group Normalization is a normalization layer that divides channels into groups and normalizes the features within each group. GN does not exploit the batch dimension, and its computation is independent of batch sizes. In the case where the group size is 1, it is equivalent to Instance Normalization.
As motivation for the method, many classical features like SIFT and HOG had groupwise features and involved groupwise normalization. For example, a HOG vector is the outcome of several spatial cells where each cell is represented by a normalized orientation histogram.
Formally, Group Normalization is defined as:
$$ \mu_{i} = \frac{1}{m}\sum_{k\in\mathcal{S}_{i}}x_{k} $$
$$ \sigma^{2}_{i} = \frac{1}{m}\sum_{k\in\mathcal{S}_{i}}\left(x_{k}\mu_{i}\right)^{2} $$
$$ \hat{x}_{i} = \frac{x_{i}  \mu_{i}}{\sqrt{\sigma^{2}_{i}+\epsilon}} $$
Here $x$ is the feature computed by a layer, and $i$ is an index. Formally, a Group Norm layer computes $\mu$ and $\sigma$ in a set $\mathcal{S}_{i}$ defined as: $\mathcal{S}_{i} = ${$k \mid k_{N} = i_{N} ,\lfloor\frac{k_{C}}{C/G}\rfloor = \lfloor\frac{I_{C}}{C/G}\rfloor $}.
Here $G$ is the number of groups, which is a predefined hyperparameter ($G = 32$ by default). $C/G$ is the number of channels per group. $\lfloor$ is the floor operation, and the final term means that the indexes $i$ and $k$ are in the same group of channels, assuming each group of channels are stored in a sequential order along the $C$ axis.
Source:TASK  PAPERS  SHARE 

Semantic Segmentation  7  18.92% 
Object Detection  6  16.22% 
Instance Segmentation  5  13.51% 
Image Classification  3  8.11% 
Image Generation  2  5.41% 
SelfSupervised Learning  2  5.41% 
FewShot Learning  2  5.41% 
Action Recognition  1  2.70% 
SelfSupervised Action Recognition  1  2.70% 
COMPONENT  TYPE 


🤖 No Components Found  You can add them if they exist; e.g. Mask RCNN uses RoIAlign 