Matrix Non-Maximum Suppression

Matrix NMS, or Matrix Non-Maximum Suppression, performs non-maximum suppression with parallel matrix operations in one shot. It is motivated by Soft-NMS. Soft-NMS decays the other detection scores as a monotonic decreasing function $f(iou)$ of their overlaps. By decaying the scores according to IoUs recursively, higher IoU detections will be eliminated with a minimum score threshold. However, such process is sequential like traditional Greedy NMS and can not be implemented in parallel.

Matrix NMS views this process from another perspective by considering how a predicted mask $m_{j}$ being suppressed. For $m_{j}$, its decay factor is affected by: (a) The penalty of each prediction $m_{i}$ on $m_{j}$ $\left(s_{i}>s_{j}\right)$, where $s_{i}$ and $s_{j}$ are the confidence scores; and (b) the probability of $m_{i}$ being suppressed. For (a), the penalty of each prediction $m_{i}$ on $m_{j}$ could be easily computed by $f\left(\right.$ iou $\left._{i, j}\right)$. For (b), the probability of $m_{i}$ being suppressed is not so elegant to be computed. However, the probability usually has positive correlation with the IoUs. So here we directly approximate the probability by the most overlapped prediction on $m_{i}$ as

$$ f\left(\text { iou. }_{, i}\right)=\min_{\forall s_{k}>s_{i}} f\left(\text { iou }_{k, i}\right) $$

To this end, the final decay factor becomes

$$ \operatorname{decay}_{j}=\min_{\forall s_{i}>s_{j}} \frac{f\left(\text { iou }_{i, j}\right)}{f\left(\text { iou }_{\cdot, i}\right)} $$

and the updated score is computed by $s_{j}=s_{j} \cdot$ decay $_{j} .$ The authors consider the two most simple decremented functions, denoted as linear $f\left(\right.$ iou $\left._{i, j}\right)=1-$ iou $_{i, j}$, and Gaussian $f\left(\right.$ iou $\left._{i, j}\right)=\exp \left(-\frac{i o u_{i, j}^{2}}{\sigma}\right)$.