Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning

CVPR 2021  ยท  Bin Li, Yin Li, Kevin W. Eliceiri ยท

We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems. GitHub repository: https://github.com/binli123/dsmil-wsi

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Multiple Instance Learning CAMELYON16 DSMIL-LC AUC 0.9165 # 10
ACC 0.8992 # 7
Multiple Instance Learning CAMELYON16 DSMIL AUC 0.8944 # 13
ACC 0.8682 # 12
Multiple Instance Learning Elephant DSMIL ACC 0.929 # 1
Multiple Instance Learning Musk v1 DSMIL ACC 0.947 # 2
Multiple Instance Learning Musk v2 DSMIL ACC 0.934 # 1
Multiple Instance Learning TCGA DSMIL-LC AUC 0.9583 # 5
ACC 0.9286 # 3
Multiple Instance Learning TCGA DSMIL AUC 0.9633 # 2
ACC 0.9190 # 4

Methods