Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels

15 Aug 2023  ·  Ching-Hsun Tseng, Shao-Ju Chien, Po-Shen Wang, Shin-Jye Lee, Wei-Huan Hu, Bin Pu, Xiao-jun Zeng ·

Motion mode (M-mode) recording is an essential part of echocardiography to measure cardiac dimension and function. However, the current diagnosis cannot build an automatic scheme, as there are three fundamental obstructs: Firstly, there is no open dataset available to build the automation for ensuring constant results and bridging M-mode echocardiography with real-time instance segmentation (RIS); Secondly, the examination is involving the time-consuming manual labelling upon M-mode echocardiograms; Thirdly, as objects in echocardiograms occupy a significant portion of pixels, the limited receptive field in existing backbones (e.g., ResNet) composed from multiple convolution layers are inefficient to cover the period of a valve movement. Existing non-local attentions (NL) compromise being unable real-time with a high computation overhead or losing information from a simplified version of the non-local block. Therefore, we proposed RAMEM, a real-time automatic M-mode echocardiography measurement scheme, contributes three aspects to answer the problems: 1) provide MEIS, a dataset of M-mode echocardiograms for instance segmentation, to enable consistent results and support the development of an automatic scheme; 2) propose panel attention, local-to-global efficient attention by pixel-unshuffling, embedding with updated UPANets V2 in a RIS scheme toward big object detection with global receptive field; 3) develop and implement AMEM, an efficient algorithm of automatic M-mode echocardiography measurement enabling fast and accurate automatic labelling among diagnosis. The experimental results show that RAMEM surpasses existing RIS backbones (with non-local attention) in PASCAL 2012 SBD and human performances in real-time MEIS tested. The code of MEIS and dataset are available at https://github.com/hanktseng131415go/RAME.

PDF Abstract

Datasets


Introduced in the Paper:

MEIS

Used in the Paper:

PASCAL VOC
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Real-time Instance Segmentation MEIS RAMEM UPANet80 V2 maskAP 43.09 # 1
boxAP 51.2 # 1
FLOPs (G) 100.85 # 2
Size (M) 40.28 # 1
avgAP (mask AP + box AP) 47.15 # 1
Frame (fps) 52.22 # 1
Real-time instance measurement MEIS MENN (RAMEM UPANet80 V2) Mean MAE 20.6 # 1
Mean MSE 12 # 1
Frame (fps) 13.76 # 2
Real-time Instance Segmentation MEIS maYOLACT ResNet50 maskAP 42.99 # 2
boxAP 49.59 # 2
FLOPs (G) 0.4826 # 1
Size (M) 30.38 # 2
avgAP (mask AP + box AP) 46.29 # 2
Frame (fps) 36.13 # 2
Real-time instance measurement MEIS AMEM (RAMEM UPANet80 V2) Mean MAE 0.144 # 2
Mean MSE 0.036 # 2
Frame (fps) 24.45 # 1
Real-time Instance Segmentation PASCAL VOC 2012 maYOLACT ResNet50 maskAP 37.27 # 2
boxAP 37.50 # 2
FLOPs (G) 48.26 # 1
Size (M) 30.41 # 2
avgAP (mask AP + box AP) 37.39 # 2
Frame (fps) 81.27 # 1
Real-time Instance Segmentation PASCAL VOC 2012 YOLACT ResNet50 maskAP 35.12 # 3
boxAP 36.65 # 3
FLOPs (G) 48.26 # 1
Size (M) 30.41 # 2
avgAP (mask AP + box AP) 35.73 # 3
Frame (fps) 81.11 # 2
Real-time Instance Segmentation PASCAL VOC 2012 RAMEM UPANet80 V2 maskAP 42.42 # 1
boxAP 42.96 # 1
FLOPs (G) 100.85 # 3
Size (M) 40.32 # 1
avgAP (mask AP + box AP) 42.69 # 1
Frame (fps) 60.93 # 3

Methods