Multiscale Deep Equilibrium Models

15 Jun 2020Shaojie BaiVladlen KoltunJ. Zico Kolter

We propose a new class of implicit networks, the multiscale deep equilibrium model (MDEQ), suited to large-scale and highly hierarchical pattern recognition domains. An MDEQ directly solves for and backpropagates through the equilibrium points of multiple feature resolutions simultaneously, using implicit differentiation to avoid storing intermediate states (and thus requiring only O(1) memory consumption)... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK LEADERBOARD
Semantic Segmentation Cityscapes val Multiscale DEQ (MDEQ-XL) mIoU 80.3% # 6
Semantic Segmentation Cityscapes val Multiscale DEQ (MDEQ-large) mIoU 77.8% # 12
Image Classification ImageNet Multiscale DEQ (MDEQ-XL) Top 1 Accuracy 79.2% # 71