TransBoost: Improving the Best ImageNet Performance using Deep Transduction

26 May 2022  ยท  Omer Belhasin, Guy Bar-Shalom, Ran El-Yaniv ยท

This paper deals with deep transductive learning, and proposes TransBoost as a procedure for fine-tuning any deep neural model to improve its performance on any (unlabeled) test set provided at training time. TransBoost is inspired by a large margin principle and is efficient and simple to use. Our method significantly improves the ImageNet classification performance on a wide range of architectures, such as ResNets, MobileNetV3-L, EfficientNetB0, ViT-S, and ConvNext-T, leading to state-of-the-art transductive performance. Additionally we show that TransBoost is effective on a wide variety of image classification datasets. The implementation of TransBoost is provided at: https://github.com/omerb01/TransBoost .

PDF Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Image Classification CIFAR-10 TransBoost-ResNet50 Percentage correct 97.61 # 69
Image Classification DTD TransBoost-ResNet50 Accuracy 76.49 # 8
Image Classification FGVC Aircraft TransBoost-ResNet50 Accuracy 83.80% # 1
Image Classification Flowers-102 TransBoost-ResNet50 Accuracy 97.85% # 33
Image Classification Food-101 TransBoost-ResNet50 Accuracy (%) 84.30 # 4
Image Classification ImageNet TransBoost-Swin-T Top 1 Accuracy 82.16% # 522
Number of params 71.71M # 790
Image Classification ImageNet TransBoost-ViT-S Top 1 Accuracy 83.67% # 374
Number of params 22.05M # 563
Image Classification ImageNet TransBoost-ConvNext-T Top 1 Accuracy 82.46% # 490
Number of params 28.59M # 640
Image Classification ImageNet TransBoost-ResNet152 Top 1 Accuracy 80.64% # 633
Number of params 60.19M # 767
Image Classification ImageNet TransBoost-MobileNetV3-L Top 1 Accuracy 76.81% # 827
Number of params 5.48M # 422
Image Classification ImageNet TransBoost-ResNet34 Top 1 Accuracy 76.70% # 832
Number of params 21.8M # 553
Image Classification ImageNet TransBoost-ResNet18 Top 1 Accuracy 73.36% # 914
Number of params 11.69M # 491
Image Classification ImageNet TransBoost-EfficientNetB0 Top 1 Accuracy 78.60% # 753
Number of params 5.29M # 414
Image Classification ImageNet TransBoost-ResNet101 Top 1 Accuracy 79.86% # 674
Number of params 44.55M # 701
Image Classification ImageNet TransBoost-ResNet50-StrikesBack Top 1 Accuracy 81.15% # 604
Number of params 25.56M # 597
Image Classification ImageNet TransBoost-ResNet50 Top 1 Accuracy 79.03% # 724
Image Classification Stanford Cars TransBoost-ResNet50 Accuracy 90.80% # 12
Image Classification SUN397 TransBoost-ResNet50 Accuracy 95.94% # 1

Methods


No methods listed for this paper. Add relevant methods here