Ev-TTA: Test-Time Adaptation for Event-Based Object Recognition

CVPR 2022  ·  Junho Kim, Inwoo Hwang, Young Min Kim ·

We introduce Ev-TTA, a simple, effective test-time adaptation algorithm for event-based object recognition. While event cameras are proposed to provide measurements of scenes with fast motions or drastic illumination changes, many existing event-based recognition algorithms suffer from performance deterioration under extreme conditions due to significant domain shifts. Ev-TTA mitigates the severe domain gaps by fine-tuning the pre-trained classifiers during the test phase using loss functions inspired by the spatio-temporal characteristics of events. Since the event data is a temporal stream of measurements, our loss function enforces similar predictions for adjacent events to quickly adapt to the changed environment online. Also, we utilize the spatial correlations between two polarities of events to handle noise under extreme illumination, where different polarities of events exhibit distinctive noise distributions. Ev-TTA demonstrates a large amount of performance gain on a wide range of event-based object recognition tasks without extensive additional training. Our formulation can be successfully applied regardless of input representations and further extended into regression tasks. We expect Ev-TTA to provide the key technique to deploy event-based vision algorithms in challenging real-world applications where significant domain shift is inevitable.

PDF Abstract CVPR 2022 PDF CVPR 2022 Abstract


Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Robust classification N-ImageNet Sorted Time Surface + Ev-TTA Accuracy (%) 44.81 # 1
Robust classification N-ImageNet Binary Event Image + Ev-TTA Accuracy (%) 43.72 # 4
Robust classification N-ImageNet Event Histogram + Ev-TTA Accuracy (%) 44.20 # 3
Robust classification N-ImageNet Timestamp Image + Ev-TTA Accuracy (%) 43.47 # 5
Robust classification N-ImageNet Time Surface + Ev-TTA Accuracy (%) 42.91 # 6
Robust classification N-ImageNet DiST + Ev-TTA Accuracy (%) 44.80 # 2


No methods listed for this paper. Add relevant methods here