Satellite Image Time Series Classification with Pixel-Set Encoders and Temporal Self-Attention

Satellite image time series, bolstered by their growing availability, are at the forefront of an extensive effort towards automated Earth monitoring by international institutions. In particular, large-scale control of agricultural parcels is an issue of major political and economic importance. In this regard, hybrid convolutional-recurrent neural architectures have shown promising results for the automated classification of satellite image time series.We propose an alternative approach in which the convolutional layers are advantageously replaced with encoders operating on unordered sets of pixels to exploit the typically coarse resolution of publicly available satellite images. We also propose to extract temporal features using a bespoke neural architecture based on self-attention instead of recurrent networks. We demonstrate experimentally that our method not only outperforms previous state-of-the-art approaches in terms of precision, but also significantly decreases processing time and memory requirements. Lastly, we release a large open-access annotated dataset as a benchmark for future work on satellite image time series.

PDF Abstract CVPR 2020 PDF CVPR 2020 Abstract

Datasets


  Add Datasets introduced or used in this paper

Reproducibility Reports


Dec 6 2020
[Re] Satellite Image Time Series Classification with Pixel-Set Encoders and Temporal Self-Attention

During the study, we were not able to reproduce the method due to a conceptual misinterpretation of ours regarding the authorsʼ adaption of the Transformer [2]. However, the publicly available implementation helped us answering our questions and proved its validity during our experiments on different datasets. Additionally, we compared the papersʼ temporal attention encoder to our adaption of it, which we came across while we were trying to reimplement and grasp the authorsʼ ideas.

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Time Series Classification s2-agri PSE+TAE mIoU 50.9 # 2
oAcc 94.2 # 2

Methods


No methods listed for this paper. Add relevant methods here