Seq2Tens: An Efficient Representation of Sequences by Low-Rank Tensor Projections

ICLR 2021  ·  Csaba Toth, Patric Bonnier, Harald Oberhauser ·

Sequential data such as time series, video, or text can be challenging to analyse as the ordered structure gives rise to complex dependencies. At the heart of this is non-commutativity, in the sense that reordering the elements of a sequence can completely change its meaning... We use a classical mathematical object -- the tensor algebra -- to capture such dependencies. To address the innate computational complexity of high degree tensors, we use compositions of low-rank tensor projections. This yields modular and scalable building blocks for neural networks that give state-of-the-art performance on standard benchmarks such as multivariate time series classification and generative models for video. read more

PDF Abstract ICLR 2021 PDF ICLR 2021 Abstract

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Time Series Classification ArabicDigits FCN-SNLST Accuracy 0.993 # 2
Time Series Classification ArabicDigits SNLST Accuracy 0.968 # 9
Time Series Classification AUSLAN FCN-SNLST Accuracy 0.993 # 1
Time Series Classification AUSLAN SNLST Accuracy 0.969 # 4
Time Series Classification CharacterTrajectories FCN-SNLST Accuracy 0.994 # 2
Time Series Classification CharacterTrajectories SNLST Accuracy 0.957 # 5
Time Series Classification CMUsubject16 SNLST Accuracy 1 # 1
Time Series Classification CMUsubject16 FCN-SNLST Accuracy 1 # 1
Time Series Classification DigitShapes SNLST Accuracy 1 # 1
Time Series Classification DigitShapes FCN-SNLST Accuracy 1 # 1
Time Series Classification ECG SNLST Accuracy 0.842 # 4
Time Series Classification ECG FCN-SNLST Accuracy 0.860 # 1
Imputation HMNIST GP-VAE (B-NLST) NLL 0.251 # 1
MSE 0.092 # 1
AUROC 0.962 # 1
Time Series Classification JapaneseVowels SNLST Accuracy 0.979 # 9
Time Series Classification JapaneseVowels FCN-SNLST Accuracy 0.980 # 8
Time Series Classification KickvsPunch SNLST Accuracy 1 # 1
Time Series Classification KickvsPunch FCN-SNLST Accuracy 1 # 1
Time Series Classification Libras SNLST Accuracy 0.773 # 7
Time Series Classification Libras FCN-SNLST Accuracy 0.957 # 2
Time Series Classification NetFlow FCN-SNLST Accuracy 0.960 # 1
Time Series Classification NetFlow SNLST Accuracy 0.793 # 9
Time Series Classification PEMS SNLST Accuracy 0.747 # 7
Time Series Classification PEMS FCN-SNLST Accuracy 0.857 # 1
Time Series Classification PenDigits SNLST Accuracy 0.954 # 2
Time Series Classification PenDigits FCN-SNLST Accuracy 0.953 # 3
Imputation PhysioNet Challenge 2012 GP-VAE (B-NLST) AUROC 0.743 # 1
Time Series Classification SHAPES FCN-SNLST Accuracy 1 # 1
Time Series Classification SHAPES SNLST Accuracy 1 # 1
Imputation Sprites GP-VAE (B-NLST) MSE 0.002 # 1
Time Series Classification UWave FCN-SNLST Accuracy 0.969 # 3
Time Series Classification UWave SNLST Accuracy 0.938 # 7
Time Series Classification Wafer SNLST Accuracy 0.981 # 6
Time Series Classification Wafer FCN-SNLST Accuracy 0.989 # 3
Time Series Classification WalkvsRun FCN-SNLST Accuracy 1 # 1
Time Series Classification WalkvsRun SNLST Accuracy 1 # 1

Methods