Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks

6 Jan 2022  ·  Lei Cheng, Ruslan Khalitov, Tong Yu, Zhirong Yang ·

Classification of long sequential data is an important Machine Learning task and appears in many application scenarios. Recurrent Neural Networks, Transformers, and Convolutional Neural Networks are three major techniques for learning from sequential data. Among these methods, Temporal Convolutional Networks (TCNs) which are scalable to very long sequences have achieved remarkable progress in time series regression. However, the performance of TCNs for sequence classification is not satisfactory because they use a skewed connection protocol and output classes at the last position. Such asymmetry restricts their performance for classification which depends on the whole sequence. In this work, we propose a symmetric multi-scale architecture called Circular Dilated Convolutional Neural Network (CDIL-CNN), where every position has an equal chance to receive information from other positions at the previous layers. Our model gives classification logits in all positions, and we can apply a simple ensemble learning to achieve a better decision. We have tested CDIL-CNN on various long sequential datasets. The experimental results show that our method has superior performance over many state-of-the-art approaches.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Long-range modeling LRA CDIL ListOps 60.6 # 6
Text 87.61 # 9
Retrieval 84.27 # 15
Image 64.49 # 13
Pathfinder 91.00 # 11
Avg 77.59 # 13
Audio Classification UCR Time Series Classification Archive CDIL FruitFlies 97.09 # 1
RightWhaleCalls 91.99 # 1
MosquitoSound 91.54 # 1