Long-range modeling
48 papers with code • 2 benchmarks • 4 datasets
A new task for testing the long-sequence modeling capabilities and efficiency of language models.
Image credit: SCROLLS: Standardized CompaRison Over Long Language Sequences
Libraries
Use these libraries to find Long-range modeling models and implementationsMost implemented papers
Simplified State Space Layers for Sequence Modeling
Models using structured state space sequence (S4) layers have achieved state-of-the-art performance on long-range sequence modeling tasks.
T-former: An Efficient Transformer for Image Inpainting
And based on this attention, a network called $T$-former is designed for image inpainting.
MambaMorph: a Mamba-based Framework for Medical MR-CT Deformable Registration
Capturing voxel-wise spatial correspondence across distinct modalities is crucial for medical image analysis.
VM-UNet: Vision Mamba UNet for Medical Image Segmentation
To our best knowledge, this is the first medical image segmentation model constructed based on the pure SSM-based model.
V4D:4D Convolutional Neural Networks for Video-level Representation Learning
Most existing 3D CNNs for video representation learning are clip-based methods, and thus do not consider video-level temporal evolution of spatio-temporal features.
DSANet: Dynamic Segment Aggregation Network for Video-Level Representation Learning
Long-range and short-range temporal modeling are two complementary and crucial aspects of video recognition.
Image Super-Resolution With Non-Local Sparse Attention
NLSA is designed to retain long-range modeling capability from NL operation while enjoying robustness and high-efficiency of sparse representation.
Sparse Factorization of Large Square Matrices
The sparse factorization method is tested for a variety of synthetic and real-world square matrices.
LongT5: Efficient Text-To-Text Transformer for Long Sequences
Recent work has shown that either (1) increasing the input length or (2) increasing model size can improve the performance of Transformer-based neural models.
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks
Recurrent Neural Networks, Transformers, and Convolutional Neural Networks are three major techniques for learning from sequential data.