Search Results for author:
Found 20 papers, 14 papers with code
Multi-modal Time Series Analysis: A Tutorial and Survey
Multi-modal time series analysis has recently emerged as a prominent research area in data mining, driven by the increasing availability of diverse data modalities, such as text, images, and structured tabular data from real-world sources.
MiniMax-01: Scaling Foundation Models with Lightning Attention
This approach enables us to conduct efficient training and inference on models with hundreds of billions of parameters across contexts spanning millions of tokens.
Scaling Laws for Linear Complexity Language Models
In this study, we present the scaling laws for linear complexity language models to establish a foundation for their scalability.
You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet
Linear attention mechanisms have gained prominence in causal language models due to their linear computational complexity and enhanced speed.
Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective
Finally, the memory state is projected back to a low-dimensional space in the Shrink stage.
Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention
This eliminates the need for cumsum in the linear attention calculation.
TAVGBench: Benchmarking Text to Audible-Video Generation
To support research in this field, we have developed a comprehensive Text to Audible-Video Generation Benchmark (TAVGBench), which contains over 1. 7 million clips with a total duration of 11. 8 thousand hours.
HGRN2: Gated Linear RNNs with State Expansion
Hierarchically gated linear RNN (HGRN, \citealt{HGRN}) has demonstrated competitive training speed and performance in language modeling while offering efficient inference.
Linear Attention Sequence Parallelism
However, for linear sequence modeling methods like linear attention, existing SP approaches do not take advantage of their right-product-first feature, resulting in sub-optimal communication efficiency and usability.
CO2: Efficient Distributed Training with Full Communication-Computation Overlap
CO2 is able to attain a high scalability even on extensive multi-node clusters constrained by very limited communication bandwidth.
Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
With its ability to process tokens in linear computational complexities, linear attention, in theory, can handle sequences of unlimited length without sacrificing speed, i. e., maintaining a constant training speed for various sequence lengths with a fixed memory consumption.
TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
TransNormerLLM evolves from the previous linear attention architecture TransNormer by making advanced modifications that include positional embedding, linear attention acceleration, gating mechanisms, tensor normalization, and inference acceleration and stabilization.
Fine-grained Audible Video Description
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD).
Audio-Visual Segmentation with Semantics
To deal with these problems, we propose a new baseline method that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process.
Linear Video Transformer with Feature Fixation
Therefore, we propose a feature fixation module to reweight the feature importance of the query and key before computing linear attention.
AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification
Fine-tuning is widely applied in image classification tasks as a transfer learning approach.
Feature Selection on Thermal-stress Dataset
These findings are likely to improve the accuracy of current stress recognition systems.
Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Space
Varying the proportions of male and female faces in the training data can have a substantial effect on behavior on the test data: we found that the seemingly obvious choice of 50:50 proportions was not the best for this dataset to reduce biased behavior on female faces, which was 71% unbiased as compared to our top unbiased rate of 84%.
Non-binary deep transfer learning for image classification
A model is either pre-trained or not pre-trained.
Ranked #2 on
Image Classification
on Caltech-256
(using extra training data)
Pairwise-GAN: Pose-based View Synthesis through Pair-Wise Training
This inspired our research which explores the performance of two models from pixel transformation in frontal facial synthesis, Pix2Pix and CycleGAN.
