1 code implementation • 17 Dec 2024 • Mingjia Shi, Yuhao Zhou, Ruiji Yu, Zekai Li, Zhiyuan Liang, Xuanlei Zhao, Xiaojiang Peng, Tanmay Rajpurohit, Shanmukha Ramakrishna Vedantam, Wangbo Zhao, Kai Wang, Yang You
Re-training the token-reduced model enhances the performance of Mamba, by effectively rebuilding the key knowledge.
1 code implementation • 22 Aug 2024 • Xuanlei Zhao, Xiaolong Jin, Kai Wang, Yang You
We present Pyramid Attention Broadcast (PAB), a real-time, high quality and training-free approach for DiT-based video generation.
2 code implementations • 15 Mar 2024 • Xuanlei Zhao, Shenggan Cheng, Chang Chen, Zangwei Zheng, Ziming Liu, Zheming Yang, Yang You
Scaling multi-dimensional transformers to long sequences is indispensable across various domains.
no code implementations • 19 Jan 2024 • Xuanlei Zhao, Shenggan Cheng, Guangyang Lu, Jiarui Fang, Haotian Zhou, Bin Jia, Ziming Liu, Yang You
The experiments demonstrate that AutoChunk can reduce over 80\% of activation memory while maintaining speed loss within 10%, extend max sequence length by 3. 2x to 11. 7x, and outperform state-of-the-art methods by a large margin.
1 code implementation • 2 Mar 2022 • Shenggan Cheng, Xuanlei Zhao, Guangyang Lu, Jiarui Fang, Zhongming Yu, Tian Zheng, Ruidong Wu, Xiwen Zhang, Jian Peng, Yang You
In this work, we present FastFold, an efficient implementation of AlphaFold for both training and inference.