Search Results for author:
Found 4 papers, 1 papers with code
BurstAttention: An Efficient Distributed Attention Framework for Extremely Long Sequences
Effective attention modules have played a crucial role in the success of Transformer-based large language models (LLMs), but the quadratic time and memory complexities of these attention modules also pose a challenge when processing long sequences.
Digital twin-assisted three-dimensional electrical capacitance tomography for multiphase flow imaging
Three-dimensional electrical capacitance tomography (3D-ECT) has shown promise for visualizing industrial multiphase flows.
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup
In BERT training, the backward computation is much more time-consuming than the forward computation, especially in the distributed training setting in which the backward computation time further includes the communication time for gradient synchronization.
CoRe: An Efficient Coarse-refined Training Framework for BERT
In the second phase, we transform the trained relaxed BERT model into the original BERT and further retrain the model.
