Search Results for author:
Found 5 papers, 2 papers with code
Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup
In BERT training, the backward computation is much more time-consuming than the forward computation, especially in the distributed training setting in which the backward computation time further includes the communication time for gradient synchronization.
CoRe: An Efficient Coarse-refined Training Framework for BERT
In the second phase, we transform the trained relaxed BERT model into the original BERT and further retrain the model.
SAS: Self-Augmentation Strategy for Language Model Pre-training
In this paper, we propose a self-augmentation strategy (SAS) where a single network is utilized for both regular pre-training and contextualized data augmentation for the training in later epochs.
Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives
Following SimCSE, contrastive learning based methods have achieved the state-of-the-art (SOTA) performance in learning sentence embeddings.
GUIM -- General User and Item Embedding with Mixture of Representation in E-commerce
Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites.
