Search Results for author:
Found 5 papers, 4 papers with code
Improving Transformer Optimization Through Better Initialization
As Transformer models are becoming larger and more expensive to train, recent research has focused on understanding and improving optimization in these models.
Improving Transformer Optimization Through Better Initialization
As Transformer models are becoming larger and more expensive to train, recent research has focused on understanding and improving optimization in these models.
Self-supervised Representation Learning From Random Data Projectors
Self-supervised representation learning~(SSRL) has advanced considerably by exploiting the transformation invariance assumption under artificially designed data augmentations.
DuETT: Dual Event Time Transformer for Electronic Health Records
Electronic health records (EHRs) recorded in hospital settings typically contain a wide range of numeric time series data that is characterized by high sparsity and irregular observations.
Improving Non-Autoregressive Translation Models Without Distillation
Empirically, we show that CMLMC achieves state-of-the-art NAR performance when trained on raw data without distillation and approaches AR performance on multiple datasets.
