Sparse annotation poses persistent challenges to training dense retrieval models; for example, it distorts the training signal when unlabeled relevant documents are used spuriously as negatives in contrastive learning.
Large pre-trained language models contain societal biases and carry along these biases to downstream tasks.
Epileptic seizures are commonly monitored through electroencephalogram (EEG) recordings due to their routine and low expense collection.
1 code implementation • 28 Jan 2022 • Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick Legresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zhang, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, Bryan Catanzaro
Next, we detail the training process, the design of our training corpus, and our data curation techniques, which we believe is a key ingredient to the success of the model.
Ranked #19 on Sentence Completion on HellaSwag
Contrastive learning has been the dominant approach to training dense retrieval models.
Anginal symptoms can connote increased cardiac risk and a need for change in cardiovascular management.
In this work we propose for the first time a transformer-based framework for unsupervised representation learning of multivariate time series.
Ranked #2 on Time Series Classification on Insectwingbeat
In this work, we investigate unsupervised representation learning on medical time series, which bears the promise of leveraging copious amounts of existing unlabeled data in order to eventually assist clinical decision making.