Search Results for author: Nafis Sadeq

Found 5 papers, 2 papers with code

Unsupervised Improvement of Factual Knowledge in Language Models

1 code implementation4 Apr 2023 Nafis Sadeq, Byungkyu Kang, Prarit Lamba, Julian McAuley

In this work, we propose an approach for influencing MLM pretraining in a way that can improve language model performance on a variety of knowledge-intensive tasks.

Language Modelling Masked Language Modeling +3

InforMask: Unsupervised Informative Masking for Language Model Pretraining

1 code implementation21 Oct 2022 Nafis Sadeq, Canwen Xu, Julian McAuley

In this paper, we propose InforMask, a new unsupervised masking strategy for training masked language models.

Language Modelling Masked Language Modeling +2

Cannot find the paper you are looking for? You can Submit a new open access paper.