Crucially, we find that $k$NN-LMs are more susceptible to leaking private information from their private datastore than parametric models.
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option.
Fine-tuning a language model on a new domain is standard practice for domain adaptation.
For the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences.
Open-domain question answering has exploded in popularity recently due to the success of dense retrieval models, which have surpassed sparse models using only a few supervised training examples.
Ranked #3 on Passage Retrieval on EntityQuestions
Petroni et al. (2019) demonstrated that it is possible to retrieve world facts from a pre-trained language model by expressing them as cloze-style prompts and interpret the model's prediction accuracy as a lower bound on the amount of factual information it encodes.
Our approach essentially builds on two independent encoders and merely uses the entity model to construct the input for the relation model.
Ranked #1 on Named Entity Recognition (NER) on ACE 2005
Recent research proposes syntax-based approaches to address the problem of generating programs from natural language specifications.
In particular, given a target model, our framework includes multiple models (constructed from the target model) to form a model family.