Search Results for author:
Found 11 papers, 4 papers with code
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees
Large language models (LLMs) are increasingly capable of completing knowledge intensive tasks by recalling information from a static pretraining corpus.
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
To address these limitations, we tackle the verifiability goal with a different philosophy: we trivialize the verification process by developing models that quote verbatim statements from trusted sources in pre-training data.
Dated Data: Tracing Knowledge Cutoffs in Large Language Models
Using this analysis, we find that effective cutoffs often differ from reported cutoffs.
StarCoder 2 and The Stack v2: The Next Generation
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #24 on
Code Generation
on MBPP
"According to ...": Prompting Language Models Improves Quoting from Pre-Training Data
Large Language Models (LLMs) may hallucinate and generate fake information, despite pre-training on factual data.
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15. 5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention.
Ranked #42 on
Code Generation
on MBPP
Data Portraits: Recording Foundation Model Training Data
Foundation models are trained on increasingly immense and opaque datasets.
Pretrained Models for Multilingual Federated Learning
Since the advent of Federated Learning (FL), research has applied these methods to natural language processing (NLP) tasks.
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction
Zero-shot cross-lingual information extraction (IE) describes the construction of an IE model for some target language, given existing annotations exclusively in some other language, typically English.
Selecting, Planning, and Rewriting: A Modular Approach for Data-to-Document Generation and Translation
In this paper, we report our system submissions to all 6 tracks of the WNGT 2019 shared task on Document-Level Generation and Translation.
Character Eyes: Seeing Language through Character-Level Taggers
Character-level models have been used extensively in recent years in NLP tasks as both supplements and replacements for closed-vocabulary token-level word representations.
