Search Results for author:
Found 22 papers, 12 papers with code
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
What Language Model to Train if You Have One Million GPU Hours?
All our models and code are open-sourced at https://huggingface. co/bigscience .
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
Here we explore the theoretical connections between the two paradigms, and show that methods such as KL-control developed for RM can also be construed as belonging to DM.
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models.
Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offs
We show that we can sample from such EBMs with arbitrary precision at the cost of sampling efficiency.
Controlling Conditional Language Models without Catastrophic Forgetting
Machine learning is shifting towards general-purpose pretrained generative models, trained in a self-supervised manner on large amounts of data, which can then be applied to solve a large number of tasks.
Unsupervised and Distributional Detection of Machine-Generated Text
We propose a method to detect those machine-generated documents leveraging repeated higher-order n-grams, which we show over-appear in machine-generated text as compared to human ones.
On Reward Maximization and Distribution Matching for Fine-Tuning Language Models
The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a "training from scratch" to a "fine-tuning'' paradigm.
Energy-Based Models for Code Generation under Compilability Constraints
Neural language models can be successfully trained on source code, leading to applications such as code completion.
Symbol-Shift Equivariant Neural Networks
Neural networks have been shown to have poor compositionality abilities: while they can produce sophisticated output given sufficient data, they perform patchy generalization and fail to generalize to new symbols (e. g. switching a name in a sentence by a less frequent one or one not seen yet).
A Distributional Approach to Controlled Text Generation
From that optimal representation we then train a target controlled Autoregressive LM through an adaptive distributional variant of Policy Gradient.
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages
Research in NLP lacks geographic diversity, and the question of how NLP can be scaled to low-resourced languages has not yet been adequately solved.
Self-Supervised and Controlled Multi-Document Opinion Summarization
We address the problem of unsupervised abstractive summarization of collections of user generated reviews with self-supervision and control.
Unsupervised Aspect-Based Multi-Document Abstractive Summarization
User-generated reviews of products or services provide valuable information to customers.
To Annotate or Not? Predicting Performance Drop under Domain Shift
In this paper, we study the problem of predicting the performance drop of modern NLP models under domain-shift, in the absence of any target domain labels.
Learning to Generate Wikipedia Summaries for Underserved Languages from Wikidata
While Wikipedia exists in 287 languages, its content is unevenly distributed among them.
Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types
We present a neural model for question generation from knowledge base triples in a "Zero-Shot" setup, that is generating questions for triples containing predicates, subject types or object types that were not seen at training time.
Ranked #14 on
Zero-shot Text Search
on BEIR
Unsupervised Open Relation Extraction
We explore methods to extract relations between named entities from free text in an unsupervised setting.
High Recall Open IE for Relation Discovery
Relation Discovery discovers predicates (relation types) from a text corpus relying on the co-occurrence of two named entities in the same sentence.
Neural Wikipedian: Generating Textual Summaries from Knowledge Base Triples
We explore the problem of generating natural language summaries for Semantic Web data.
