Search Results for author:
Found 12 papers, 8 papers with code
DataComp-LM: In search of the next generation of training sets for language models
We introduce DataComp for Language Models (DCLM), a testbed for controlled dataset experiments with the goal of improving language models.
Linearizing Large Language Models
Linear transformers have emerged as a subquadratic-time alternative to softmax attention and have garnered significant interest due to their fixed-size recurrent state that lowers inference cost.
A Critical Evaluation of AI Feedback for Aligning Large Language Models
RLAIF first performs supervised fine-tuning (SFT) using demonstrations from a teacher model and then further fine-tunes the model with reinforcement learning (RL), using feedback from a critic model.
The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation
State-of-the-art language generation models can degenerate when applied to open-ended generation problems such as text completion, story generation, or dialog modeling.
Lexi: Self-Supervised Learning of the UI Language
Along with text, these resources include visual content such as UI screenshots and images of application icons referenced in the text.
BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
We present BlenderBot 3, a 175B parameter dialogue model capable of open-domain conversation with access to the internet and a long-term memory, and having been trained on a large number of user defined tasks.
Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedback
We then study various algorithms for improving from such feedback, including standard supervised learning, rejection sampling, model-guiding and reward-based learning, in order to make recommendations on which type of feedback and algorithms work best.
DIRECTOR: Generator-Classifiers For Supervised Language Modeling
Current language models achieve low perplexity but their resulting generations still suffer from toxic responses, repetitiveness and contradictions.
Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation
Current language generation models suffer from issues such as repetition, incoherence, and hallucinations.
Learning Lexical Subspaces in a Distributional Vector Space
In this paper, we propose LexSub, a novel approach towards unifying lexical and distributional semantics.
A Compositional Approach to Language Modeling
Traditional language models treat language as a finite state automaton on a probability space over words.
Contrastive Entropy: A new evaluation metric for unnormalized language models
In this paper, we address the last problem and propose a new discriminative entropy based intrinsic metric that works for both traditional word level models and unnormalized language models like sentence level models.
