Search Results for author:
Found 11 papers, 7 papers with code
Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models
However, asynchronous training relies on an underexplored regime, online but off-policy RLHF: learning on samples from previous iterations of our model.
The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization
This work is the first to openly reproduce the Reinforcement Learning from Human Feedback (RLHF) scaling behaviors reported in OpenAI's seminal TL;DR summarization work.
Language Model Alignment with Elastic Reset
We periodically reset the online model to an exponentially moving average (EMA) of itself, then reset the EMA model to the initial model.
Learning Multi-Agent Communication with Contrastive Learning
By examining the relationship between messages sent and received, we propose to learn to communicate using contrastive learning to maximize the mutual information between messages of a given trajectory.
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into $L$ simplices of $V$ dimensions each using a softmax operation.
Pretraining Representations for Data-Efficient Reinforcement Learning
Data efficiency is a key challenge for deep reinforcement learning.
Ranked #3 on
Atari Games 100k
on Atari 100k
(using extra training data)
Pretraining Reward-Free Representations for Data-Efficient Reinforcement Learning
Data efficiency poses a major challenge for deep reinforcement learning.
Emergent Communication under Competition
First, we show that communication is proportional to cooperation, and it can occur for partially competitive scenarios using standard learning algorithms.
Selfish Emergent Communication
Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel.
Systematic Generalization: What Is Required and Can It Be Learned?
Numerous models for grounded language understanding have been recently proposed, including (i) generic models that can be easily adapted to any given task and (ii) intuitively appealing modular models that require background knowledge to be instantiated.
Commonsense mining as knowledge base completion? A study on the impact of novelty
Commonsense knowledge bases such as ConceptNet represent knowledge in the form of relational triples.
