Search Results for author:
Found 4 papers, 1 papers with code
Transforming and Combining Rewards for Aligning Large Language Models
A common approach for aligning language models to human preferences is to first learn a reward model from preference data, and then use this reward model to update the language model.
Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hacking
However, even pretrain reward ensembles do not eliminate reward hacking: we show several qualitative reward hacking phenomena that are not mitigated by ensembling because all reward models in the ensemble exhibit similar error patterns.
Detecting Underspecification with Local Ensembles
We present local ensembles, a method for detecting underspecification -- when many possible predictors are consistent with the training data and model class -- at test time in a pre-trained model.
BriarPatches: Pixel-Space Interventions for Inducing Demographic Parity
We introduce the BriarPatch, a pixel-space intervention that obscures sensitive attributes from representations encoded in pre-trained classifiers.
