Search Results for author:
Found 20 papers, 7 papers with code
Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
We advocate for using embedding-based input in reward model research as an accelerated solution to those challenges.
Understanding Chain-of-Thought in LLMs through Information Theory
Large Language Models (LLMs) have shown impressive performance in complex reasoning tasks through Chain-of-Thought (CoT) reasoning, allowing models to break down problems into manageable sub-tasks.
Rethinking Bradley-Terry Models in Preference-Based Reward Modeling: Foundations, Theory, and Alternatives
The Bradley-Terry (BT) model is a common and successful practice in reward modeling for Large Language Model (LLM) alignment.
ACC-Debate: An Actor-Critic Approach to Multi-Agent Debate
Large language models (LLMs) have demonstrated a remarkable ability to serve as general-purpose tools for various language-based tasks.
Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimation
We introduce Adversarial Policy Optimization (AdvPO), a novel solution to the pervasive issue of reward over-optimization in Reinforcement Learning from Human Feedback (RLHF) for Large Language Models (LLMs).
Achievable Fairness on Your Data With Utility Guarantees
In machine learning fairness, training models that minimize disparity across different sensitive groups often leads to diminished accuracy, a phenomenon known as the fairness-accuracy trade-off.
Measuring and Reducing LLM Hallucination without Gold-Standard Answers
FEWL leverages the answers from off-the-shelf LLMs that serve as a proxy of gold-standard answers.
Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits
Off-Policy Evaluation (OPE) in contextual bandits is crucial for assessing new policies using existing data without costly experimentation.
Deep Concept Removal
We address the problem of concept removal in deep neural networks, aiming to learn representations that do not encode certain specified concepts (e. g., gender etc.)
Invariant Learning via Probability of Sufficient and Necessary Causes
To capture the information of sufficient and necessary causes, we employ a classical concept, the probability of sufficiency and necessary causes (PNS), which indicates the probability of whether one is the necessary and sufficient cause.
Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment
However, a major challenge faced by practitioners is the lack of clear guidance on evaluating whether LLM outputs align with social norms, values, and regulations.
Conformal Off-Policy Prediction in Contextual Bandits
Most off-policy evaluation methods for contextual bandits have focused on the expected outcome of a policy, which is estimated via methods that at best provide only asymptotic guarantees.
Regularized Training of Nearest Neighbor Language Models
In particular, we find that the added L2 regularization seems to improve the performance for high-frequency words without deteriorating the performance for low frequency ones.
Meta Learning for Causal Direction
Based on recent developments in meta learning as well as in causal inference, we introduce a novel generative model that allows distinguishing cause and effect in the small data setting.
Robust Pruning at Initialization
Overparameterized Neural Networks (NN) display state-of-the-art performance.
MetaFun: Meta-Learning with Iterative Functional Updates
We develop a functional encoder-decoder approach to supervised meta-learning, where labeled data is encoded into an infinite-dimensional functional representation rather than a finite-dimensional one.
Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings
Current meta-learning approaches focus on learning functional representations of relationships between variables, i. e. on estimating conditional expectations in regression.
Automated Model Selection with Bayesian Quadrature
We present a novel technique for tailoring Bayesian quadrature (BQ) to model selection.
Towards A Unified Analysis of Random Fourier Features
We study both the standard random Fourier features method for which we improve the existing bounds on the number of features required to guarantee the corresponding minimax risk convergence rate of kernel ridge regression, as well as a data-dependent modification which samples features proportional to \emph{ridge leverage scores} and further reduces the required number of features.
Spatial Mapping with Gaussian Processes and Nonstationary Fourier Features
The use of covariance kernels is ubiquitous in the field of spatial statistics.
