Search Results for author:
Found 20 papers, 7 papers with code
Spurious Rewards: Rethinking Training Signals in RLVR
We show that reinforcement learning with verifiable rewards (RLVR) can elicit strong mathematical reasoning in certain models even with spurious rewards that have little, no, or even negative correlation with the correct answer.
Policy-Based Trajectory Clustering in Offline Reinforcement Learning
We introduce a novel task of clustering trajectories from offline reinforcement learning (RL) datasets, where each cluster center represents the policy that generated its trajectories.
Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models
Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities.
Highlighting What Matters: Promptable Embeddings for Attribute-Focused Image Retrieval
To evaluate current retrievers on handling attribute-focused queries, we build COCO-Facet, a COCO-based benchmark with 9, 112 queries about diverse attributes of interest.
Reinforcement Learning for Reasoning in Large Language Models with One Training Example
We also show the critical role of promoting exploration (e. g., by adding entropy loss with an appropriate coefficient) in 1-shot RLVR training.
LoRe: Personalizing LLMs via Low-Rank Reward Modeling
Personalizing large language models (LLMs) to accommodate diverse user preferences is essential for enhancing alignment and user satisfaction.
Is Your World Simulator a Good Story Presenter? A Consecutive Events-Based Benchmark for Future Long Video Generation
However, they still struggle to coherently present multiple sequential events in the stories specified by the prompts, which is foreseeable an essential capability for future long video generation scenarios.
Hybrid Preference Optimization for Alignment: Provably Faster Convergence Rates by Combining Offline Preferences with Online Exploration
Our results yield improved sample efficiency of hybrid RLHF over pure offline and online exploration.
On Erroneous Agreements of CLIP Image Embeddings
Recent research suggests that the failures of Vision-Language Models (VLMs) at visual reasoning often stem from erroneous agreements -- when semantically distinct images are ambiguously encoded by the CLIP image encoder into embeddings with high cosine similarity.
Transformers are Efficient Compilers, Provably
Transformer-based large language models (LLMs) have demonstrated surprisingly robust performance across a wide range of language-related tasks, including programming language understanding and generation.
Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
Reinforcement learning with human feedback (RLHF), as a widely adopted approach in current large language model pipelines, is \textit{bottlenecked by the size of human preference data}.
CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning
Three main data selection approaches are: (1) leveraging external non-CLIP models to aid data selection, (2) training new CLIP-style embedding models that are more effective at selecting high-quality data than the original OpenAI CLIP model, and (3) designing better metrics or strategies universally applicable to any CLIP embedding without requiring specific model properties (e. g., CLIPScore is one popular metric).
Offline Multi-task Transfer RL with Representational Penalization
We study the problem of representation transfer in offline Reinforcement Learning (RL), where a learner has access to episodic data from a number of source tasks collected a priori, and aims to learn a shared representation to be used in finding a good policy for a target task.
Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning
In recent years, data selection has emerged as a core issue for large-scale visual-language model pretraining, especially on noisy web-curated datasets.
Free from Bellman Completeness: Trajectory Stitching via Model-based Return-conditioned Supervised Learning
Off-policy dynamic programming (DP) techniques such as $Q$-learning have proven to be important in sequential decision-making problems.
Robust Offline Reinforcement Learning -- Certify the Confidence Interval
Currently, reinforcement learning (RL), especially deep RL, has received more and more attention in the research area.
LabelBench: A Comprehensive Framework for Benchmarking Adaptive Label-Efficient Learning
Labeled data are critical to modern machine learning applications, but obtaining labels can be expensive.
A Benchmark for Low-Switching-Cost Reinforcement Learning
A ubiquitous requirement in many practical reinforcement learning (RL) applications, including medical treatment, recommendation system, education and robotics, is that the deployed policy that actually interacts with the environment cannot change frequently.
Deep Q-Learning with Low Switching Cost
We initiate the study on deep reinforcement learning problems that require low switching cost, i. e., small number of policy switches during training.
Hypothesis Transfer Learning via Transformation Functions
We consider the Hypothesis Transfer Learning (HTL) problem where one incorporates a hypothesis trained on the source domain into the learning procedure of the target domain.
