Search Results for author:
Found 65 papers, 36 papers with code
Understanding Impact of Human Feedback via Influence Functions
By quantifying the impact of human feedback on reward models, we believe that influence functions can enhance feedback interpretability and contribute to scalable oversight in RLHF, helping labelers provide more accurate and consistent feedback.
Improbable Bigrams Expose Vulnerabilities of Incomplete Tokens in Byte-Level Tokenizers
In this work, we investigate incomplete tokens, i. e., undecodable tokens with stray bytes resulting from byte-level byte-pair encoding (BPE) tokenization.
MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control
In this work, we introduce MobileSafetyBench, a benchmark designed to evaluate the safety of device-control agents within a realistic mobile environment based on Android emulators.
When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs
Recent advancements in Large Language Models (LLMs) have established them as agentic systems capable of planning and interacting with various tools.
Latent Action Pretraining from Videos
We introduce Latent Action Pretraining for general Action models (LAPA), an unsupervised method for pretraining Vision-Language-Action (VLA) models without ground-truth robot action labels.
Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models
Fine-tuning text-to-image diffusion models with human feedback is an effective method for aligning model behavior with human intentions.
DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing
Through extensive experiments, we show that our method achieves stronger protection and improved mask robustness with lower computational costs compared to the strongest baseline.
Margin Matching Preference Optimization: Enhanced Model Alignment with Granular Feedback
Large language models (LLMs) fine-tuned with alignment techniques, such as reinforcement learning from human feedback, have been instrumental in developing some of the most capable AI systems to date.
By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting
We design a visual prompt that directs MLLMs to utilize visualized sensor data alongside the target sensory task descriptions.
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
Human feedback plays a critical role in learning and refining reward models for text-to-image generation, but the optimal form the feedback should take for learning an accurate reward function has not been conclusively established.
Aligning Large Language Models with Self-generated Preference Data
To tackle this problem, we propose a new framework that boosts the alignment of LLMs through Self-generated Preference data (Selfie) using only a very small amount of human-annotated preference data.
Benchmarking Mobile Device Control Agents across Diverse Configurations
In this work, we introduce B-MoCA: a novel benchmark with interactive environments for evaluating and developing mobile device control agents.
Identity Decoupling for Multi-Subject Personalization of Text-to-Image Models
Experimental results show that our MuDI can produce high-quality personalized images without identity mixing, even for highly similar subjects as shown in Figure 1.
Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models
To investigate this issue in depth, we introduce the Text-Image Alignment Assessment (TIA2) benchmark, which comprises a diverse collection of text prompts, images, and human annotations.
Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences
Customizing robotic behaviors to be aligned with diverse human preferences is an underexplored challenge in the field of embodied AI.
InstructBooth: Instruction-following Personalized Text-to-Image Generation
In this work, we introduce InstructBooth, a novel method designed to enhance image-text alignment in personalized text-to-image models without sacrificing the personalization ability.
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models
We focus on diffusion models, defining the fine-tuning task as an RL problem, and updating the pre-trained text-to-image diffusion models using policy gradient to maximize the feedback-trained reward.
Preference Transformer: Modeling Human Preferences using Transformers for RL
In this paper, we present Preference Transformer, a neural architecture that models human preferences using transformers.
Aligning Text-to-Image Models using Human Feedback
Our results demonstrate the potential for learning from human feedback to significantly improve text-to-image models.
Controllability-Aware Unsupervised Skill Discovery
One of the key capabilities of intelligent agents is the ability to discover useful skills without external supervision.
Multi-View Masked World Models for Visual Robotic Manipulation
In this paper, we investigate how to learn good representations with multi-view data and utilize them for visual robotic manipulation.
Instruction-Following Agents with Multimodal Transformer
Our \ours method consists of a multimodal transformer that encodes visual observations and language instructions, and a transformer-based policy that predicts actions based on encoded representations.
HARP: Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics.
Masked World Models for Visual Control
Yet the current approaches typically train a single model end-to-end for learning both visual representations and dynamics, making it difficult to accurately model the interaction between robots and small objects.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
+1
Reward Uncertainty for Exploration in Preference-based Reinforcement Learning
Our intuition is that disagreement in learned reward model reflects uncertainty in tailored human feedback and could be useful for exploration.
Reinforcement Learning with Action-Free Pre-Training from Videos
Our framework consists of two phases: we pre-train an action-free latent video prediction model, and then utilize the pre-trained representations for efficiently learning action-conditional world models on unseen environments.
SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning
In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor.
Improving Transferability of Representations via Augmentation-Aware Self-Supervision
Recent unsupervised representation learning methods have shown to be effective in a range of vision tasks by learning representations invariant to data augmentations such as random cropping and color jittering.
B-Pref: Benchmarking Preference-Based Reinforcement Learning
However, it is difficult to quantify the progress in preference-based RL due to the lack of a commonly adopted benchmark.
URLB: Unsupervised Reinforcement Learning Benchmark
Deep Reinforcement Learning (RL) has emerged as a powerful paradigm to solve a range of complex yet specific control tasks.
Towards More Generalizable One-shot Visual Imitation Learning
We then study the multi-task setting, where multi-task training is followed by (i) one-shot imitation on variations within the training tasks, (ii) one-shot imitation on new tasks, and (iii) fine-tuning on new tasks.
Autoregressive Latent Video Prediction with High-Fidelity Image Generator
Video prediction is an important yet challenging problem; burdened with the tasks of generating future frames and learning environment dynamics.
Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback
A promising approach to solving challenging long-horizon tasks has been to extract behavior priors (skills) by fitting generative models to large offline datasets of demonstrations.
Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble
Recent advance in deep offline reinforcement learning (RL) has made it possible to train strong robotic agents from offline datasets.
Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments
To showcase the benefits, we interfaced SCENIC to an existing RTS environment Google Research Football(GRF) simulator and introduced a benchmark consisting of 32 realistic scenarios, encoded in SCENIC, to train RL agents and testing their generalization capabilities.
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
We also show that our method is able to utilize real-time human feedback to effectively prevent reward exploitation and learn new behaviors that are difficult to specify with standard reward functions.
Decision Transformer: Reinforcement Learning via Sequence Modeling
In particular, we present Decision Transformer, an architecture that casts the problem of RL as conditional sequence modeling.
Ranked #3 on
Offline RL
on D4RL
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings
Recent advances in off-policy deep reinforcement learning (RL) have led to impressive success in complex tasks from visual observations.
Ranked #33 on
Atari Games
on Atari 2600 Amidar
State Entropy Maximization with Random Encoders for Efficient Exploration
Recent exploration methods have proven to be a recipe for improving sample-efficiency in deep reinforcement learning (RL).
R-LAtte: Attention Module for Visual Control via Reinforcement Learning
Attention mechanisms are generic inductive biases that have played a critical role in improving the state-of-the-art in supervised learning, unsupervised pre-training and generative modeling for multiple domains including vision, language and speech.
Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets
As it turns out, fine-tuning offline RL agents is a non-trivial challenge, due to distribution shift – the agent encounters out-of-distribution samples during online interaction, which may cause bootstrapping error in Q-learning and instability during fine-tuning.
Compute- and Memory-Efficient Reinforcement Learning with Latent Experience Replay
In this paper, we present Latent Vector Experience Replay (LeVER), a simple modification of existing off-policy RL methods, to address these computational and memory requirements without sacrificing the performance of RL agents.
Weighted Bellman Backups for Improved Signal-to-Noise in Q-Updates
Furthermore, since our weighted Bellman backups rely on maintaining an ensemble, we investigate how weighted Bellman backups interact with other benefits previously derived from ensembles: (a) Bootstrap; (b) UCB Exploration.
MASKER: Masked Keyword Regularization for Reliable Text Classification
We claim that one central obstacle to the reliability is the over-reliance of the model on a limited number of keywords, instead of looking at the whole context.
Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment
In this work, we show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task during task learning with an object-centric relational RL agent.
Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning
Model-based reinforcement learning (RL) has shown great potential in various control tasks in terms of both sample-efficiency and final performance.
Decoupling Representation Learning from Reinforcement Learning
In an effort to overcome limitations of reward-driven feature learning in deep reinforcement learning (RL) from images, we propose decoupling representation learning from policy learning.
Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning
Despite the significant progress of deep reinforcement learning (RL) in solving sequential decision making problems, RL agents often overfit to training environments and struggle to adapt to new, unseen environments.
Learning to Sample with Local and Global Contexts in Experience Replay Buffer
Experience replay, which enables the agents to remember and reuse experience from the past, has played a significant role in the success of off-policy reinforcement learning (RL).
SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning
Off-policy deep reinforcement learning (RL) has been successful in a range of challenging domains.
Context-aware Dynamics Model for Generalization in Model-Based Reinforcement Learning
Model-based reinforcement learning (RL) enjoys several benefits, such as data-efficiency and planning, by learning a model of the environment's dynamics.
Model-based Reinforcement Learning
reinforcement-learning
+1
Reinforcement Learning with Augmented Data
To this end, we present Reinforcement Learning with Augmented Data (RAD), a simple plug-and-play module that can enhance most RL algorithms.
Regularizing Class-wise Predictions via Self-knowledge Distillation
Deep neural networks with millions of parameters may suffer from poor generalization due to overfitting.
Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
Deep reinforcement learning (RL) agents often fail to generalize to unseen environments (yet semantically similar to trained agents), particularly when they are trained on high-dimensional state spaces, such as images.
Robust Determinantal Generative Classifier for Noisy Labels and Adversarial Attacks
For instance, on CIFAR-10 dataset containing 45% noisy training labels, we improve the test accuracy of a deep model optimized by the state-of-the-art noise-handling training method from33. 34% to 43. 02%.
Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild
Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: the performance on previous tasks drastically degrades when learning a new task.
Robust Inference via Generative Classifiers for Handling Noisy Labels
Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets.
Using Pre-Training Can Improve Model Robustness and Uncertainty
He et al. (2018) have called into question the utility of pre-training by showing that training from scratch can often yield similar performance to pre-training.
Learning to Specialize with Knowledge Distillation for Visual Question Answering
The proposed framework is model-agnostic and applicable to any tasks other than VQA, e. g., image classification with a large number of labels but few per-class examples, which is known to be difficult under existing MCL schemes.
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Detecting test samples drawn sufficiently far away from the training distribution statistically or adversarially is a fundamental requirement for deploying a good classifier in many real-world machine learning applications.
Ranked #2 on
Out-of-Distribution Detection
on MS-1M vs. IJB-C
Hierarchical Novelty Detection for Visual Object Recognition
The essential ingredients of our methods are confidence-calibrated classifiers, data relabeling, and the leave-one-out strategy for modeling novel classes under the hierarchical taxonomy.
Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples
The problem of detecting whether a test sample is from in-distribution (i. e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications.
Confident Multiple Choice Learning
Ensemble methods are arguably the most trustworthy techniques for boosting the performance of machine learning models.
Simplified Stochastic Feedforward Neural Networks
In this paper, we aim at developing efficient training methods for SFNN, in particular using known architectures and pre-trained parameters of DNN.
