Search Results for author:
Found 20 papers, 8 papers with code
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
We introduce GeneralPoints, an arithmetic reasoning card game, and adopt V-IRL, a real-world navigation environment, to assess how models trained with SFT and RL generalize to unseen variants in both textual and visual domains.
Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
Finally, our framework uses these task rewards to fine-tune the entire VLM with RL.
Is Offline Decision Making Possible with Only Few Samples? Reliable Decisions in Data-Starved Bandits via Trust Region Enhancement
Surprisingly, in this work, we demonstrate that even in such a data-starved setting it may still be possible to find a policy competitive with the optimal one.
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
To understand the roots of these errors, we explore the gap between the visual embedding space of CLIP and vision-only self-supervised learning.
LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models
Developing such algorithms requires tasks that can gauge progress on algorithm design, provide accessible and reproducible evaluations for multi-turn interactions, and cover a range of task properties and challenges in improving reinforcement learning algorithms.
White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
This leads to a family of white-box transformer-like deep network architectures, named CRATE, which are mathematically fully interpretable.
RLIF: Interactive Imitation Learning as Reinforcement Learning
We also provide a unified framework to analyze our RL method and DAgger; for which we present the asymptotic analysis of the suboptimal gap for both methods as well as the non-asymptotic sample complexity bound of our method.
Investigating the Catastrophic Forgetting in Multimodal Large Language Models
However, catastrophic forgetting, a notorious phenomenon where the fine-tuned model fails to retain similar performance compared to the pre-trained model, still remains an inherent problem in multimodal LLMs (MLLM).
Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning
Our approach, calibrated Q-learning (Cal-QL), accomplishes this by learning a conservative value function initialization that underestimates the value of the learned policy from offline data, while also being calibrated, in the sense that the learned Q-values are at a reasonable scale.
Closed-Loop Transcription via Convolutional Sparse Coding
Our method is arguably the first to demonstrate that a concatenation of multiple convolution sparse coding/decoding layers leads to an interpretable and effective autoencoder for modeling the distribution of large-scale natural image datasets.
Understanding the Complexity Gains of Single-Task RL with a Curriculum
Under mild regularity conditions on the curriculum, we show that sequentially solving each task in the multi-task RL problem is more computationally efficient than solving the original single-task problem, without any explicit exploration bonuses or other exploration strategies.
Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity
Reinforcement learning provides an automated framework for learning behaviors from high-level reward specifications, but in practice the choice of reward function can be crucial for good results -- while in principle the reward only needs to specify what the task is, in reality practitioners often need to design more detailed rewards that provide the agent with some hints about how the task should be completed.
Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning
In both OWSP and OWMP settings, we demonstrate that adding {\em intermediate rewards} to subgoals is more computationally efficient than only rewarding the agent once it completes the goal of reaching a terminal state.
Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training
Furthermore, we show that our ConvNorm can reduce the layerwise spectral norm of the weight matrices and hence improve the Lipschitzness of the network, leading to easier training and improved robustness for deep ConvNets.
Measuring GAN Training in Real Time
Generative Adversarial Networks (GAN) are popular generative models of images.
Geometric Analysis of Nonconvex Optimization Landscapes for Overcomplete Learning
Learning overcomplete representations finds many applications in machine learning and data analytics.
Understanding l4-based Dictionary Learning: Interpretation, Stability, and Robustness
Recently, the $\ell^4$-norm maximization has been proposed to solve the sparse dictionary learning (SDL) problem.
Analysis of the Optimization Landscapes for Overcomplete Representation Learning
In this work, we show these problems can be formulated as $\ell^4$-norm optimization problems with spherical constraint, and study the geometric properties of their nonconvex optimization landscapes.
Complete Dictionary Learning via $\ell^4$-Norm Maximization over the Orthogonal Group
Most existing methods solve the dictionary (and sparse representations) based on heuristic algorithms, usually without theoretical guarantees for either optimality or complexity.
Learning to Reconstruct 3D Manhattan Wireframes from a Single Image
In this paper, we propose a method to obtain a compact and accurate 3D wireframe representation from a single image by effectively exploiting global structural regularities.
