Search Results for author:
Found 34 papers, 16 papers with code
Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters
Motivated by the success of ensembles for uncertainty estimation in supervised learning, we take a renewed look at how ensembles of $Q$-functions can be leveraged as the primary source of pessimism for offline reinforcement learning (RL).
GPT-4 Technical Report
We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.
Ranked #1 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
A Minimalist Approach to Offline Reinforcement Learning
Offline reinforcement learning (RL) defines the task of learning from a fixed batch of data.
Scaling Instruction-Finetuned Language Models
We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation).
Ranked #1 on
Multi-task Language Understanding
on BBH-nlp
Large Language Models are Zero-Shot Reasoners
Pretrained large language models (LLMs) are widely used in many sub-fields of natural language processing (NLP) and generally known as excellent few-shot learners with task-specific exemplars.
Ranked #1 on
Arithmetic Reasoning
on MultiArith
Human-centric Dialog Training via Offline Reinforcement Learning
We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL).
VaxNeRF: Revisiting the Classic for Voxel-Accelerated Neural Radiance Field
We hope VaxNeRF -- a careful combination of a classic technique with a deep method (that arguably replaced it) -- can empower and accelerate new NeRF extensions and applications, with its simplicity, portability, and reliable performance gains.
Can Wikipedia Help Offline Reinforcement Learning?
In this paper, we look to take advantage of this formulation of reinforcement learning as sequence modeling and investigate the transferability of pre-trained sequence models on other domains (vision, language) when finetuned on offline RL tasks (control, games).
For SALE: State-Action Representation Learning for Deep Reinforcement Learning
In the field of reinforcement learning (RL), representation learning is a proven tool for complex image-based tasks, but is often overlooked for environments with low-level states, such as physical control problems.
Generalized Decision Transformer for Offline Hindsight Information Matching
We present Generalized Decision Transformer (GDT) for solving any HIM problem, and show how different choices for the feature function and the anti-causal aggregator not only recover DT as a special case, but also lead to novel Categorical DT (CDT) and Bi-directional DT (BDT) for matching different statistics of the future.
Domain Prompt Learning for Efficiently Adapting CLIP to Unseen Domains
We propose Domain Prompt Learning (DPL) as a novel approach for domain inference in the form of conditional prompt generation.
Ranked #4 on
Transfer Learning
on Office-Home
Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning
These results show which implementation or code details are co-adapted and co-evolved with algorithms, and which are transferable across algorithms: as examples, we identified that tanh Gaussian policy and network sizes are highly adapted to algorithmic types, while layer normalization and ELU are critical for MPO's performances but also transfer to noticeable gains in SAC.
A System for Morphology-Task Generalization via Unified Representation and Behavior Distillation
The rise of generalist large-scale models in natural language and vision has made us expect that a massive data-driven approach could achieve broader generalization in other domains such as continuous control.
Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning
Progress in deep reinforcement learning (RL) research is largely enabled by benchmark task environments.
Collective Intelligence for 2D Push Manipulations with Mobile Robots
While natural systems often present collective intelligence that allows them to self-organize and adapt to changes, the equivalent is missing in most artificial systems.
Weakly-Supervised Reinforcement Learning for Controllable Behavior
Reinforcement learning (RL) is a powerful framework for learning to take actions to solve tasks.
EMaQ: Expected-Max Q-Learning Operator for Simple Yet Effective Offline and Online RL
In this work, we closely investigate an important simplification of BCQ -- a prior approach for offline RL -- which removes a heuristic design choice and naturally restricts extracted policies to remain exactly within the support of a given behavior policy.
Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learning
Learning to reach goal states and learning diverse skills through mutual information (MI) maximization have been proposed as principled frameworks for self-supervised reinforcement learning, allowing agents to acquire broadly applicable multitask policies with minimal reward engineering.
Why so pessimistic? Estimating uncertainties for offline RL through ensembles, and why their independence matters.
Motivated by the success of MSG, we investigate whether efficient approximations to ensembles can be as effective.
Distributional Decision Transformer for Hindsight Information Matching
Inspired by distributional and state-marginal matching literatures in RL, we demonstrate that all these approaches are essentially doing hindsight information matching (HIM) -- training policies that can output the rest of trajectory that matches a given future state information statistics.
Why Should I Trust You, Bellman? Evaluating the Bellman Objective with Off-Policy Data
In this work, we analyze the effectiveness of the Bellman equation as a proxy objective for value prediction accuracy in off-policy evaluation.
Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximization
While reward maximization is at the core of RL, reward engineering is not the only -- sometimes nor the easiest -- way for specifying complex behaviors.
Why Should I Trust You, Bellman? The Bellman Error is a Poor Replacement for Value Error
In this work, we study the use of the Bellman equation as a surrogate objective for value prediction accuracy.
Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning
Despite the simplicity of this objective, the compositional nature of building diverse blueprints from a set of blocks leads to an explosion of complexity in structures that agents encounter.
World Robot Challenge 2020 -- Partner Robot: A Data-Driven Approach for Room Tidying with Mobile Manipulator
Tidying up a household environment using a mobile manipulator poses various challenges in robotics, such as adaptation to large real-world environmental variations, and safe and robust deployment in the presence of humans. The Partner Robot Challenge in World Robot Challenge (WRC) 2020, a global competition held in September 2021, benchmarked tidying tasks in the real home environments, and importantly, tested for full system performances. For this challenge, we developed an entire household service robot system, which leverages a data-driven approach to adapt to numerous edge cases that occur during the execution, instead of classical manual pre-programmed solutions.
Deep Billboards towards Lossless Real2Sim in Virtual Reality
An aspirational goal for virtual reality (VR) is to bring in a rich diversity of real world objects losslessly.
Mind's Eye: Grounded Language Model Reasoning through Simulation
By training solely on written text, current language models (LMs) miss the grounded experience of humans in the real-world -- their failure to relate language to the physical world causes knowledge to be misrepresented and obvious mistakes in their reasoning.
Large Language Models Can Self-Improve
We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74. 4%->82. 1% on GSM8K, 78. 2%->83. 0% on DROP, 90. 0%->94. 4% on OpenBookQA, and 63. 4%->67. 9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label.
Ranked #1 on
Question Answering
on DROP
Aligning Text-to-Image Models using Human Feedback
Our results demonstrate the potential for learning from human feedback to significantly improve text-to-image models.
Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learning
Without manually-designed controller nor human demonstrations, we demonstrate that with careful Sim2Real considerations, our policies trained with RL in simulation enable two xArm6 robots to solve the U-shape assembly task with a success rate of above90% in simulation, and 50% on real hardware without any additional real-world fine-tuning.
Learning a Universal Human Prior for Dexterous Manipulation from Human Preference
Generating human-like behavior on robots is a great challenge especially in dexterous manipulation tasks with robotic hands.
Multimodal Web Navigation with Instruction-Finetuned Foundation Models
The progress of autonomous web navigation has been hindered by the dependence on billions of exploratory interactions via online reinforcement learning, and domain-specific model designs that make it difficult to leverage generalization from rich out-of-domain data.
DreamSparse: Escaping from Plato's Cave with 2D Frozen Diffusion Model Given Sparse Views
Leveraging the strong image priors in the pre-trained diffusion models, DreamSparse is capable of synthesizing high-quality novel views for both object and scene-level images and generalising to open-set images.
