Search Results for author: Amy Zhang

Found 50 papers, 21 papers with code

Confidence Contours: Uncertainty-Aware Annotation for Medical Semantic Segmentation

no code implementations15 Aug 2023 Andre Ye, Quan Ze Chen, Amy Zhang

However, as these annotations cannot represent an individual annotator's uncertainty, models trained on them produce uncertainty maps that are difficult to interpret.

Image Segmentation Medical Image Segmentation +1

Structure in Reinforcement Learning: A Survey and Open Problems

no code implementations28 Jun 2023 Aditya Mohan, Amy Zhang, Marius Lindauer

We amalgamate these diverse methodologies under a unified framework, shedding light on the role of structure in the learning problem, and classify these methods into distinct patterns of incorporating structure.

reinforcement-learning Reinforcement Learning (RL)

Generalization Across Observation Shifts in Reinforcement Learning

no code implementations7 Jun 2023 Anuj Mahajan, Amy Zhang

We focus on bisimulation metrics, which provide a powerful means for abstracting task relevant components of the observation and learning a succinct representation space for training the agent using reinforcement learning.


LIV: Language-Image Representations and Rewards for Robotic Control

1 code implementation1 Jun 2023 Yecheng Jason Ma, William Liang, Vaidehi Som, Vikash Kumar, Amy Zhang, Osbert Bastani, Dinesh Jayaraman

We present Language-Image Value learning (LIV), a unified objective for vision-language representation and reward learning from action-free videos with text annotations.

Contrastive Learning Imitation Learning

A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem

no code implementations26 May 2023 Paul Barde, Jakob Foerster, Derek Nowrouzezahrai, Amy Zhang

Training multiple agents to coordinate is an important problem with applications in robotics, game theory, economics, and social sciences.

Multi-agent Reinforcement Learning

Sequence Modeling is a Robust Contender for Offline Reinforcement Learning

1 code implementation23 May 2023 Prajjwal Bhargava, Rohan Chitnis, Alborz Geramifard, Shagun Sodhani, Amy Zhang

Our key findings are: (1) Sequence Modeling requires more data than Q-Learning to learn competitive policies but is more robust; (2) Sequence Modeling is a substantially better choice than both Q-Learning and Imitation Learning in sparse-reward and low-quality data settings; and (3) Sequence Modeling and Imitation Learning are preferable as task horizon increases, or when data is obtained from human demonstrators.

D4RL Imitation Learning +5

Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning

1 code implementation3 Apr 2023 Tongzhou Wang, Antonio Torralba, Phillip Isola, Amy Zhang

In goal-reaching reinforcement learning (RL), the optimal value function has a particular geometry, called quasimetric structure.

reinforcement-learning Reinforcement Learning (RL)

Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement

no code implementations20 Mar 2023 Michael Chang, Alyssa L. Dayan, Franziska Meier, Thomas L. Griffiths, Sergey Levine, Amy Zhang

Object rearrangement is a challenge for embodied agents because solving these tasks requires generalizing across a combinatorially large set of configurations of entities and their locations.

Confidence-aware 3D Gaze Estimation and Evaluation Metric

no code implementations17 Mar 2023 Qiaojie Zheng, Jiucai Zhang, Amy Zhang, Xiaoli Zhang

To address the unreliable and overconfident issues, we introduce a confidence-aware model that predicts uncertainties together with gaze angle estimations.

Gaze Estimation

Dual RL: Unification and New Methods for Reinforcement and Imitation Learning

1 code implementation16 Feb 2023 Harshit Sikchi, Qinqing Zheng, Amy Zhang, Scott Niekum

We show that several state-of-the-art off-policy deep reinforcement learning (RL) algorithms, under both online and offline, RL and imitation learning (IL) settings, can be viewed as dual RL approaches in a unified framework.

Imitation Learning Offline RL +2

Provably Efficient Offline Goal-Conditioned Reinforcement Learning with General Function Approximation and Single-Policy Concentrability

no code implementations7 Feb 2023 Hanlin Zhu, Amy Zhang

To the best of our knowledge, this is the first algorithm with general function approximation and single-policy concentrability that is both statistically efficient and computationally stable.

Contrastive Distillation Is a Sample-Efficient Self-Supervised Loss Policy for Transfer Learning

no code implementations21 Dec 2022 Chris Lengerich, Gabriel Synnaeve, Amy Zhang, Hugh Leather, Kurt Shuster, François Charton, Charysse Redwood

Traditional approaches to RL have focused on learning decision policies directly from episodic decisions, while slowly and implicitly learning the semantics of compositional representations needed for generalization.

Few-Shot Learning Language Modelling +2

Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks

2 code implementations27 Oct 2022 Edwin Zhang, Yujie Lu, William Wang, Amy Zhang

Training generalist agents is difficult across several axes, requiring us to deal with high-dimensional inputs (space), long horizons (time), and multiple and new tasks.

reinforcement-learning Reinforcement Learning (RL)

Latent State Marginalization as a Low-cost Approach for Improving Exploration

1 code implementation3 Oct 2022 Dinghuai Zhang, Aaron Courville, Yoshua Bengio, Qinqing Zheng, Amy Zhang, Ricky T. Q. Chen

While the maximum entropy (MaxEnt) reinforcement learning (RL) framework -- often touted for its exploration and robustness capabilities -- is usually motivated from a probabilistic perspective, the use of deep probabilistic models has not gained much traction in practice due to their inherent complexity.

Continuous Control Reinforcement Learning (RL) +1

VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training

1 code implementation30 Sep 2022 Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, Amy Zhang

Given the inherent cost and scarcity of in-domain, task-specific robot data, learning from large, diverse, offline human videos has emerged as a promising path towards acquiring a generally useful visual representation for control; however, how these human videos can be used for general-purpose reward learning remains an open question.

Offline RL Open-Ended Question Answering +2

Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning

no code implementations27 Apr 2022 Philippe Hansen-Estruch, Amy Zhang, Ashvin Nair, Patrick Yin, Sergey Levine

We learn this representation using a metric form of this abstraction, and show its ability to generalize to new goals in simulation manipulation tasks.

reinforcement-learning Reinforcement Learning (RL)

Robust Policy Learning over Multiple Uncertainty Sets

no code implementations14 Feb 2022 Annie Xie, Shagun Sodhani, Chelsea Finn, Joelle Pineau, Amy Zhang

Reinforcement learning (RL) agents need to be robust to variations in safety-critical environments.

Reinforcement Learning (RL)

Online Decision Transformer

2 code implementations11 Feb 2022 Qinqing Zheng, Amy Zhang, Aditya Grover

Recent work has shown that offline reinforcement learning (RL) can be formulated as a sequence modeling problem (Chen et al., 2021; Janner et al., 2021) and solved via approaches similar to large-scale language modeling.

D4RL Efficient Exploration +2

Learning Representations for Pixel-based Control: What Matters and Why?

no code implementations15 Nov 2021 Manan Tomar, Utkarsh A. Mishra, Amy Zhang, Matthew E. Taylor

A wide range of methods have been proposed to enable efficient learning, leading to sample complexities similar to those in the full state setting.

Contrastive Learning Representation Learning

Block Contextual MDPs for Continual Learning

no code implementations13 Oct 2021 Shagun Sodhani, Franziska Meier, Joelle Pineau, Amy Zhang

In this work, we propose to examine this continual reinforcement learning setting through the block contextual MDP (BC-MDP) framework, which enables us to relax the assumption of stationarity.

Continual Learning Generalization Bounds +2

Learning Minimal Representations with Model Invariance

no code implementations29 Sep 2021 Manan Tomar, Amy Zhang, Matthew E. Taylor

The common representation acts as a implicit invariance objective to avoid the different spurious correlations captured by individual predictors.

Self-Supervised Learning

Provably Efficient Representation Learning in Low-rank Markov Decision Processes

no code implementations22 Jun 2021 Weitong Zhang, Jiafan He, Dongruo Zhou, Amy Zhang, Quanquan Gu

The success of deep reinforcement learning (DRL) is due to the power of learning a representation that is suitable for the underlying exploration and exploitation task.

reinforcement-learning Reinforcement Learning (RL) +1

MBRL-Lib: A Modular Library for Model-based Reinforcement Learning

3 code implementations20 Apr 2021 Luis Pineda, Brandon Amos, Amy Zhang, Nathan O. Lambert, Roberto Calandra

MBRL-Lib is designed as a platform for both researchers, to easily develop, debug and compare new algorithms, and non-expert user, to lower the entry-bar of deploying state-of-the-art algorithms.

Model-based Reinforcement Learning reinforcement-learning +1

Multi-Task Reinforcement Learning with Context-based Representations

2 code implementations11 Feb 2021 Shagun Sodhani, Amy Zhang, Joelle Pineau

We posit that an efficient approach to knowledge transfer is through the use of multiple context-dependent, composable representations shared across a family of tasks.

Multi-Task Learning reinforcement-learning +1

Invariant Representations for Reinforcement Learning without Reconstruction

no code implementations ICLR 2021 Amy Zhang, Rowan Thomas McAllister, Roberto Calandra, Yarin Gal, Sergey Levine

We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.

Causal Inference reinforcement-learning +2

Automating Document Classification with Distant Supervision to Increase the Efficiency of Systematic Reviews

no code implementations9 Dec 2020 Xiaoxiao Li, Rabah Al-Zaidy, Amy Zhang, Stefan Baral, Le Bao, C. Lee Giles

Conclusions: In sum, the automated procedure of document classification presented here could improve both the precision and efficiency of systematic reviews, as well as facilitating live reviews, where reviews are updated regularly.

Document Classification General Classification

Intervention Design for Effective Sim2Real Transfer

1 code implementation3 Dec 2020 Melissa Mozifian, Amy Zhang, Joelle Pineau, David Meger

The goal of this work is to address the recent success of domain randomization and data augmentation for the sim2real setting.

Causal Inference Data Augmentation

Learning Robust State Abstractions for Hidden-Parameter Block MDPs

2 code implementations ICLR 2021 Amy Zhang, Shagun Sodhani, Khimya Khetarpal, Joelle Pineau

Further, we provide transfer and generalization bounds based on task and state similarity, along with sample complexity bounds that depend on the aggregate number of samples across tasks, rather than the number of tasks, a significant improvement over prior work that use the same environment assumptions.

Generalization Bounds Meta Reinforcement Learning +2

Learning Invariant Representations for Reinforcement Learning without Reconstruction

2 code implementations18 Jun 2020 Amy Zhang, Rowan McAllister, Roberto Calandra, Yarin Gal, Sergey Levine

We study how representation learning can accelerate reinforcement learning from rich observations, such as images, without relying either on domain knowledge or pixel-reconstruction.

Causal Inference reinforcement-learning +2

Plan2Vec: Unsupervised Representation Learning by Latent Plans

1 code implementation7 May 2020 Ge Yang, Amy Zhang, Ari S. Morcos, Joelle Pineau, Pieter Abbeel, Roberto Calandra

In this paper we introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning.

Motion Planning reinforcement-learning +2

Invariant Causal Prediction for Block MDPs

1 code implementation ICML 2020 Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, Doina Precup

Generalization across environments is critical to the successful application of reinforcement learning algorithms to real-world challenges.

Causal Inference Variable Selection

Stable Policy Optimization via Off-Policy Divergence Regularization

1 code implementation9 Mar 2020 Ahmed Touati, Amy Zhang, Joelle Pineau, Pascal Vincent

Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO) are among the most successful policy gradient approaches in deep reinforcement learning (RL).

Reinforcement Learning (RL)

Learning Causal State Representations of Partially Observable Environments

no code implementations25 Jun 2019 Amy Zhang, Zachary C. Lipton, Luis Pineda, Kamyar Azizzadenesheli, Anima Anandkumar, Laurent Itti, Joelle Pineau, Tommaso Furlanello

In this paper, we propose an algorithm to approximate causal states, which are the coarsest partition of the joint history of actions and observations in partially-observable Markov decision processes (POMDP).

Causal Inference

Natural Environment Benchmarks for Reinforcement Learning

2 code implementations14 Nov 2018 Amy Zhang, Yuxin Wu, Joelle Pineau

While current benchmark reinforcement learning (RL) tasks have been useful to drive progress in the field, they are in many ways poor substitutes for learning with real-world data.

reinforcement-learning Reinforcement Learning (RL)

Decoupling Dynamics and Reward for Transfer Learning

1 code implementation27 Apr 2018 Amy Zhang, Harsh Satija, Joelle Pineau

Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure.

reinforcement-learning Reinforcement Learning (RL) +1

Mapping the world population one building at a time

no code implementations15 Dec 2017 Tobias G. Tiecke, Xian-Ming Liu, Amy Zhang, Andreas Gros, Nan Li, Gregory Yetman, Talip Kilic, Siobhan Murray, Brian Blankespoor, Espen B. Prydz, Hai-Anh H. Dang

Obtaining high accuracy in estimation of population distribution in rural areas remains a very challenging task due to the simultaneous requirements of sufficient sensitivity and resolution to detect very sparse populations through remote sensing as well as reliable performance at a global scale.

Disaster Response

Building Detection from Satellite Images on a Global Scale

no code implementations27 Jul 2017 Amy Zhang, Xian-Ming Liu, Andreas Gros, Tobias Tiecke

Our work is some of the first to create population density maps from building detection on a large scale.

Density Estimation

Feedback Neural Network for Weakly Supervised Geo-Semantic Segmentation

no code implementations8 Dec 2016 Xianming Liu, Amy Zhang, Tobias Tiecke, Andreas Gros, Thomas S. Huang

Learning from weakly-supervised data is one of the main challenges in machine learning and computer vision, especially for tasks such as image semantic segmentation where labeling is extremely expensive and subjective.

Semantic Segmentation Weakly-supervised Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.