Search Results for author: Ruohan Zhang

Found 22 papers, 4 papers with code

TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

no code implementations12 Mar 2024 Shivin Dass, Wensi Ai, Yuqian Jiang, Samik Singh, Jiaheng Hu, Ruohan Zhang, Peter Stone, Ben Abbatematteo, Roberto Martín-Martín

This problem is more severe in mobile manipulation, where collecting demonstrations is harder than in stationary manipulation due to the lack of available and easy-to-use teleoperation interfaces.

Imitation Learning

DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation

no code implementations12 Mar 2024 Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, C. Karen Liu

Imitation learning from human hand motion data presents a promising avenue for imbuing robots with human-like dexterity in real-world manipulation tasks.

Imitation Learning

NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities

no code implementations2 Nov 2023 Ruohan Zhang, Sharon Lee, Minjune Hwang, Ayano Hiranaka, Chen Wang, Wensi Ai, Jin Jie Ryan Tan, Shreya Gupta, Yilun Hao, Gabrael Levine, Ruohan Gao, Anthony Norcia, Li Fei-Fei, Jiajun Wu

We present Neural Signal Operated Intelligent Robots (NOIR), a general-purpose, intelligent brain-robot interface system that enables humans to command robots to perform everyday activities through brain signals.


Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AI

1 code implementation3 Oct 2023 Emily Jin, Jiaheng Hu, Zhuoyi Huang, Ruohan Zhang, Jiajun Wu, Li Fei-Fei, Roberto Martín-Martín

We present Mini-BEHAVIOR, a novel benchmark for embodied AI that challenges agents to use reasoning and decision-making skills to solve complex activities that resemble everyday human challenges.

Decision Making

Primitive Skill-based Robot Learning from Human Evaluative Feedback

no code implementations28 Jul 2023 Ayano Hiranaka, Minjune Hwang, Sharon Lee, Chen Wang, Li Fei-Fei, Jiajun Wu, Ruohan Zhang

By combining them, SEED reduces the human effort required in RLHF and increases safety in training robot manipulation with RL in real-world settings.

reinforcement-learning Reinforcement Learning (RL) +1

VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

1 code implementation12 Jul 2023 Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Li Fei-Fei

The composed value maps are then used in a model-based planning framework to zero-shot synthesize closed-loop robot trajectories with robustness to dynamic perturbations.

Language Modelling Robot Manipulation

Modeling Dynamic Environments with Scene Graph Memory

no code implementations27 May 2023 Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Emily Jin, Chengshu Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, Roberto Martín-Martín

We evaluate our method in the Dynamic House Simulator, a new benchmark that creates diverse dynamic graphs following the semantic patterns typically seen at homes, and show that NEP can be trained to predict the locations of objects in a variety of environments with diverse object movement dynamics, outperforming baselines both in terms of new scene adaptability and overall accuracy.

Link Prediction

Partial-View Object View Synthesis via Filtered Inversion

no code implementations3 Apr 2023 Fan-Yun Sun, Jonathan Tremblay, Valts Blukis, Kevin Lin, Danfei Xu, Boris Ivanovic, Peter Karkus, Stan Birchfield, Dieter Fox, Ruohan Zhang, Yunzhu Li, Jiajun Wu, Marco Pavone, Nick Haber

At inference, given one or more views of a novel real-world object, FINV first finds a set of latent codes for the object by inverting the generative model from multiple initial seeds.


A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions

no code implementations7 Aug 2021 Tianzi Zang, Yanmin Zhu, Haobing Liu, Ruohan Zhang, Jiadi Yu

In this survey paper, we first proposed a two-level taxonomy of cross-domain recommendation which classifies different recommendation scenarios and recommendation tasks.

Recommendation Systems

Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks

no code implementations13 Jul 2021 Ruohan Zhang, Faraz Torabi, Garrett Warnell, Peter Stone

A longstanding goal of artificial intelligence is to create artificial agents capable of learning to perform tasks that require sequential decision making.

Decision Making

Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation

1 code implementation NeurIPS 2021 Lin Guan, Mudit Verma, Sihang Guo, Ruohan Zhang, Subbarao Kambhampati

We focus on the task of learning from feedback, in which the human trainer not only gives binary evaluative "good" or "bad" feedback for queried state-action pairs, but also provides a visual explanation by annotating relevant features in images.

Atari Games Data Augmentation +3

Efficiently Guiding Imitation Learning Agents with Human Gaze

no code implementations28 Feb 2020 Akanksha Saran, Ruohan Zhang, Elaine Schaertl Short, Scott Niekum

Based on similarities between the attention of reinforcement learning agents and human gaze, we propose a novel approach for utilizing gaze data in a computationally efficient manner, as part of an auxiliary loss function, which guides a network to have higher activations in image regions where the human's gaze fixated.

Atari Games Imitation Learning

Leveraging Human Guidance for Deep Reinforcement Learning Tasks

no code implementations21 Sep 2019 Ruohan Zhang, Faraz Torabi, Lin Guan, Dana H. Ballard, Peter Stone

Reinforcement learning agents can learn to solve sequential decision tasks by interacting with the environment.

Imitation Learning reinforcement-learning +1

Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset

1 code implementation15 Mar 2019 Ruohan Zhang, Calen Walshe, Zhuode Liu, Lin Guan, Karl S. Muller, Jake A. Whritner, Luxin Zhang, Mary M. Hayhoe, Dana H. Ballard

We hope that the scale and quality of this dataset can provide more opportunities to researchers in the areas of visual attention, imitation learning, and reinforcement learning.

Imitation Learning

An initial attempt of combining visual selective attention with deep reinforcement learning

no code implementations11 Nov 2018 Liu Yuezhang, Ruohan Zhang, Dana H. Ballard

We visualize and analyze the feature maps of DQN on a toy problem Catch, and propose an approach to combine visual selective attention with deep reinforcement learning.

Atari Games feature selection +3

AGIL: Learning Attention from Human for Visuomotor Tasks

no code implementations ECCV 2018 Ruohan Zhang, Zhuode Liu, Luxin Zhang, Jake A. Whritner, Karl S. Muller, Mary M. Hayhoe, Dana H. Ballard

When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze.

Atari Games Imitation Learning

Dual Decomposed Learning with Factorwise Oracle for Structural SVM of Large Output Domain

no code implementations NeurIPS 2016 Ian En-Hsu Yen, Xiangru Huang, Kai Zhong, Ruohan Zhang, Pradeep K. Ravikumar, Inderjit S. Dhillon

In this work, we show that, by decomposing training of Structural Support Vector Machine (SVM) into a series of multiclass SVM problems connected through messages, one can replace expensive structured oracle with Factorwise Maximization Oracle (FMO) that allows efficient implementation of complexity sublinear to the factor domain.

Cannot find the paper you are looking for? You can Submit a new open access paper.