no code implementations • 5 Nov 2024 • Soroush Nasiriany, Sean Kirmani, Tianli Ding, Laura Smith, Yuke Zhu, Danny Driess, Dorsa Sadigh, Ted Xiao
Our method, RT-Affordance, is a hierarchical model that first proposes an affordance plan given the task language, and then conditions the policy on this affordance plan to perform manipulation.
no code implementations • 4 Jun 2024 • Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, Yuke Zhu
We advocate using realistic physical simulation as a means to scale environments, tasks, and datasets for robot learning methods.
no code implementations • 1 Mar 2024 • Tian Gao, Soroush Nasiriany, Huihan Liu, Quantao Yang, Yuke Zhu
Imitation learning has shown great potential for enabling robots to acquire complex manipulation behaviors.
no code implementations • 12 Feb 2024 • Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, Quan Vuong, Tingnan Zhang, Tsang-Wei Edward Lee, Kuang-Huei Lee, Peng Xu, Sean Kirmani, Yuke Zhu, Andy Zeng, Karol Hausman, Nicolas Heess, Chelsea Finn, Sergey Levine, Brian Ichter
In each iteration, the image is annotated with a visual representation of proposals that the VLM can refer to (e. g., candidate robot actions, localizations, or trajectories).
2 code implementations • 26 Oct 2023 • Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, Dieter Fox
Imitation learning from a large set of human demonstrations has proved to be an effective paradigm for building capable robot agents.
no code implementations • 15 Nov 2022 • Huihan Liu, Soroush Nasiriany, Lance Zhang, Zhiyao Bao, Yuke Zhu
To harness the capabilities of state-of-the-art robot learning models while embracing their imperfections, we present Sirius, a principled framework for humans and robots to collaborate through a division of work.
no code implementations • 20 Oct 2022 • Soroush Nasiriany, Tian Gao, Ajay Mandlekar, Yuke Zhu
Imitation learning offers a promising path for robots to learn general-purpose behaviors, but traditionally has exhibited limited scalability due to high data supervision requirements and brittle generalization.
1 code implementation • 7 Oct 2021 • Soroush Nasiriany, Huihan Liu, Yuke Zhu
Realistic manipulation tasks require a robot to interact with an environment with a prolonged sequence of motor actions.
1 code implementation • 6 Aug 2021 • Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, Roberto Martín-Martín
Based on the study, we derive a series of lessons including the sensitivity to different algorithmic design choices, the dependence on the quality of the demonstrations, and the variability based on the stopping criteria due to the different objectives in training and evaluation.
no code implementations • 23 Apr 2021 • Soroush Nasiriany, Vitchyr H. Pong, Ashvin Nair, Alexander Khazatsky, Glen Berseth, Sergey Levine
Contextual policies provide this capability in principle, but the representation of the context determines the degree of generalization and expressivity.
7 code implementations • 25 Sep 2020 • Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martín-Martín, Abhishek Joshi, Kevin Lin, Abhiram Maddukuri, Soroush Nasiriany, Yifeng Zhu
robosuite is a simulation framework for robot learning powered by the MuJoCo physics engine.
1 code implementation • NeurIPS 2019 • Soroush Nasiriany, Vitchyr H. Pong, Steven Lin, Sergey Levine
Planning methods can solve temporally extended sequential decision making problems by composing simple behaviors.