no code implementations • 26 Jun 2024 • Shengyi Qian, Kaichun Mo, Valts Blukis, David F. Fouhey, Dieter Fox, Ankit Goyal
Our results suggest that 3D-aware pretraining is a promising approach to improve sample efficiency and generalization of vision-based robotic manipulation policies.
no code implementations • 19 May 2024 • Zoey Chen, Aaron Walsman, Marius Memmel, Kaichun Mo, Alex Fang, Karthikeya Vemuri, Alan Wu, Dieter Fox, Abhishek Gupta
We present an integrated end-to-end pipeline that generates simulation scenes complete with articulated kinematic and dynamic structures from real-world images and use these for training robotic control policies.
no code implementations • 7 Jan 2024 • Xianghui Xie, Xi Wang, Nikos Athanasiou, Bharat Lal Bhatnagar, Chun-Hao P. Huang, Kaichun Mo, Hao Chen, Xia Jia, Zerui Zhang, Liangxian Cui, Xiao Lin, Bingqiao Qian, Jie Xiao, Wenfei Yang, Hyeongjin Nam, Daniel Sungho Jung, Kihoon Kim, Kyoung Mu Lee, Otmar Hilliges, Gerard Pons-Moll
Modeling the interaction between humans and objects has been an emerging research direction in recent years.
no code implementations • 25 Dec 2023 • Yijia Weng, Kaichun Mo, Ruoxi Shi, Yanchao Yang, Leonidas J. Guibas
In this work, we therefore for the first time formulate and propose a novel learning problem on this question and set up a benchmark suite including tasks, data, and evaluation metrics for studying the problem.
no code implementations • 4 Nov 2023 • Yi Li, Muru Zhang, Markus Grotz, Kaichun Mo, Dieter Fox
Segmentation and tracking of unseen object instances in discrete frames pose a significant challenge in dynamic industrial robotic contexts, such as distribution warehouses.
no code implementations • NeurIPS 2023 • Chuanruo Ning, Ruihai Wu, Haoran Lu, Kaichun Mo, Hao Dong
Our framework explicitly estimates the geometric similarity across different categories, identifying local areas that differ from shapes in the training categories for efficient exploration while concurrently transferring affordance knowledge to similar parts of the objects.
1 code implementation • CVPR 2023 • Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas
We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.
no code implementations • CVPR 2024 • Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas
A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.
no code implementations • 16 Feb 2023 • Lin Gao, Jia-Mu Sun, Kaichun Mo, Yu-Kun Lai, Leonidas J. Guibas, Jie Yang
We propose SCENEHGN, a hierarchical graph network for 3D indoor scenes that takes into account the full hierarchy from the room level to the object level, then finally to the object part level.
no code implementations • CVPR 2024 • Razvan-George Pasca, Alexey Gavryushin, Muhammad Hamza, Yen-Ling Kuo, Kaichun Mo, Luc van Gool, Otmar Hilliges, Xi Wang
This task requires an understanding of the spatio-temporal context formed by past actions on objects, coined action context.
no code implementations • 1 Nov 2022 • Jeonghyun Kim, Kaichun Mo, Minhyuk Sung, Woontack Woo
We propose Seg&Struct, a supervised learning framework leveraging the interplay between part segmentation and structure inference and demonstrating their synergy in an integrated framework.
no code implementations • ICCV 2023 • Boxiao Pan, Bokui Shen, Davis Rempe, Despoina Paschalidou, Kaichun Mo, Yanchao Yang, Leonidas J. Guibas
In this work, we introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras.
no code implementations • 5 Jul 2022 • Yan Zhao, Ruihai Wu, Zhehuan Chen, Yourong Zhang, Qingnan Fan, Kaichun Mo, Hao Dong
It is essential yet challenging for future home-assistant robots to understand and manipulate diverse 3D objects in daily human environments.
no code implementations • CVPR 2022 • Yining Hong, Kaichun Mo, Li Yi, Leonidas J. Guibas, Antonio Torralba, Joshua B. Tenenbaum, Chuang Gan
Specifically, FixNet consists of a perception module to extract the structured representation from the 3D point cloud, a physical dynamics prediction module to simulate the results of interactions on 3D objects, and a functionality prediction module to evaluate the functionality and choose the correct fix.
1 code implementation • 20 Apr 2022 • Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C. Karen Liu, Leonidas J. Guibas
We perform an extensive study of the benefits of leveraging the eye gaze for ego-centric human motion prediction with various state-of-the-art architectures.
no code implementations • 19 Dec 2021 • Mingxin Yu, Lin Shao, Zhehuan Chen, Tianhao Wu, Qingnan Fan, Kaichun Mo, Hao Dong
Part assembly is a typical but challenging task in robotics, where robots assemble a set of individual parts into a complete shape.
no code implementations • ICLR 2022 • Chuanyu Pan, Yanchao Yang, Kaichun Mo, Yueqi Duan, Leonidas Guibas
We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.
no code implementations • ICLR 2022 • Qi Li, Kaichun Mo, Yanchao Yang, Hang Zhao, Leonidas Guibas
While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).
no code implementations • 1 Dec 2021 • Yian Wang, Ruihai Wu, Kaichun Mo, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, Hao Dong
Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.
1 code implementation • 18 Sep 2021 • Shuo Cheng, Kaichun Mo, Lin Shao
In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses.
1 code implementation • 29 Jun 2021 • Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, Leonidas Guibas
Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.
no code implementations • ICLR 2022 • Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, Hao Dong
In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.
1 code implementation • ICCV 2021 • Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani
One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment.
1 code implementation • 4 Dec 2020 • Songfang Han, Jiayuan Gu, Kaichun Mo, Li Yi, Siyu Hu, Xuejin Chen, Hao Su
However, there remains a much more difficult and under-explored issue on how to generalize the learned skills over unseen object categories that have very different shape geometry distributions.
1 code implementation • 12 Aug 2020 • Jie Yang, Kaichun Mo, Yu-Kun Lai, Leonidas J. Guibas, Lin Gao
While significant progress has been made, especially with recent deep generative models, it remains a challenge to synthesize high-quality shapes with rich geometric details and complex structure, in a controllable manner.
3 code implementations • NeurIPS 2020 • Jialei Huang, Guanqi Zhan, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas Guibas, Hao Dong
Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.
no code implementations • 12 Jun 2020 • He Wang, Zetian Jiang, Li Yi, Kaichun Mo, Hao Su, Leonidas J. Guibas
We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics.
1 code implementation • ECCV 2020 • Yichen Li, Kaichun Mo, Lin Shao, Minhyuk Sung, Leonidas Guibas
Autonomous assembly is a crucial capability for robots in many applications.
1 code implementation • CVPR 2020 • Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su
To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.
1 code implementation • ECCV 2020 • Kaichun Mo, He Wang, Xinchen Yan, Leonidas J. Guibas
3D generative shape modeling is a fundamental research area in computer vision and interactive computer graphics, with many real-world applications.
1 code implementation • ICLR 2020 • Tiange Luo, Kaichun Mo, Zhiao Huang, Jiarui Xu, Siyu Hu, Li-Wei Wang, Hao Su
We address the problem of discovering 3D parts for objects in unseen categories.
1 code implementation • CVPR 2020 • Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas
Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation.
2 code implementations • 1 Aug 2019 • Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas
We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families; and (iii) can be used to generate a great diversity of realistic structured shape geometries.
5 code implementations • CVPR 2019 • Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Leonidas J. Guibas, Hao Su
We present PartNet: a consistent, large-scale dataset of 3D objects annotated with fine-grained, instance-level, and hierarchical 3D part information.
Ranked #3 on 3D Instance Segmentation on PartNet
1 code implementation • 24 Feb 2018 • Kaichun Mo, Haoxiang Li, Zhe Lin, Joon-Young Lee
Synthetic data suffers from domain gap to the real-world scenes while visual inputs rendered from 3D reconstructed scenes have undesired holes and artifacts.
Robotics
109 code implementations • CVPR 2017 • Charles R. Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas
Point cloud is an important type of geometric data structure.
Ranked #1 on 3D Face Reconstruction on !(()&&!|*|*|