Search Results for author: Kaichun Mo

Found 36 papers, 17 papers with code

3D-MVP: 3D Multiview Pretraining for Robotic Manipulation

no code implementations26 Jun 2024 Shengyi Qian, Kaichun Mo, Valts Blukis, David F. Fouhey, Dieter Fox, Ankit Goyal

Our results suggest that 3D-aware pretraining is a promising approach to improve sample efficiency and generalization of vision-based robotic manipulation policies.

Decoder Robot Manipulation +1

URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images

no code implementations19 May 2024 Zoey Chen, Aaron Walsman, Marius Memmel, Kaichun Mo, Alex Fang, Karthikeya Vemuri, Alan Wu, Dieter Fox, Abhishek Gupta

We present an integrated end-to-end pipeline that generates simulation scenes complete with articulated kinematic and dynamic structures from real-world images and use these for training robotic control policies.

Scene Generation

Towards Learning Geometric Eigen-Lengths Crucial for Fitting Tasks

no code implementations25 Dec 2023 Yijia Weng, Kaichun Mo, Ruoxi Shi, Yanchao Yang, Leonidas J. Guibas

In this work, we therefore for the first time formulate and propose a novel learning problem on this question and set up a benchmark suite including tasks, data, and evaluation metrics for studying the problem.

Common Sense Reasoning

STOW: Discrete-Frame Segmentation and Tracking of Unseen Objects for Warehouse Picking Robots

no code implementations4 Nov 2023 Yi Li, Muru Zhang, Markus Grotz, Kaichun Mo, Dieter Fox

Segmentation and tracking of unseen object instances in discrete frames pose a significant challenge in dynamic industrial robotic contexts, such as distribution warehouses.

Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects

no code implementations NeurIPS 2023 Chuanruo Ning, Ruihai Wu, Haoran Lu, Kaichun Mo, Hao Dong

Our framework explicitly estimates the geometric similarity across different categories, identifying local areas that differ from shapes in the training categories for efficient exploration while concurrently transferring affordance knowledge to similar parts of the objects.

Efficient Exploration Few-Shot Learning

JacobiNeRF: NeRF Shaping with Mutual Information Gradients

1 code implementation CVPR 2023 Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas

We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.

Instance Segmentation Semantic Segmentation

SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometry

no code implementations16 Feb 2023 Lin Gao, Jia-Mu Sun, Kaichun Mo, Yu-Kun Lai, Leonidas J. Guibas, Jie Yang

We propose SCENEHGN, a hierarchical graph network for 3D indoor scenes that takes into account the full hierarchy from the room level to the object level, then finally to the object part level.

Scene Generation

Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing

no code implementations1 Nov 2022 Jeonghyun Kim, Kaichun Mo, Minhyuk Sung, Woontack Woo

We propose Seg&Struct, a supervised learning framework leveraging the interplay between part segmentation and structure inference and demonstrating their synergy in an integrated framework.

Segmentation

COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos

no code implementations ICCV 2023 Boxiao Pan, Bokui Shen, Davis Rempe, Despoina Paschalidou, Kaichun Mo, Yanchao Yang, Leonidas J. Guibas

In this work, we introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras.

Collision Avoidance Synthetic Data Generation

DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulation

no code implementations5 Jul 2022 Yan Zhao, Ruihai Wu, Zhehuan Chen, Yourong Zhang, Qingnan Fan, Kaichun Mo, Hao Dong

It is essential yet challenging for future home-assistant robots to understand and manipulate diverse 3D objects in daily human environments.

3D geometry

Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction

no code implementations CVPR 2022 Yining Hong, Kaichun Mo, Li Yi, Leonidas J. Guibas, Antonio Torralba, Joshua B. Tenenbaum, Chuang Gan

Specifically, FixNet consists of a perception module to extract the structured representation from the 3D point cloud, a physical dynamics prediction module to simulate the results of interactions on 3D objects, and a functionality prediction module to evaluate the functionality and choose the correct fix.

GIMO: Gaze-Informed Human Motion Prediction in Context

1 code implementation20 Apr 2022 Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C. Karen Liu, Leonidas J. Guibas

We perform an extensive study of the benefits of leveraging the eye gaze for ego-centric human motion prediction with various state-of-the-art architectures.

Human motion prediction motion prediction

Object Pursuit: Building a Space of Objects via Discriminative Weight Generation

no code implementations ICLR 2022 Chuanyu Pan, Yanchao Yang, Kaichun Mo, Yueqi Duan, Leonidas Guibas

We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.

Disentanglement Object

IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

no code implementations ICLR 2022 Qi Li, Kaichun Mo, Yanchao Yang, Hang Zhao, Leonidas Guibas

While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).

Object

AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions

no code implementations1 Dec 2021 Yian Wang, Ruihai Wu, Kaichun Mo, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, Hao Dong

Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.

Friction

Learning to Regrasp by Learning to Place

1 code implementation18 Sep 2021 Shuo Cheng, Kaichun Mo, Lin Shao

In this paper, we explore whether a robot can learn to regrasp a diverse set of objects to achieve various desired grasp poses.

Diversity Object

O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning

1 code implementation29 Jun 2021 Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, Leonidas Guibas

Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.

Diversity Object

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

no code implementations ICLR 2022 Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, Hao Dong

In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.

Where2Act: From Pixels to Actions for Articulated 3D Objects

1 code implementation ICCV 2021 Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani

One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment.

Compositionally Generalizable 3D Structure Prediction

1 code implementation4 Dec 2020 Songfang Han, Jiayuan Gu, Kaichun Mo, Li Yi, Siyu Hu, Xuejin Chen, Hao Su

However, there remains a much more difficult and under-explored issue on how to generalize the learned skills over unseen object categories that have very different shape geometry distributions.

3D Shape Reconstruction Object +1

DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation

1 code implementation12 Aug 2020 Jie Yang, Kaichun Mo, Yu-Kun Lai, Leonidas J. Guibas, Lin Gao

While significant progress has been made, especially with recent deep generative models, it remains a challenge to synthesize high-quality shapes with rich geometric details and complex structure, in a controllable manner.

3D Shape Generation

Generative 3D Part Assembly via Dynamic Graph Learning

3 code implementations NeurIPS 2020 Jialei Huang, Guanqi Zhan, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas Guibas, Hao Dong

Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.

Graph Learning Graph Neural Network +2

Rethinking Sampling in 3D Point Cloud Generative Adversarial Networks

no code implementations12 Jun 2020 He Wang, Zetian Jiang, Li Yi, Kaichun Mo, Hao Su, Leonidas J. Guibas

We further study how different evaluation metrics weigh the sampling pattern against the geometry and propose several perceptual metrics forming a sampling spectrum of metrics.

Clustering valid

SAPIEN: A SimulAted Part-based Interactive ENvironment

1 code implementation CVPR 2020 Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, Hao Su

To achieve this task, a simulated environment with physically realistic simulation, sufficient articulated objects, and transferability to the real robot is indispensable.

Attribute Reinforcement Learning

PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions

1 code implementation ECCV 2020 Kaichun Mo, He Wang, Xinchen Yan, Leonidas J. Guibas

3D generative shape modeling is a fundamental research area in computer vision and interactive computer graphics, with many real-world applications.

3D Shape Generation

StructEdit: Learning Structural Shape Variations

1 code implementation CVPR 2020 Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas

Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation.

StructureNet: Hierarchical Graph Networks for 3D Shape Generation

2 code implementations1 Aug 2019 Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas

We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families; and (iii) can be used to generate a great diversity of realistic structured shape geometries.

3D Shape Generation

The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation

1 code implementation24 Feb 2018 Kaichun Mo, Haoxiang Li, Zhe Lin, Joon-Young Lee

Synthetic data suffers from domain gap to the real-world scenes while visual inputs rendered from 3D reconstructed scenes have undesired holes and artifacts.

Robotics

Cannot find the paper you are looking for? You can Submit a new open access paper.