1 code implementation • ECCV 2020 • Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, Leonidas Guibas
Due to the scarcity and unsuitability of existent 3D-oriented linguistic resources for this task, we first develop two large-scale and complementary visio-linguistic datasets: i) extbf{ extit{Sr3D}}, which contains 83. 5K template-based utterances leveraging extit{spatial relations} with other fine-grained object classes to localize a referred object in a given scene, and ii) extbf{ extit{Nr3D}} which contains 41. 5K extit{natural, free-form}, utterances collected by deploying a 2-player object reference game in 3D scenes.
1 code implementation • ECCV 2020 • Oshri Halimi, Ido Imanuel, Or Litany, Giovanni Trappolini, Emanuele Rodolà, Leonidas Guibas, Ron Kimmel
Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.
no code implementations • 28 Jan 2025 • Maximilian Dax, Jordi Berbel, Jan Stria, Leonidas Guibas, Urs Bergmann
We generate abstractions of buildings, reflecting the essential aspects of their geometry and structure, by learning to invert procedural models.
1 code implementation • 3 Jan 2025 • Wenyan Cong, Kevin Wang, Jiahui Lei, Colton Stearns, Yuanhao Cai, Dilin Wang, Rakesh Ranjan, Matt Feiszli, Leonidas Guibas, Zhangyang Wang, Weiyao Wang, Zhiwen Fan
Efficiently reconstructing accurate 3D models from monocular video is a key challenge in computer vision, critical for advancing applications in virtual reality, robotics, and scene understanding.
no code implementations • 5 Dec 2024 • Yiqing Liang, Mikhail Okunev, Mikaela Angelina Uy, Runfeng Li, Leonidas Guibas, James Tompkin, Adam W. Harley
Gaussian splatting methods are emerging as a popular approach for converting multi-view image data into scene representations that allow view synthesis.
1 code implementation • 29 Nov 2024 • Yang You, Yixin Li, Congyue Deng, Yue Wang, Leonidas Guibas
Vision foundation models, particularly the ViT family, have revolutionized image understanding by providing rich semantic features.
no code implementations • 27 Nov 2024 • Shengqu Cai, Eric Chan, Yunzhi Zhang, Leonidas Guibas, Jiajun Wu, Gordon Wetzstein
We first leverage a text-to-image diffusion model's in-context generation ability to create grids of images and curate a large paired dataset with the help of a Visual-Language Model.
no code implementations • 30 Oct 2024 • Qianxu Wang, Congyue Deng, Tyler Ga Wei Lum, Yuanpei Chen, Yaodong Yang, Jeannette Bohg, Yixin Zhu, Leonidas Guibas
One-shot transfer of dexterous grasps to novel scenes with object and context variations has been a challenging problem.
1 code implementation • 24 Oct 2024 • Hansheng Chen, Bokui Shen, Yulin Liu, Ruoxi Shi, Linqi Zhou, Connor Z. Lin, Jiayuan Gu, Hao Su, Gordon Wetzstein, Leonidas Guibas
Multi-view image diffusion models have significantly advanced open-domain 3D object generation.
no code implementations • 23 Sep 2024 • Zilu Li, Guandao Yang, Qingqing Zhao, Xi Deng, Leonidas Guibas, Bharath Hariharan, Gordon Wetzstein
This paper proposes a novel approach to construct learnable parametric control variates functions from arbitrary neural network architectures.
no code implementations • 18 Sep 2024 • Jihyeon Je, Jiayi Liu, Guandao Yang, Boyang Deng, Shengqu Cai, Gordon Wetzstein, Or Litany, Leonidas Guibas
In contrast, learning-based methods may be more robust to noise, but often overlook partial symmetries due to the scarcity of annotated data.
no code implementations • 30 Aug 2024 • Xiaoshuai Zhang, Zhicheng Wang, Howard Zhou, Soham Ghosh, Danushen Gnanapragasam, Varun Jampani, Hao Su, Leonidas Guibas
To advance the state of the art in the creation of 3D foundation models, this paper introduces the ConDense framework for 3D pre-training utilizing existing pre-trained 2D networks and large-scale multi-view datasets.
no code implementations • 25 Aug 2024 • Rundong Luo, Haoran Geng, Congyue Deng, Puhao Li, Zan Wang, Baoxiong Jia, Leonidas Guibas, Siyuan Huang
We also demonstrate our applications in 3D printing, robot manipulation, and sequential part generation, showing our strength in realistic tasks with the demand for high physical plausibility.
1 code implementation • 7 Aug 2024 • William Yicheng Zhu, Keren Ye, Junjie Ke, Jiahui Yu, Leonidas Guibas, Peyman Milanfar, Feng Yang
Recognizing and disentangling visual attributes from objects is a foundation to many computer vision applications.
no code implementations • 19 Jul 2024 • Yang You, Mikaela Angelina Uy, Jiaqi Han, Rahul Thomas, Haotong Zhang, Suya You, Leonidas Guibas
Reverse engineering 3D computer-aided design (CAD) models from images is an important task for many downstream applications including interactive editing, manufacturing, architecture, robotics, etc.
1 code implementation • 18 Jul 2024 • Songlin Li, Despoina Paschalidou, Leonidas Guibas
PASTA comprises two main components: An autoregressive transformer that generates objects as a sequence of cuboidal primitives and a blending network, implemented with a transformer decoder that composes the sequences of cuboids and synthesizes high quality meshes for each object.
no code implementations • 18 Jul 2024 • Boyang Deng, Richard Tucker, Zhengqi Li, Leonidas Guibas, Noah Snavely, Gordon Wetzstein
To achieve this goal, we build on recent work on video diffusion, used within an autoregressive framework that can easily scale to long sequences.
1 code implementation • 5 Jul 2024 • Yuxuan Kuang, Junjie Ye, Haoran Geng, Jiageng Mao, Congyue Deng, Leonidas Guibas, He Wang, Yue Wang
First, RAM extracts unified affordance at scale from diverse sources of demonstrations including robotic data, human-object interaction (HOI) data, and custom data to construct a comprehensive affordance memory.
1 code implementation • 28 Jun 2024 • Sara Sabour, Lily Goli, George Kopanas, Mark Matthews, Dmitry Lagun, Leonidas Guibas, Alec Jacobson, David J. Fleet, Andrea Tagliasacchi
3D Gaussian Splatting (3DGS) is a promising technique for 3D reconstruction, offering efficient training and rendering speeds, making it suitable for real-time applications. However, current methods require highly controlled environments (no moving people or wind-blown elements, and consistent lighting) to meet the inter-view consistency assumption of 3DGS.
1 code implementation • 26 Jun 2024 • Colton Stearns, Adam Harley, Mikaela Uy, Florian Dubost, Federico Tombari, Gordon Wetzstein, Leonidas Guibas
We evaluate on the Nvidia Dynamic Scenes dataset and the DyCheck iPhone dataset, and show that Gaussian Marbles significantly outperforms other Gaussian baselines in quality, and is on-par with non-Gaussian representations, all while maintaining the efficiency, compositionality, editability, and tracking benefits of Gaussians.
no code implementations • 9 Jun 2024 • Yunchao Zhang, Guandao Yang, Leonidas Guibas, Yanchao Yang
In particular, it can be challenging to animate and move objects in the scene, which requires coordination among many Gaussians.
no code implementations • 27 May 2024 • Zhengfei Kuang, Shengqu Cai, Hao He, Yinghao Xu, Hongsheng Li, Leonidas Guibas, Gordon Wetzstein
Research on video generation has recently made tremendous progress, enabling high-quality videos to be generated from text prompts or images.
1 code implementation • 27 May 2024 • Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, Kostas Daniilidis
We introduce 4D Motion Scaffolds (MoSca), a modern 4D reconstruction system designed to reconstruct and synthesize novel views of dynamic scenes from monocular videos captured casually in the wild.
no code implementations • 26 Apr 2024 • IAn Huang, Guandao Yang, Leonidas Guibas
Specifically, we design a vision-based edit generator and state evaluator to work together to find the correct sequence of actions to achieve the goal.
no code implementations • CVPR 2024 • Nicolas Ugrinovic, Boxiao Pan, Georgios Pavlakos, Despoina Paschalidou, Bokui Shen, Jordi Sanchez-Riera, Francesc Moreno-Noguer, Leonidas Guibas
We introduce MultiPhys, a method designed for recovering multi-person motion from monocular videos.
1 code implementation • CVPR 2024 • Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Abhishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, Varun Jampani
Given that such models can classify, delineate, and localize objects in 2D, we ask whether they also represent their 3D structure?
no code implementations • 5 Apr 2024 • Yang Zheng, Qingqing Zhao, Guandao Yang, Wang Yifan, Donglai Xiang, Florian Dubost, Dmitry Lagun, Thabo Beeler, Federico Tombari, Leonidas Guibas, Gordon Wetzstein
This marks a significant advancement towards modeling photorealistic digital humans using physically based inverse rendering with physics in the loop.
1 code implementation • CVPR 2024 • Yijia Weng, Bowen Wen, Jonathan Tremblay, Valts Blukis, Dieter Fox, Leonidas Guibas, Stan Birchfield
We address the problem of building digital twins of unknown articulated objects from two RGBD scans of the object at different articulation states.
no code implementations • 18 Mar 2024 • Xinle Cheng, Congyue Deng, Adam Harley, Yixin Zhu, Leonidas Guibas
We demonstrate that our technique yields correspondences that are not only smoother but also more accurate, with the possibility of better reflecting the knowledge embedded in the large-scale vision models that we are studying.
1 code implementation • 18 Mar 2024 • Hansheng Chen, Ruoxi Shi, Yulin Liu, Bokui Shen, Jiayuan Gu, Gordon Wetzstein, Hao Su, Leonidas Guibas
Open-domain 3D object synthesis has been lagging behind image synthesis due to limited data and higher computational complexity.
no code implementations • 23 Feb 2024 • Francis Engelmann, Ayca Takmaz, Jonas Schult, Elisabetta Fedele, Johanna Wald, Songyou Peng, Xi Wang, Or Litany, Siyu Tang, Federico Tombari, Marc Pollefeys, Leonidas Guibas, Hongbo Tian, Chunjie Wang, Xiaosheng Yan, Bingwen Wang, Xuanyang Zhang, Xiao Liu, Phuc Nguyen, Khoi Nguyen, Anh Tran, Cuong Pham, Zhening Huang, Xiaoyang Wu, Xi Chen, Hengshuang Zhao, Lei Zhu, Joan Lasenby
This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023.
no code implementations • CVPR 2024 • Boyuan Chen, Zhuo Xu, Sean Kirmani, Brian Ichter, Danny Driess, Pete Florence, Dorsa Sadigh, Leonidas Guibas, Fei Xia
By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA.
1 code implementation • CVPR 2024 • Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, Gordon Wetzstein
These metrics lack the flexibility to generalize to different evaluation criteria and might not align well with human preferences.
1 code implementation • 5 Jan 2024 • Jiawei Yang, Katie Z Luo, Jiefeng Li, Congyue Deng, Leonidas Guibas, Dilip Krishnan, Kilian Q Weinberger, Yonglong Tian, Yue Wang
In the second stage, we train a lightweight transformer block to predict clean features from raw ViT outputs, leveraging the derived estimates of the clean features as supervision.
1 code implementation • 23 Dec 2023 • Yang You, Kai Xiong, Zhening Yang, Zhengxiang Huang, Junwei Zhou, Ruoxi Shi, Zhou Fang, Adam W. Harley, Leonidas Guibas, Cewu Lu
We introduce PACE (Pose Annotations in Cluttered Environments), a large-scale benchmark designed to advance the development and evaluation of pose estimation methods in cluttered scenarios.
no code implementations • CVPR 2024 • Ziyu Wan, Despoina Paschalidou, IAn Huang, Hongyu Liu, Bokui Shen, Xiaoyu Xiang, Jing Liao, Leonidas Guibas
The increased demand for 3D data in AR/VR, robotics and gaming applications, gave rise to powerful generative pipelines capable of synthesizing high-quality 3D objects.
no code implementations • 3 Dec 2023 • Haoran Geng, Songlin Wei, Congyue Deng, Bokui Shen, He Wang, Leonidas Guibas
More concretely, given an articulated object, we first observe all the semantic parts on it, conditioned on which an instruction interpreter proposes possible action programs that concretize the natural language instruction.
1 code implementation • CVPR 2024 • Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, David B. Lindell
Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.
no code implementations • 28 Nov 2023 • Congyue Deng, Jiawei Yang, Leonidas Guibas, Yue Wang
To that end, we introduce a modification to the NeRF rendering equation which is as simple as a few lines of code change for any NeRF variations, while greatly improving the rendering quality of view-dependent effects.
no code implementations • 5 Nov 2023 • Yang You, Bokui Shen, Congyue Deng, Haoran Geng, Songlin Wei, He Wang, Leonidas Guibas
Remarkably, our model demonstrates robust generalization capabilities to novel and previously unencountered complex tasks without any preliminary demonstrations.
no code implementations • NeurIPS 2023 • Mikaela Angelina Uy, Kiyohiro Nakayama, Guandao Yang, Rahul Krishna Thomas, Leonidas Guibas, Ke Li
Volume rendering requires evaluating an integral along each ray, which is numerically approximated with a finite sum that corresponds to the exact integral along the ray under piecewise constant volume density.
no code implementations • 25 Oct 2023 • Qianxu Wang, Haotong Zhang, Congyue Deng, Yang You, Hao Dong, Yixin Zhu, Leonidas Guibas
Central to SparseDFF is a feature refinement network, optimized with a contrastive loss between views and a point-pruning mechanism for feature continuity.
no code implementations • 10 Oct 2023 • Wen-Hsuan Chu, Adam W. Harley, Pavel Tokmakov, Achal Dave, Leonidas Guibas, Katerina Fragkiadaki
This begs the question: can we re-purpose these large-scale pre-trained static image models for open-vocabulary video tracking?
1 code implementation • 7 Sep 2023 • Nikhil Raghuraman, Adam W. Harley, Leonidas Guibas
Current machine learning methods struggle to solve Bongard problems, which are a type of IQ test that requires deriving an abstract "concept" from a set of positive and negative "support" images, and then classifying whether or not a new query image depicts the key concept.
Ranked #2 on
Few-Shot Image Classification
on Bongard-HOI
(using extra training data)
no code implementations • CVPR 2024 • Nilesh Kulkarni, Davis Rempe, Kyle Genova, Abhijit Kundu, Justin Johnson, David Fouhey, Leonidas Guibas
This interaction field guides the sampling of an object-conditioned human motion diffusion model, so as to encourage plausible contacts and affordance semantics.
1 code implementation • 9 Jun 2023 • IAn Huang, Vrishab Krishna, Omoruyi Atekha, Leonidas Guibas
What constitutes the "vibe" of a particular scene?
1 code implementation • 25 May 2023 • Jiahui Lei, Congyue Deng, Bokui Shen, Leonidas Guibas, Kostas Daniilidis
We propose Neural 3D Articulation Prior (NAP), the first 3D deep generative model to synthesize 3D articulated object models.
no code implementations • 4 May 2023 • Connor Z. Lin, Koki Nagano, Jan Kautz, Eric R. Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, Sameh Khamis
To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing.
no code implementations • CVPR 2023 • Bokui Shen, Xinchen Yan, Charles R. Qi, Mahyar Najibi, Boyang Deng, Leonidas Guibas, Yin Zhou, Dragomir Anguelov
Modeling the 3D world from sensor data for simulation is a scalable way of developing testing and validation environments for robotic learning problems such as autonomous driving.
1 code implementation • 4 Apr 2023 • Ivica Kičić, Pantelis R. Vlachas, Georgios Arampatzis, Michail Chatzimanolakis, Leonidas Guibas, Petros Koumoutsakos
To the best of our knowledge, AdaLED is the first framework that couples a surrogate model with a computational solver to achieve online adaptive learning of effective dynamics.
1 code implementation • CVPR 2023 • Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas
We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.
1 code implementation • CVPR 2023 • Bingfan Zhu, Yanchao Yang, Xulong Wang, Youyi Zheng, Leonidas Guibas
We propose VDN-NeRF, a method to train neural radiance fields (NeRFs) for better geometry under non-Lambertian surface and dynamic lighting conditions that cause significant variation in the radiance of a point when viewed from different angles.
no code implementations • CVPR 2023 • Jiahui Lei, Congyue Deng, Karl Schmeckpeper, Leonidas Guibas, Kostas Daniilidis
First, we introduce equivariant shape representations to this problem to eliminate the complexity induced by the variation in object configuration.
no code implementations • CVPR 2023 • Mikaela Angelina Uy, Ricardo Martin-Brualla, Leonidas Guibas, Ke Li
To address this issue, we introduce SCADE, a novel technique that improves NeRF reconstruction quality on sparse, unconstrained input views for in-the-wild indoor scenes.
no code implementations • ICCV 2023 • Sherwin Bahmani, Jeong Joon Park, Despoina Paschalidou, Xingguang Yan, Gordon Wetzstein, Leonidas Guibas, Andrea Tagliasacchi
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.
no code implementations • 16 Mar 2023 • Konstantinos Tertikas, Despoina Paschalidou, Boxiao Pan, Jeong Joon Park, Mikaela Angelina Uy, Ioannis Emiris, Yannis Avrithis, Leonidas Guibas
Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.
no code implementations • CVPR 2024 • Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas
A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.
no code implementations • CVPR 2023 • Xiaoshuai Zhang, Abhijit Kundu, Thomas Funkhouser, Leonidas Guibas, Hao Su, Kyle Genova
We address efficient and structure-aware 3D scene representation from images.
no code implementations • CVPR 2023 • Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard, Srinath Sridhar, Leonidas Guibas
Humans universally dislike the task of cleaning up a messy room.
no code implementations • 12 Jan 2023 • Nikolaos Zygouras, Nikolaos Panagiotou, Yang Li, Dimitrios Gunopulos, Leonidas Guibas
Travel time estimation is a critical task, useful to many urban applications at the individual citizen and the stakeholder level.
1 code implementation • CVPR 2023 • Panos Achlioptas, IAn Huang, Minhyuk Sung, Sergey Tulyakov, Leonidas Guibas
In this work, we aim to facilitate the task of editing the geometry of 3D models through the use of natural language.
1 code implementation • CVPR 2023 • Konstantinos Tertikas, Despoina Paschalidou, Boxiao Pan, Jeong Joon Park, Mikaela Angelina Uy, Ioannis Emiris, Yannis Avrithis, Leonidas Guibas
Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.
no code implementations • 20 Dec 2022 • Yajie Bao, Yang Li, Shao-Lun Huang, Lin Zhang, Lizhong Zheng, Amir Zamir, Leonidas Guibas
Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks.
no code implementations • CVPR 2023 • Arjun R. Akula, Brendan Driscoll, Pradyumna Narayana, Soravit Changpinyo, Zhiwei Jia, Suyash Damle, Garima Pruthi, Sugato Basu, Leonidas Guibas, William T. Freeman, Yuanzhen Li, Varun Jampani
Towards this goal, we introduce MetaCLUE, a set of vision tasks on visual metaphor.
1 code implementation • 9 Dec 2022 • IAn Huang, Panos Achlioptas, Tianyi Zhang, Sergey Tulyakov, Minhyuk Sung, Leonidas Guibas
Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision.
no code implementations • CVPR 2023 • Zhen Wang, Shijie Zhou, Jeong Joon Park, Despoina Paschalidou, Suya You, Gordon Wetzstein, Leonidas Guibas, Achuta Kadambi
One school of thought is to encode a latent vector for each point (point latents).
1 code implementation • CVPR 2023 • Congyue Deng, Chiyu "Max'' Jiang, Charles R. Qi, Xinchen Yan, Yin Zhou, Leonidas Guibas, Dragomir Anguelov
Formulating single-view reconstruction as an image-conditioned 3D generation problem, we optimize the NeRF representations by minimizing a diffusion loss on its arbitrary view renderings with a pretrained image diffusion model under the input-view constraint.
no code implementations • CVPR 2023 • Minjung Son, Jeong Joon Park, Leonidas Guibas, Gordon Wetzstein
Generative models have shown great promise in synthesizing photorealistic 3D objects, but they require large amounts of training data.
no code implementations • 29 Oct 2022 • Sidhika Balachandar, Adrien Poulenard, Congyue Deng, Leonidas Guibas
We present OAVNN: Orientation Aware Vector Neuron Network, an extension of the Vector Neuron Network.
no code implementations • CVPR 2023 • Panos Achlioptas, Maks Ovsjanikov, Leonidas Guibas, Sergey Tulyakov
To embark on this journey, we introduce and share with the research community a large-scale dataset that contains emotional reactions and free-form textual explanations for 85, 007 publicly available images, analyzed by 6, 283 annotators who were asked to indicate and explain how and why they felt in a particular way when observing a specific image, producing a total of 526, 749 responses.
no code implementations • 18 Jul 2022 • Connor Z. Lin, Niloy J. Mitra, Gordon Wetzstein, Leonidas Guibas, Paul Guerrero
Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis.
1 code implementation • 29 Jun 2022 • Sherwin Bahmani, Jeong Joon Park, Despoina Paschalidou, Hao Tang, Gordon Wetzstein, Leonidas Guibas, Luc van Gool, Radu Timofte
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
no code implementations • CVPR 2022 • Abhijit Kundu, Kyle Genova, Xiaoqi Yin, Alireza Fathi, Caroline Pantofaru, Leonidas Guibas, Andrea Tagliasacchi, Frank Dellaert, Thomas Funkhouser
Our model builds a panoptic radiance field representation of any scene from just color images.
1 code implementation • 28 Jan 2022 • Jiahan Li, Shitong Luo, Congyue Deng, Chaoran Cheng, Jiaqi Guan, Leonidas Guibas, Jian Peng, Jianzhu Ma
In this work, we propose the Orientation-Aware Graph Neural Networks (OAGNNs) to better sense the geometric characteristics in protein structure (e. g. inner-residue torsion angles, inter-residue orientations).
2 code implementations • CVPR 2022 • Yuefan Shen, Yanchao Yang, Mi Yan, He Wang, Youyi Zheng, Leonidas Guibas
Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.
no code implementations • CVPR 2022 • Mikaela Angelina Uy, Yen-Yu Chang, Minhyuk Sung, Purvi Goel, Joseph Lambourne, Tolga Birdal, Leonidas Guibas
We propose Point2Cyl, a supervised network transforming a raw 3D point cloud to a set of extrusion cylinders.
Ranked #3 on
CAD Reconstruction
on Fusion 360 Gallery
(IoU metric)
2 code implementations • CVPR 2022 • Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.
no code implementations • ICLR 2022 • Chuanyu Pan, Yanchao Yang, Kaichun Mo, Yueqi Duan, Leonidas Guibas
We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.
2 code implementations • CVPR 2022 • Juil Koo, IAn Huang, Panos Achlioptas, Leonidas Guibas, Minhyuk Sung
We introduce PartGlot, a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language.
no code implementations • ICLR 2022 • Qi Li, Kaichun Mo, Yanchao Yang, Hang Zhao, Leonidas Guibas
While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).
no code implementations • 1 Dec 2021 • Yian Wang, Ruihai Wu, Kaichun Mo, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, Hao Dong
Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.
2 code implementations • NeurIPS 2021 • Tolga Birdal, Aaron Lou, Leonidas Guibas, Umut Şimşekli
Disobeying the classical wisdom of statistical learning theory, modern deep neural networks generalize well even though they typically contain millions of parameters.
no code implementations • NeurIPS 2021 • Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang
Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.
1 code implementation • CVPR 2022 • Jiayi Chen, Yingda Yin, Tolga Birdal, Baoquan Chen, Leonidas Guibas, He Wang
Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem.
1 code implementation • 29 Jul 2021 • Yanchao Yang, Hanxiang Ren, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas
Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.
2 code implementations • 27 Jul 2021 • Yuefan Shen, Yanchao Yang, Youyi Zheng, C. Karen Liu, Leonidas Guibas
We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth.
1 code implementation • 29 Jun 2021 • Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, Leonidas Guibas
Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.
no code implementations • ICLR 2022 • Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, Hao Dong
In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.
no code implementations • NeurIPS 2021 • Wamiq Reyaz Para, Shariq Farooq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas Guibas, Peter Wonka
Sketches can be represented as graphs, with the primitives as nodes and the constraints as edges.
no code implementations • NeurIPS 2021 • Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang
To reduce the huge amount of pose annotations needed for category-level learning, we propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.
no code implementations • 17 May 2021 • Ge Zhang, Or Litany, Srinath Sridhar, Leonidas Guibas
We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images.
4 code implementations • ICCV 2021 • Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas
Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds.
1 code implementation • 31 Mar 2021 • Luca Moschella, Simone Melzi, Luca Cosmo, Filippo Maggioli, Or Litany, Maks Ovsjanikov, Leonidas Guibas, Emanuele Rodolà
Spectral geometric methods have brought revolutionary changes to the field of geometry processing.
1 code implementation • CVPR 2021 • Mikaela Angelina Uy, Vladimir G. Kim, Minhyuk Sung, Noam Aigerman, Siddhartha Chaudhuri, Leonidas Guibas
In fact, we use the embedding space to guide the shape pairs used to train the deformation module, so that it invests its capacity in learning deformations between meaningful shape pairs.
5 code implementations • CVPR 2021 • Panos Achlioptas, Maks Ovsjanikov, Kilichbek Haydarov, Mohamed Elhoseiny, Leonidas Guibas
We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language.
no code implementations • CVPR 2021 • Tolga Birdal, Vladislav Golyanik, Christian Theobalt, Leonidas Guibas
We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.
1 code implementation • CVPR 2021 • Jiahui Huang, He Wang, Tolga Birdal, Minhyuk Sung, Federica Arrigoni, Shi-Min Hu, Leonidas Guibas
We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds.
1 code implementation • ICCV 2021 • Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani
One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment.
1 code implementation • 20 Dec 2020 • Haowen Deng, Mai Bui, Nassir Navab, Leonidas Guibas, Slobodan Ilic, Tolga Birdal
For the former we contributed our own dataset composed of five indoor scenes where it is unavoidable to capture images corresponding to views that are hard to uniquely identify.
1 code implementation • 8 Dec 2020 • Qihang Fang, Yingda Yin, Qingnan Fan, Fei Xia, Siyan Dong, Sheng Wang, Jue Wang, Leonidas Guibas, Baoquan Chen
These approaches localize the camera in the discrete pose space and are agnostic to the localization-driven scene property, which restricts the camera pose accuracy in the coarse scale.
1 code implementation • CVPR 2021 • Siyan Dong, Qingnan Fan, He Wang, Ji Shi, Li Yi, Thomas Funkhouser, Baoquan Chen, Leonidas Guibas
Localizing the camera in a known indoor environment is a key building block for scene mapping, robot navigation, AR, etc.
1 code implementation • ICCV 2021 • Wamiq Para, Paul Guerrero, Tom Kelly, Leonidas Guibas, Peter Wonka
We generate layouts in three steps.
3 code implementations • NeurIPS 2020 • Jialei Huang, Guanqi Zhan, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas Guibas, Hao Dong
Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.
1 code implementation • 14 Jun 2020 • Chiyu "Max" Jiang, Jingwei Huang, Andrea Tagliasacchi, Leonidas Guibas
We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
1 code implementation • 7 Jun 2020 • Amir Zamir, Alexander Sax, Teresa Yeo, Oğuzhan Kar, Nikhil Cheerla, Rohan Suri, Zhangjie Cao, Jitendra Malik, Leonidas Guibas
Visual perception entails solving a wide set of tasks, e. g., object detection, depth estimation, etc.
1 code implementation • 23 May 2020 • Jingwei Huang, Chiyu Max Jiang, Baiqiang Leng, Bin Wang, Leonidas Guibas
Given a pair of shapes, our framework provides a novel shape feature-preserving mapping function that continuously deforms one model to the other by minimizing fitting and rigidity losses based on the non-rigid iterative-closest-point (ICP) algorithm.
Graphics Computational Geometry
2 code implementations • ECCV 2020 • Mai Bui, Tolga Birdal, Haowen Deng, Shadi Albarqouni, Leonidas Guibas, Slobodan Ilic, Nassir Navab
We present a multimodal camera relocalization framework that captures ambiguities and uncertainties with continuous mixture models defined on the manifold of camera poses.
1 code implementation • ECCV 2020 • Mikaela Angelina Uy, Jingwei Huang, Minhyuk Sung, Tolga Birdal, Leonidas Guibas
We introduce a new problem of retrieving 3D models that are deformable to a given query shape and present a novel deep deformation-aware embedding to solve this retrieval task.
no code implementations • CVPR 2020 • Tolga Birdal, Michael Arbel, Umut Şimşekli, Leonidas Guibas
We introduce a new paradigm, $\textit{measure synchronization}$, for synchronizing graphs with measure-valued edges.
1 code implementation • ECCV 2020 • Yichen Li, Kaichun Mo, Lin Shao, Minhyuk Sung, Leonidas Guibas
Autonomous assembly is a crucial capability for robots in many applications.
1 code implementation • CVPR 2020 • Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu, Chiyu Max Jiang, Leonidas Guibas, Matthias Nießner, Thomas Funkhouser
In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.
no code implementations • 26 Feb 2020 • Or Litany, Ari Morcos, Srinath Sridhar, Leonidas Guibas, Judy Hoffman
We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision.
no code implementations • 6 Feb 2020 • Zhangsihao Yang, Or Litany, Tolga Birdal, Srinath Sridhar, Leonidas Guibas
In this work, we wish to challenge this practice and use a neural network to learn descriptors directly from the raw mesh.
1 code implementation • 27 Jan 2020 • Oshri Halimi, Ido Imanuel, Or Litany, Giovanni Trappolini, Emanuele Rodolà, Leonidas Guibas, Ron Kimmel
Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.
2 code implementations • 21 Jan 2020 • Christiane Sommer, Yumin Sun, Leonidas Guibas, Daniel Cremers, Tolga Birdal
We propose a new method for segmentation-free joint estimation of orthogonal planes, their intersection lines, relationship graph and corners lying at the intersection of three orthogonal planes.
2 code implementations • ECCV 2020 • Jeffrey O. Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, Jitendra Malik
When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.
2 code implementations • ECCV 2020 • Yongheng Zhao, Tolga Birdal, Jan Eric Lenssen, Emanuele Menegatti, Leonidas Guibas, Federico Tombari
We present a 3D capsule module for processing point clouds that is equivariant to 3D rotations and translations, as well as invariant to permutations of the input points.
2 code implementations • CVPR 2020 • Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song
We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space.
1 code implementation • 23 Dec 2019 • Alexander Sax, Jeffrey O. Zhang, Bradley Emi, Amir Zamir, Silvio Savarese, Leonidas Guibas, Jitendra Malik
How much does having visual priors about the world (e. g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e. g. navigating a complex environment)?
1 code implementation • ICCV 2019 • Manuel Dahnert, Angela Dai, Leonidas Guibas, Matthias Nießner
We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.
1 code implementation • ICML 2020 • Trevor Standley, Amir R. Zamir, Dawn Chen, Leonidas Guibas, Jitendra Malik, Silvio Savarese
Many computer vision applications require solving multiple tasks in real-time.
1 code implementation • ICCV 2019 • Ruqi Huang, Marie-Julie Rakotosaona, Panos Achlioptas, Leonidas Guibas, Maks Ovsjanikov
This paper proposes a learning-based framework for reconstructing 3D shapes from functional operators, compactly encoded as small-sized matrices.
1 code implementation • ICCV 2019 • Jingwei Huang, Yichao Zhou, Thomas Funkhouser, Leonidas Guibas
In this work, we introduce the novel problem of identifying dense canonical 3D coordinate frames from a single RGB image.
no code implementations • CVPR 2020 • Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Li Yi, Leonidas Guibas, Hao Zhang
While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaCoSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user-specified) K parts.
1 code implementation • CVPR 2019 • Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas Guibas, Qi-Xing Huang
Our approach alternates between transformation synchronization using weighted relative transformations and predicting new weights of the input relative transformations using a neural network.
no code implementations • ICCV 2019 • Anastasia Dubrovina, Fei Xia, Panos Achlioptas, Mira Shalah, Raphael Groscot, Leonidas Guibas
We present a novel neural network architecture, termed Decomposer-Composer, for semantic structure-aware 3D shape modeling.
1 code implementation • 31 Dec 2018 • Alexander Sax, Bradley Emi, Amir R. Zamir, Leonidas Guibas, Silvio Savarese, Jitendra Malik
This skill set (hereafter mid-level perception) provides the policy with a more processed state of the world compared to raw images.
1 code implementation • CVPR 2019 • Li Yi, Wang Zhao, He Wang, Minhyuk Sung, Leonidas Guibas
We introduce a novel 3D object proposal approach named Generative Shape Proposal Network (GSPN) for instance segmentation in point cloud data.
Ranked #28 on
3D Object Detection
on ScanNetV2
1 code implementation • CVPR 2019 • Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas Guibas
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).
Ranked #23 on
Semantic Segmentation
on ScanNet
(test mIoU metric)
2 code implementations • CVPR 2019 • Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, Leonidas Guibas
Fitting geometric primitives to 3D point cloud data bridges a gap between low-level digitized 3D data and high-level structural information on the underlying 3D shapes.
1 code implementation • 19 Sep 2018 • Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, Leonidas Guibas
In this paper, we explore how the observation of different articulation states provides evidence for part structure and motion of 3D objects.
1 code implementation • 4 Jul 2018 • Minhyuk Sung, Anastasia Dubrovina, Vladimir G. Kim, Leonidas Guibas
Modeling relations between components of 3D objects is essential for many geometry editing tasks.
Graphics I.3.5
1 code implementation • ICLR 2019 • Jan Svoboda, Jonathan Masci, Federico Monti, Michael M. Bronstein, Leonidas Guibas
Deep learning systems have become ubiquitous in many aspects of our lives.
1 code implementation • NeurIPS 2018 • Minhyuk Sung, Hao Su, Ronald Yu, Leonidas Guibas
Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes.
1 code implementation • CVPR 2018 • Amir Zamir, Alexander Sax, William Shen, Leonidas Guibas, Jitendra Malik, Silvio Savarese
The product is a computational taxonomic map for task transfer learning.
no code implementations • CVPR 2013 • Nan Hu, Raif M. Rustamov, Leonidas Guibas
In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes.
2 code implementations • 5 Feb 2018 • Jingwei Huang, Hao Su, Leonidas Guibas
In this paper, we describe a robust algorithm for 2-Manifold generation of various kinds of ShapeNet Models.
Computational Geometry
1 code implementation • 17 Oct 2017 • Li Yi, Lin Shao, Manolis Savva, Haibin Huang, Yang Zhou, Qirui Wang, Benjamin Graham, Martin Engelcke, Roman Klokov, Victor Lempitsky, Yuan Gan, Pengyu Wang, Kun Liu, Fenggen Yu, Panpan Shui, Bingyang Hu, Yan Zhang, Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Minki Jeong, Jaehoon Choi, Changick Kim, Angom Geetchandra, Narasimha Murthy, Bhargava Ramu, Bharadwaj Manda, M. Ramanathan, Gautam Kumar, P Preetham, Siddharth Srivastava, Swati Bhugra, Brejesh lall, Christian Haene, Shubham Tulsiani, Jitendra Malik, Jared Lafer, Ramsey Jones, Siyuan Li, Jie Lu, Shi Jin, Jingyi Yu, Qi-Xing Huang, Evangelos Kalogerakis, Silvio Savarese, Pat Hanrahan, Thomas Funkhouser, Hao Su, Leonidas Guibas
We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database.
1 code implementation • 6 Aug 2017 • Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas
The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input.
Graphics I.3.5
3 code implementations • ICML 2018 • Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, Leonidas Guibas
Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling.
no code implementations • 5 May 2017 • Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, Leonidas Guibas
We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures.
no code implementations • CVPR 2018 • Cewu Lu, Hao Su, Yongyi Lu, Li Yi, Chi-Keung Tang, Leonidas Guibas
Important high-level vision tasks such as human-object interaction, image captioning and robotic manipulation require rich semantic descriptions of objects at part level.
no code implementations • CVPR 2017 • Li Yi, Hao Su, Xingwen Guo, Leonidas Guibas
To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases.
Ranked #57 on
3D Part Segmentation
on ShapeNet-Part
4 code implementations • CVPR 2017 • Haoqiang Fan, Hao Su, Leonidas Guibas
Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image.
Ranked #2 on
3D Reconstruction
on Data3D−R2N2
(using extra training data)
3D Object Reconstruction From A Single Image
3D Reconstruction
no code implementations • CVPR 2018 • Nan Hu, Qi-Xing Huang, Boris Thibert, Leonidas Guibas
In this paper we propose an optimization-based framework to multiple object matching.
16 code implementations • 9 Dec 2015 • Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qi-Xing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, Fisher Yu
We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects.