Search Results for author: Leonidas Guibas

Found 124 papers, 67 papers with code

NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors

1 code implementation CVPR 2023 Congyue Deng, Chiyu "Max'' Jiang, Charles R. Qi, Xinchen Yan, Yin Zhou, Leonidas Guibas, Dragomir Anguelov

Formulating single-view reconstruction as an image-conditioned 3D generation problem, we optimize the NeRF representations by minimizing a diffusion loss on its arbitrary view renderings with a pretrained image diffusion model under the input-view constraint.

3D Reconstruction

Robust Watertight Manifold Surface Generation Method for ShapeNet Models

2 code implementations5 Feb 2018 Jingwei Huang, Hao Su, Leonidas Guibas

In this paper, we describe a robust algorithm for 2-Manifold generation of various kinds of ShapeNet Models.

Computational Geometry

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

4 code implementations CVPR 2017 Haoqiang Fan, Hao Su, Leonidas Guibas

Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image.

Ranked #2 on 3D Reconstruction on Data3D−R2N2 (using extra training data)

3D Object Reconstruction From A Single Image 3D Reconstruction

Learning Representations and Generative Models for 3D Point Clouds

3 code implementations ICML 2018 Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, Leonidas Guibas

Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling.

Representation Learning

Vector Neurons: A General Framework for SO(3)-Equivariant Networks

4 code implementations ICCV 2021 Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, Leonidas Guibas

Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds.

ArtEmis: Affective Language for Visual Art

3 code implementations CVPR 2021 Panos Achlioptas, Maks Ovsjanikov, Kilichbek Haydarov, Mohamed Elhoseiny, Leonidas Guibas

We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language.

Deep Knowledge Tracing

6 code implementations NeurIPS 2015 Chris Piech, Jonathan Spencer, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, Jascha Sohl-Dickstein

Knowledge tracing---where a machine models the knowledge of a student as they interact with coursework---is a well established problem in computer supported education.

Knowledge Tracing

Adversarial Texture Optimization from RGB-D Scans

1 code implementation CVPR 2020 Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu, Chiyu Max Jiang, Leonidas Guibas, Matthias Nießner, Thomas Funkhouser

In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.

Surface Reconstruction Texture Synthesis

Supervised Fitting of Geometric Primitives to 3D Point Clouds

2 code implementations CVPR 2019 Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, Leonidas Guibas

Fitting geometric primitives to 3D point cloud data bridges a gap between low-level digitized 3D data and high-level structural information on the underlying 3D shapes.

Shape Representation Of 3D Point Clouds

GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation

1 code implementation8 Jan 2024 Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, Gordon Wetzstein

These metrics lack the flexibility to generalize to different evaluation criteria and might not align well with human preferences.

Text to 3D

Generative 3D Part Assembly via Dynamic Graph Learning

3 code implementations NeurIPS 2020 Jialei Huang, Guanqi Zhan, Qingnan Fan, Kaichun Mo, Lin Shao, Baoquan Chen, Leonidas Guibas, Hao Dong

Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.

Graph Learning Pose Estimation +1

Category-Level Articulated Object Pose Estimation

2 code implementations CVPR 2020 Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song

We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space.

Object Pose Estimation

Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies

1 code implementation31 Dec 2018 Alexander Sax, Bradley Emi, Amir R. Zamir, Leonidas Guibas, Silvio Savarese, Jitendra Malik

This skill set (hereafter mid-level perception) provides the policy with a more processed state of the world compared to raw images.

Object Detection

Learning to Navigate Using Mid-Level Visual Priors

1 code implementation23 Dec 2019 Alexander Sax, Jeffrey O. Zhang, Bradley Emi, Amir Zamir, Silvio Savarese, Leonidas Guibas, Jitendra Malik

How much does having visual priors about the world (e. g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e. g. navigating a complex environment)?

Navigate reinforcement-learning +2

ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes

1 code implementation ECCV 2020 Panos Achlioptas, Ahmed Abdelreheem, Fei Xia, Mohamed Elhoseiny, Leonidas Guibas

Due to the scarcity and unsuitability of existent 3D-oriented linguistic resources for this task, we first develop two large-scale and complementary visio-linguistic datasets: i) extbf{ extit{Sr3D}}, which contains 83. 5K template-based utterances leveraging extit{spatial relations} with other fine-grained object classes to localize a referred object in a given scene, and ii) extbf{ extit{Nr3D}} which contains 41. 5K extit{natural, free-form}, utterances collected by deploying a 2-player object reference game in 3D scenes.

Object

TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes

1 code implementation CVPR 2019 Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Nießner, Leonidas Guibas

We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).

3D Semantic Segmentation

Where2Act: From Pixels to Actions for Articulated 3D Objects

1 code implementation ICCV 2021 Kaichun Mo, Leonidas Guibas, Mustafa Mukadam, Abhinav Gupta, Shubham Tulsiani

One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment.

MeshODE: A Robust and Scalable Framework for Mesh Deformation

1 code implementation23 May 2020 Jingwei Huang, Chiyu Max Jiang, Baiqiang Leng, Bin Wang, Leonidas Guibas

Given a pair of shapes, our framework provides a novel shape feature-preserving mapping function that continuously deforms one model to the other by minimizing fitting and rigidity losses based on the non-rigid iterative-closest-point (ICP) algorithm.

Graphics Computational Geometry

Generic 3D Diffusion Adapter Using Controlled Multi-View Editing

1 code implementation18 Mar 2024 Hansheng Chen, Ruoxi Shi, Yulin Liu, Bokui Shen, Jiayuan Gu, Gordon Wetzstein, Hao Su, Leonidas Guibas

Open-domain 3D object synthesis has been lagging behind image synthesis due to limited data and higher computational complexity.

Image Generation Image to 3D +2

From Planes to Corners: Multi-Purpose Primitive Detection in Unorganized 3D Point Clouds

2 code implementations21 Jan 2020 Christiane Sommer, Yumin Sun, Leonidas Guibas, Daniel Cremers, Tolga Birdal

We propose a new method for segmentation-free joint estimation of orthogonal planes, their intersection lines, relationship graph and corners lying at the intersection of three orthogonal planes.

3D-Aware Video Generation

1 code implementation29 Jun 2022 Sherwin Bahmani, Jeong Joon Park, Despoina Paschalidou, Hao Tang, Gordon Wetzstein, Leonidas Guibas, Luc van Gool, Radu Timofte

Generative models have emerged as an essential building block for many image synthesis and editing tasks.

Image Generation Video Generation

Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks

2 code implementations ECCV 2020 Jeffrey O. Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, Jitendra Malik

When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.

Imitation Learning Incremental Learning +3

ShapeFlow: Learnable Deformations Among 3D Shapes

1 code implementation14 Jun 2020 Chiyu "Max" Jiang, Jingwei Huang, Andrea Tagliasacchi, Leonidas Guibas

We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.

Disentanglement Style Transfer

Joint Embedding of 3D Scan and CAD Objects

1 code implementation ICCV 2019 Manuel Dahnert, Angela Dai, Leonidas Guibas, Matthias Nießner

We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.

Retrieval

MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization

1 code implementation CVPR 2021 Jiahui Huang, He Wang, Tolga Birdal, Minhyuk Sung, Federica Arrigoni, Shi-Min Hu, Leonidas Guibas

We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds.

Motion Estimation Motion Segmentation +1

Deep Part Induction from Articulated Object Pairs

1 code implementation19 Sep 2018 Li Yi, Haibin Huang, Difan Liu, Evangelos Kalogerakis, Hao Su, Leonidas Guibas

In this paper, we explore how the observation of different articulation states provides evidence for part structure and motion of 3D objects.

Object

Deformation-Aware 3D Model Embedding and Retrieval

1 code implementation ECCV 2020 Mikaela Angelina Uy, Jingwei Huang, Minhyuk Sung, Tolga Birdal, Leonidas Guibas

We introduce a new problem of retrieving 3D models that are deformable to a given query shape and present a novel deep deformation-aware embedding to solve this retrieval task.

3D Object Reconstruction Metric Learning +1

Joint Learning of 3D Shape Retrieval and Deformation

1 code implementation CVPR 2021 Mikaela Angelina Uy, Vladimir G. Kim, Minhyuk Sung, Noam Aigerman, Siddhartha Chaudhuri, Leonidas Guibas

In fact, we use the embedding space to guide the shape pairs used to train the deformation module, so that it invests its capacity in learning deformations between meaningful shape pairs.

3D Shape Retrieval Retrieval

JacobiNeRF: NeRF Shaping with Mutual Information Gradients

1 code implementation CVPR 2023 Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas

We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.

Instance Segmentation Semantic Segmentation

Learning Transformation Synchronization

1 code implementation CVPR 2019 Xiangru Huang, Zhenxiao Liang, Xiaowei Zhou, Yao Xie, Leonidas Guibas, Qi-Xing Huang

Our approach alternates between transformation synchronization using weighted relative transformations and predicting new weights of the input relative transformations using a neural network.

Quaternion Equivariant Capsule Networks for 3D Point Clouds

2 code implementations ECCV 2020 Yongheng Zhao, Tolga Birdal, Jan Eric Lenssen, Emanuele Menegatti, Leonidas Guibas, Federico Tombari

We present a 3D capsule module for processing point clouds that is equivariant to 3D rotations and translations, as well as invariant to permutations of the input points.

Pose Estimation

PartGlot: Learning Shape Part Segmentation from Language Reference Games

2 code implementations CVPR 2022 Juil Koo, IAn Huang, Panos Achlioptas, Leonidas Guibas, Minhyuk Sung

We introduce PartGlot, a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language.

LADIS: Language Disentanglement for 3D Shape Editing

1 code implementation9 Dec 2022 IAn Huang, Panos Achlioptas, Tianyi Zhang, Sergey Tulyakov, Minhyuk Sung, Leonidas Guibas

Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision.

Disentanglement

Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions

1 code implementation NeurIPS 2018 Minhyuk Sung, Hao Su, Ronald Yu, Leonidas Guibas

Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes.

Segmentation

Generating Part-Aware Editable 3D Shapes Without 3D Supervision

1 code implementation CVPR 2023 Konstantinos Tertikas, Despoina Paschalidou, Boxiao Pan, Jeong Joon Park, Mikaela Angelina Uy, Ioannis Emiris, Yannis Avrithis, Leonidas Guibas

Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.

O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning

1 code implementation29 Jun 2021 Kaichun Mo, Yuzhe Qin, Fanbo Xiang, Hao Su, Leonidas Guibas

Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.

Object

Towards Accurate Active Camera Localization

1 code implementation8 Dec 2020 Qihang Fang, Yingda Yin, Qingnan Fan, Fei Xia, Siyan Dong, Sheng Wang, Jue Wang, Leonidas Guibas, Baoquan Chen

These approaches localize the camera in the discrete pose space and are agnostic to the localization-driven scene property, which restricts the camera pose accuracy in the coarse scale.

Camera Localization Pose Estimation +1

ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling

1 code implementation6 Aug 2017 Minhyuk Sung, Hao Su, Vladimir G. Kim, Siddhartha Chaudhuri, Leonidas Guibas

The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input.

Graphics I.3.5

6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference

2 code implementations ECCV 2020 Mai Bui, Tolga Birdal, Haowen Deng, Shadi Albarqouni, Leonidas Guibas, Slobodan Ilic, Nassir Navab

We present a multimodal camera relocalization framework that captures ambiguities and uncertainties with continuous mixture models defined on the manifold of camera poses.

Camera Relocalization

Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks

2 code implementations NeurIPS 2021 Tolga Birdal, Aaron Lou, Leonidas Guibas, Umut Şimşekli

Disobeying the classical wisdom of statistical learning theory, modern deep neural networks generalize well even though they typically contain millions of parameters.

Learning Theory Topological Data Analysis

VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalization

1 code implementation CVPR 2023 Bingfan Zhu, Yanchao Yang, Xulong Wang, Youyi Zheng, Leonidas Guibas

We propose VDN-NeRF, a method to train neural radiance fields (NeRFs) for better geometry under non-Lambertian surface and dynamic lighting conditions that cause significant variation in the radiance of a point when viewed from different angles.

ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation

1 code implementation29 Jul 2021 Yanchao Yang, Hanxiang Ren, He Wang, Bokui Shen, Qingnan Fan, Youyi Zheng, C. Karen Liu, Leonidas Guibas

Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.

Hallucination Inductive Bias +2

DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis

2 code implementations27 Jul 2021 Yuefan Shen, Yanchao Yang, Youyi Zheng, C. Karen Liu, Leonidas Guibas

We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth.

Contrastive Learning Image Generation

OperatorNet: Recovering 3D Shapes From Difference Operators

1 code implementation ICCV 2019 Ruqi Huang, Marie-Julie Rakotosaona, Panos Achlioptas, Leonidas Guibas, Maks Ovsjanikov

This paper proposes a learning-based framework for reconstructing 3D shapes from functional operators, compactly encoded as small-sized matrices.

The Whole Is Greater Than the Sum of Its Nonrigid Parts

1 code implementation27 Jan 2020 Oshri Halimi, Ido Imanuel, Or Litany, Giovanni Trappolini, Emanuele Rodolà, Leonidas Guibas, Ron Kimmel

Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.

Object

Towards Precise Completion of Deformable Shapes

1 code implementation ECCV 2020 Oshri Halimi, Ido Imanuel, Or Litany, Giovanni Trappolini, Emanuele Rodolà, Leonidas Guibas, Ron Kimmel

Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.

Object

Adaptive learning of effective dynamics: Adaptive real-time, online modeling for complex systems

1 code implementation4 Apr 2023 Ivica Kičić, Pantelis R. Vlachas, Georgios Arampatzis, Michail Chatzimanolakis, Leonidas Guibas, Petros Koumoutsakos

To the best of our knowledge, AdaLED is the first framework that couples a surrogate model with a computational solver to achieve online adaptive learning of effective dynamics.

Weather Forecasting

Learning Fuzzy Set Representations of Partial Shapes on Dual Embedding Spaces

1 code implementation4 Jul 2018 Minhyuk Sung, Anastasia Dubrovina, Vladimir G. Kim, Leonidas Guibas

Modeling relations between components of 3D objects is essential for many geometry editing tasks.

Graphics I.3.5

Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation

1 code implementation20 Dec 2020 Haowen Deng, Mai Bui, Nassir Navab, Leonidas Guibas, Slobodan Ilic, Tolga Birdal

For the former we contributed our own dataset composed of five indoor scenes where it is unavoidable to capture images corresponding to views that are hard to uniquely identify.

Camera Relocalization Pose Estimation

Domain Adaptation on Point Clouds via Geometry-Aware Implicits

1 code implementation CVPR 2022 Yuefan Shen, Yanchao Yang, Mi Yan, He Wang, Youyi Zheng, Leonidas Guibas

Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.

Autonomous Driving Unsupervised Domain Adaptation

Learning Spectral Unions of Partial Deformable 3D Shapes

1 code implementation31 Mar 2021 Luca Moschella, Simone Melzi, Luca Cosmo, Filippo Maggioli, Or Litany, Maks Ovsjanikov, Leonidas Guibas, Emanuele Rodolà

Spectral geometric methods have brought revolutionary changes to the field of geometry processing.

Cross-Image Context Matters for Bongard Problems

1 code implementation7 Sep 2023 Nikhil Raghuraman, Adam W. Harley, Leonidas Guibas

Current machine learning methods struggle to solve Bongard problems, which are a type of IQ test that requires deriving an abstract "concept" from a set of positive and negative "support" images, and then classifying whether or not a new query image depicts the key concept.

Ranked #2 on Few-Shot Image Classification on Bongard-HOI (using extra training data)

Few-Shot Image Classification Few-Shot Learning

Graph Matching with Anchor Nodes: A Learning Approach

no code implementations CVPR 2013 Nan Hu, Raif M. Rustamov, Leonidas Guibas

In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes.

Graph Matching

Stable and Informative Spectral Signatures for Graph Matching

no code implementations CVPR 2014 Nan Hu, Raif M. Rustamov, Leonidas Guibas

We also introduce the pairwise heat kernel distance as a stable second order compatibility term; we justify its plausibility by showing that in a certain limiting case it converges to the classical adjacency matrix-based second order compatibility function.

Graph Matching Informativeness

Distributable Consistent Multi-Object Matching

no code implementations CVPR 2018 Nan Hu, Qi-Xing Huang, Boris Thibert, Leonidas Guibas

In this paper we propose an optimization-based framework to multiple object matching.

Object

GRASS: Generative Recursive Autoencoders for Shape Structures

no code implementations5 May 2017 Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, Leonidas Guibas

We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures.

Beyond Holistic Object Recognition: Enriching Image Understanding with Part States

no code implementations CVPR 2018 Cewu Lu, Hao Su, Yongyi Lu, Li Yi, Chi-Keung Tang, Leonidas Guibas

Important high-level vision tasks such as human-object interaction, image captioning and robotic manipulation require rich semantic descriptions of objects at part level.

Human-Object Interaction Detection Image Captioning +1

SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation

no code implementations CVPR 2017 Li Yi, Hao Su, Xingwen Guo, Leonidas Guibas

To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases.

3D Part Segmentation

Learning Program Embeddings to Propagate Feedback on Student Code

no code implementations22 May 2015 Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, Leonidas Guibas

Providing feedback, both assessing final work and giving hints to stuck students, is difficult for open-ended assignments in massive online classes which can range from thousands to millions of students.

3D-Assisted Image Feature Synthesis for Novel Views of an Object

no code implementations26 Nov 2014 Hao Su, Fan Wang, Li Yi, Leonidas Guibas

In this paper, given a single input image of an object, we synthesize new features for other views of the same object.

Image Retrieval Object +1

Scalable Semidefinite Relaxation for Maximum A Posterior Estimation

no code implementations19 May 2014 Qixing Huang, Yuxin Chen, Leonidas Guibas

Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs.

Composite Shape Modeling via Latent Space Factorization

no code implementations ICCV 2019 Anastasia Dubrovina, Fei Xia, Panos Achlioptas, Mira Shalah, Raphael Groscot, Leonidas Guibas

We present a novel neural network architecture, termed Decomposer-Composer, for semantic structure-aware 3D shape modeling.

3D Shape Modeling

AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss

no code implementations CVPR 2020 Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Li Yi, Leonidas Guibas, Hao Zhang

While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaCoSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user-specified) K parts.

Instance Segmentation Segmentation +1

Continuous Geodesic Convolutions for Learning on 3D Shapes

no code implementations6 Feb 2020 Zhangsihao Yang, Or Litany, Tolga Birdal, Srinath Sridhar, Leonidas Guibas

In this work, we wish to challenge this practice and use a neural network to learn descriptors directly from the raw mesh.

Representation Learning Through Latent Canonicalizations

no code implementations26 Feb 2020 Or Litany, Ari Morcos, Srinath Sridhar, Leonidas Guibas, Judy Hoffman

We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision.

Disentanglement

Synchronizing Probability Measures on Rotations via Optimal Transport

no code implementations CVPR 2020 Tolga Birdal, Michael Arbel, Umut Şimşekli, Leonidas Guibas

We introduce a new paradigm, $\textit{measure synchronization}$, for synchronizing graphs with measure-valued edges.

Pose Estimation

Quantum Permutation Synchronization

no code implementations CVPR 2021 Tolga Birdal, Vladislav Golyanik, Christian Theobalt, Leonidas Guibas

We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.

StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects

no code implementations17 May 2021 Ge Zhang, Or Litany, Srinath Sridhar, Leonidas Guibas

We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images.

3D Reconstruction Object

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

no code implementations ICLR 2022 Ruihai Wu, Yan Zhao, Kaichun Mo, Zizheng Guo, Yian Wang, Tianhao Wu, Qingnan Fan, Xuelin Chen, Leonidas Guibas, Hao Dong

In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.

Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation

no code implementations NeurIPS 2021 Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang

Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.

Object Pose Estimation +1

Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds

no code implementations NeurIPS 2021 Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang

To reduce the huge amount of pose annotations needed for category-level learning, we propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.

Object Pose Estimation +1

AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions

no code implementations1 Dec 2021 Yian Wang, Ruihai Wu, Kaichun Mo, Jiaqi Ke, Qingnan Fan, Leonidas Guibas, Hao Dong

Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.

Friction

IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes

no code implementations ICLR 2022 Qi Li, Kaichun Mo, Yanchao Yang, Hang Zhao, Leonidas Guibas

While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).

Object

Object Pursuit: Building a Space of Objects via Discriminative Weight Generation

no code implementations ICLR 2022 Chuanyu Pan, Yanchao Yang, Kaichun Mo, Yueqi Duan, Leonidas Guibas

We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.

Disentanglement Object

NeuForm: Adaptive Overfitting for Neural Shape Editing

no code implementations18 Jul 2022 Connor Z. Lin, Niloy J. Mitra, Gordon Wetzstein, Leonidas Guibas, Paul Guerrero

Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis.

Affection: Learning Affective Explanations for Real-World Visual Data

no code implementations CVPR 2023 Panos Achlioptas, Maks Ovsjanikov, Leonidas Guibas, Sergey Tulyakov

To embark on this journey, we introduce and share with the research community a large-scale dataset that contains emotional reactions and free-form textual explanations for 85, 007 publicly available images, analyzed by 6, 283 annotators who were asked to indicate and explain how and why they felt in a particular way when observing a specific image, producing a total of 526, 749 responses.

Breaking the Symmetry: Resolving Symmetry Ambiguities in Equivariant Neural Networks

no code implementations29 Oct 2022 Sidhika Balachandar, Adrien Poulenard, Congyue Deng, Leonidas Guibas

We present OAVNN: Orientation Aware Vector Neuron Network, an extension of the Vector Neuron Network.

SinGRAF: Learning a 3D Generative Radiance Field for a Single Scene

no code implementations CVPR 2023 Minjung Son, Jeong Joon Park, Leonidas Guibas, Gordon Wetzstein

Generative models have shown great promise in synthesizing photorealistic 3D objects, but they require large amounts of training data.

An Information-Theoretic Approach to Transferability in Task Transfer Learning

no code implementations20 Dec 2022 Yajie Bao, Yang Li, Shao-Lun Huang, Lin Zhang, Lizhong Zheng, Amir Zamir, Leonidas Guibas

Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks.

Model Selection Transfer Learning

HTTE: A Hybrid Technique For Travel Time Estimation In Sparse Data Environments

no code implementations12 Jan 2023 Nikolaos Zygouras, Nikolaos Panagiotou, Yang Li, Dimitrios Gunopulos, Leonidas Guibas

Travel time estimation is a critical task, useful to many urban applications at the individual citizen and the stakeholder level.

Travel Time Estimation

Category-Level Multi-Part Multi-Joint 3D Shape Assembly

no code implementations10 Mar 2023 Yichen Li, Kaichun Mo, Yueqi Duan, He Wang, Jiequan Zhang, Lin Shao, Wojciech Matusik, Leonidas Guibas

A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.

Graph Learning Graph Representation Learning

PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision

no code implementations16 Mar 2023 Konstantinos Tertikas, Despoina Paschalidou, Boxiao Pan, Jeong Joon Park, Mikaela Angelina Uy, Ioannis Emiris, Yannis Avrithis, Leonidas Guibas

Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.

CC3D: Layout-Conditioned Generation of Compositional 3D Scenes

no code implementations ICCV 2023 Sherwin Bahmani, Jeong Joon Park, Despoina Paschalidou, Xingguang Yan, Gordon Wetzstein, Leonidas Guibas, Andrea Tagliasacchi

In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.

Inductive Bias

SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimates

no code implementations CVPR 2023 Mikaela Angelina Uy, Ricardo Martin-Brualla, Leonidas Guibas, Ke Li

To address this issue, we introduce SCADE, a novel technique that improves NeRF reconstruction quality on sparse, unconstrained input views for in-the-wild indoor scenes.

3D Reconstruction Monocular Depth Estimation +1

GINA-3D: Learning to Generate Implicit Neural Assets in the Wild

no code implementations CVPR 2023 Bokui Shen, Xinchen Yan, Charles R. Qi, Mahyar Najibi, Boyang Deng, Leonidas Guibas, Yin Zhou, Dragomir Anguelov

Modeling the 3D world from sensor data for simulation is a scalable way of developing testing and validation environments for robotic learning problems such as autonomous driving.

Autonomous Driving Representation Learning

Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization

no code implementations4 May 2023 Connor Z. Lin, Koki Nagano, Jan Kautz, Eric R. Chan, Umar Iqbal, Leonidas Guibas, Gordon Wetzstein, Sameh Khamis

To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing.

Face Model Face Reconstruction

NAP: Neural 3D Articulation Prior

no code implementations25 May 2023 Jiahui Lei, Congyue Deng, Bokui Shen, Leonidas Guibas, Kostas Daniilidis

We propose Neural 3D Articulation Prior (NAP), the first 3D deep generative model to synthesize 3D articulated object models.

Denoising Graph Attention +1

NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis

no code implementations14 Jul 2023 Nilesh Kulkarni, Davis Rempe, Kyle Genova, Abhijit Kundu, Justin Johnson, David Fouhey, Leonidas Guibas

This interaction field guides the sampling of an object-conditioned human motion diffusion model, so as to encourage plausible contacts and affordance semantics.

Motion Synthesis valid

Zero-Shot Open-Vocabulary Tracking with Large Pre-Trained Models

no code implementations10 Oct 2023 Wen-Hsuan Chu, Adam W. Harley, Pavel Tokmakov, Achal Dave, Leonidas Guibas, Katerina Fragkiadaki

This begs the question: can we re-purpose these large-scale pre-trained static image models for open-vocabulary video tracking?

Object Object Tracking +5

SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation

no code implementations25 Oct 2023 Qianxu Wang, Haotong Zhang, Congyue Deng, Yang You, Hao Dong, Yixin Zhu, Leonidas Guibas

Central to SparseDFF is a feature refinement network, optimized with a contrastive loss between views and a point-pruning mechanism for feature continuity.

One-Shot Learning

NeRF Revisited: Fixing Quadrature Instability in Volume Rendering

no code implementations NeurIPS 2023 Mikaela Angelina Uy, Kiyohiro Nakayama, Guandao Yang, Rahul Krishna Thomas, Leonidas Guibas, Ke Li

Volume rendering requires evaluating an integral along each ray, which is numerically approximated with a finite sum that corresponds to the exact integral along the ray under piecewise constant volume density.

Make a Donut: Hierarchical EMD-Space Planning for Zero-Shot Deformable Manipulation with Tools

no code implementations5 Nov 2023 Yang You, Bokui Shen, Congyue Deng, Haoran Geng, Songlin Wei, He Wang, Leonidas Guibas

Remarkably, our model demonstrates robust generalization capabilities to novel and previously unencountered complex tasks without any preliminary demonstrations.

Deformable Object Manipulation Model Predictive Control

Rethinking Directional Integration in Neural Radiance Fields

no code implementations28 Nov 2023 Congyue Deng, Jiawei Yang, Leonidas Guibas, Yue Wang

To that end, we introduce a modification to the NeRF rendering equation which is as simple as a few lines of code change for any NeRF variations, while greatly improving the rendering quality of view-dependent effects.

3D Reconstruction Disentanglement +2

4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling

no code implementations29 Nov 2023 Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, David B. Lindell

Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.

SAGE: Bridging Semantic and Actionable Parts for GEneralizable Articulated-Object Manipulation under Language Instructions

no code implementations3 Dec 2023 Haoran Geng, Songlin Wei, Congyue Deng, Bokui Shen, He Wang, Leonidas Guibas

To address this problem, we propose SAGE, a novel framework that bridges the understanding of semantic and actionable parts of articulated objects to achieve generalizable manipulation under language instructions.

Object

CAD: Photorealistic 3D Generation via Adversarial Distillation

no code implementations11 Dec 2023 Ziyu Wan, Despoina Paschalidou, IAn Huang, Hongyu Liu, Bokui Shen, Xiaoyu Xiang, Jing Liao, Leonidas Guibas

The increased demand for 3D data in AR/VR, robotics and gaming applications, gave rise to powerful generative pipelines capable of synthesizing high-quality 3D objects.

ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Process

no code implementations16 Jan 2024 Kiyohiro Nakayama, Mikaela Angelina Uy, Yang You, Ke Li, Leonidas Guibas

We introduce ProvNeRF, a model that enriches a traditional NeRF representation by incorporating per-point provenance, modeling likely source locations for each point.

Novel View Synthesis

Zero-Shot Image Feature Consensus with Deep Functional Maps

no code implementations18 Mar 2024 Xinle Cheng, Congyue Deng, Adam Harley, Yixin Zhu, Leonidas Guibas

We demonstrate that our technique yields correspondences that are not only smoother but also more accurate, with the possibility of better reflecting the knowledge embedded in the large-scale vision models that we are studying.

Cannot find the paper you are looking for? You can Submit a new open access paper.