Search Results for author:
Found 82 papers, 49 papers with code
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Due to the scarcity and unsuitability of existent 3D-oriented linguistic resources for this task, we first develop two large-scale and complementary visio-linguistic datasets: i) extbf{ extit{Sr3D}}, which contains 83. 5K template-based utterances leveraging extit{spatial relations} with other fine-grained object classes to localize a referred object in a given scene, and ii) extbf{ extit{Nr3D}} which contains 41. 5K extit{natural, free-form}, utterances collected by deploying a 2-player object reference game in 3D scenes.
Towards Precise Completion of Deformable Shapes
Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
Our model builds a panoptic radiance field representation of any scene from just color images.
Directed Weight Neural Networks for Protein Structure Representation Learning
In this work, we propose the Directed Weight Neural Network for better capturing geometric relations among different amino acids.
Point2Cyl: Reverse Engineering 3D Objects from Point Clouds to Extrusion Cylinders
We propose Point2Cyl, a supervised network transforming a raw 3D point cloud to a set of extrusion cylinders.
Domain Adaptation on Point Clouds via Geometry-Aware Implicits
Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.
Object Pursuit: Building a Space of Objects via Discriminative Weight Generation
We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.
Efficient Geometry-aware 3D Generative Adversarial Networks
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.
PartGlot: Learning Shape Part Segmentation from Language Reference Games
We introduce PartGlot, a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language.
IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes
While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).
AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions
Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.
Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks
Disobeying the classical wisdom of statistical learning theory, modern deep neural networks generalize well even though they typically contain millions of parameters.
Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation
Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.
Projective Manifold Gradient Layer for Deep Rotation Regression
Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem.
ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.
DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis
We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth.
O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning
Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.
VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects
In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.
SketchGen: Generating Constrained CAD Sketches
Sketches can be represented as graphs, with the primitives as nodes and the constraints as edges.
Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds
To reduce the huge amount of pose annotations needed for category-level learning, we propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.
StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects
We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images.
Vector Neurons: A General Framework for SO(3)-Equivariant Networks
Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds.
Learning Spectral Unions of Partial Deformable 3D Shapes
Spectral geometric methods have brought revolutionary changes to the field of geometry processing.
Quantum Permutation Synchronization
We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.
Joint Learning of 3D Shape Retrieval and Deformation
In fact, we use the embedding space to guide the shape pairs used to train the deformation module, so that it invests its capacity in learning deformations between meaningful shape pairs.
ArtEmis: Affective Language for Visual Art
We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language.
MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds.
Where2Act: From Pixels to Actions for Articulated 3D Objects
One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment.
Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation
For the former we contributed our own dataset composed of five indoor scenes where it is unavoidable to capture images corresponding to views that are hard to uniquely identify.
Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments
Localizing the camera in a known indoor environment is a key building block for scene mapping, robot navigation, AR, etc.
Towards Accurate Active Camera Localization
The past solutions are mostly based on Markov Localization, which reduces the position-wise camera uncertainty for localization.
Generative 3D Part Assembly via Dynamic Graph Learning
Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.
ShapeFlow: Learnable Deformations Among 3D Shapes
We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
Robust Learning Through Cross-Task Consistency
Visual perception entails solving a wide set of tasks, e. g., object detection, depth estimation, etc.
MeshODE: A Robust and Scalable Framework for Mesh Deformation
Given a pair of shapes, our framework provides a novel shape feature-preserving mapping function that continuously deforms one model to the other by minimizing fitting and rigidity losses based on the non-rigid iterative-closest-point (ICP) algorithm.
Graphics Computational Geometry
6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference
We present a multimodal camera relocalization framework that captures ambiguities and uncertainties with continuous mixture models defined on the manifold of camera poses.
Deformation-Aware 3D Model Embedding and Retrieval
We introduce a new problem of retrieving 3D models that are deformable to a given query shape and present a novel deep deformation-aware embedding to solve this retrieval task.
Synchronizing Probability Measures on Rotations via Optimal Transport
We introduce a new paradigm, $\textit{measure synchronization}$, for synchronizing graphs with measure-valued edges.
Learning 3D Part Assembly from a Single Image
Autonomous assembly is a crucial capability for robots in many applications.
Adversarial Texture Optimization from RGB-D Scans
In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.
Representation Learning Through Latent Canonicalizations
We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision.
Continuous Geodesic Convolutions for Learning on 3D Shapes
In this work, we wish to challenge this practice and use a neural network to learn descriptors directly from the raw mesh.
The Whole Is Greater Than the Sum of Its Nonrigid Parts
Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.
From Planes to Corners: Multi-Purpose Primitive Detection in Unorganized 3D Point Clouds
We propose a new method for segmentation-free joint estimation of orthogonal planes, their intersection lines, relationship graph and corners lying at the intersection of three orthogonal planes.
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.
Quaternion Equivariant Capsule Networks for 3D Point Clouds
We present a 3D capsule module for processing point clouds that is equivariant to 3D rotations and translations, as well as invariant to permutations of the input points.
Category-Level Articulated Object Pose Estimation
We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space.
Learning to Navigate Using Mid-Level Visual Priors
How much does having visual priors about the world (e. g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e. g. navigating a complex environment)?
Joint Embedding of 3D Scan and CAD Objects
We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.
Which Tasks Should Be Learned Together in Multi-task Learning?
Many computer vision applications require solving multiple tasks in real-time.
OperatorNet: Recovering 3D Shapes From Difference Operators
This paper proposes a learning-based framework for reconstructing 3D shapes from functional operators, compactly encoded as small-sized matrices.
FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image
In this work, we introduce the novel problem of identifying dense canonical 3D coordinate frames from a single RGB image.
AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss
While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaCoSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user-specified) K parts.
Learning Transformation Synchronization
Our approach alternates between transformation synchronization using weighted relative transformations and predicting new weights of the input relative transformations using a neural network.
Composite Shape Modeling via Latent Space Factorization
We present a novel neural network architecture, termed Decomposer-Composer, for semantic structure-aware 3D shape modeling.
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
This skill set (hereafter mid-level perception) provides the policy with a more processed state of the world compared to raw images.
GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud
We introduce a novel 3D object proposal approach named Generative Shape Proposal Network (GSPN) for instance segmentation in point cloud data.
Ranked #16 on
3D Object Detection
on ScanNetV2
TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).
Ranked #11 on
Semantic Segmentation
on ScanNet
Supervised Fitting of Geometric Primitives to 3D Point Clouds
Fitting geometric primitives to 3D point cloud data bridges a gap between low-level digitized 3D data and high-level structural information on the underlying 3D shapes.
Deep Part Induction from Articulated Object Pairs
In this paper, we explore how the observation of different articulation states provides evidence for part structure and motion of 3D objects.
Learning Fuzzy Set Representations of Partial Shapes on Dual Embedding Spaces
Modeling relations between components of 3D objects is essential for many geometry editing tasks.
Graphics I.3.5
PeerNets: Exploiting Peer Wisdom Against Adversarial Attacks
Deep learning systems have become ubiquitous in many aspects of our lives.
Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions
Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes.
Taskonomy: Disentangling Task Transfer Learning
The product is a computational taxonomic map for task transfer learning.
Graph Matching with Anchor Nodes: A Learning Approach
In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes.
Robust Watertight Manifold Surface Generation Method for ShapeNet Models
In this paper, we describe a robust algorithm for 2-Manifold generation of various kinds of ShapeNet Models.
Computational Geometry
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database.
ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling
The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input.
Graphics I.3.5
Learning Representations and Generative Models for 3D Point Clouds
Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling.
GRASS: Generative Recursive Autoencoders for Shape Structures
We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures.
Beyond Holistic Object Recognition: Enriching Image Understanding with Part States
Important high-level vision tasks such as human-object interaction, image captioning and robotic manipulation require rich semantic descriptions of objects at part level.
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation
To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases.
Ranked #24 on
3D Part Segmentation
on ShapeNet-Part
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image.
Ranked #2 on
3D Reconstruction
on Data3D−R2N2
(using extra training data)
3D Object Reconstruction From A Single Image
3D Reconstruction
Distributable Consistent Multi-Object Matching
In this paper we propose an optimization-based framework to multiple object matching.
ShapeNet: An Information-Rich 3D Model Repository
We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects.
Deep Knowledge Tracing
Knowledge tracing---where a machine models the knowledge of a student as they interact with coursework---is a well established problem in computer supported education.
Ranked #1 on
Knowledge Tracing
on Assistments
Learning Program Embeddings to Propagate Feedback on Student Code
Providing feedback, both assessing final work and giving hints to stuck students, is difficult for open-ended assignments in massive online classes which can range from thousands to millions of students.
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Object viewpoint estimation from 2D images is an essential task in computer vision.
3D-Assisted Image Feature Synthesis for Novel Views of an Object
In this paper, given a single input image of an object, we synthesize new features for other views of the same object.
Scalable Semidefinite Relaxation for Maximum A Posterior Estimation
Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs.
Stable and Informative Spectral Signatures for Graph Matching
We also introduce the pairwise heat kernel distance as a stable second order compatibility term; we justify its plausibility by showing that in a certain limiting case it converges to the classical adjacency matrix-based second order compatibility function.
