Search Results for author:
Found 114 papers, 64 papers with code
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Due to the scarcity and unsuitability of existent 3D-oriented linguistic resources for this task, we first develop two large-scale and complementary visio-linguistic datasets: i) extbf{ extit{Sr3D}}, which contains 83. 5K template-based utterances leveraging extit{spatial relations} with other fine-grained object classes to localize a referred object in a given scene, and ii) extbf{ extit{Nr3D}} which contains 41. 5K extit{natural, free-form}, utterances collected by deploying a 2-player object reference game in 3D scenes.
Towards Precise Completion of Deformable Shapes
Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.
Rethinking Directional Integration in Neural Radiance Fields
To that end, we introduce a modification to the NeRF rendering equation which is as simple as a few lines of code change for any NeRF variations, while greatly improving the rendering quality of view-dependent effects.
Make a Donut: Language-Guided Hierarchical EMD-Space Planning for Zero-shot Deformable Object Manipulation
Deformable object manipulation stands as one of the most captivating yet formidable challenges in robotics.
SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulation
the end-effector parameters between the demonstration and the target manipulation.
Zero-Shot Open-Vocabulary Tracking with Large Pre-Trained Models
This begs the question: can we re-purpose these large-scale pre-trained static image models for open-vocabulary video tracking?
Cross-Image Context Matters for Bongard Problems
Current machine learning methods struggle to solve Bongard problems, which are a type of IQ test that requires deriving an abstract "concept" from a set of positive and negative "support" images, and then classifying whether or not a new query image depicts the key concept.
Ranked #2 on
Few-Shot Image Classification
on Bongard-HOI
(using extra training data)
NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis
This interaction field guides the sampling of an object-conditioned human motion diffusion model, so as to encourage plausible contacts and affordance semantics.
Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptions
What constitutes the "vibe" of a particular scene?
NAP: Neural 3D Articulation Prior
We propose Neural 3D Articulation Prior (NAP), the first 3D deep generative model to synthesize 3D articulated object models.
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization
To tackle this problem, we propose a novel method for constructing implicit 3D morphable face models that are both generalizable and intuitive for editing.
Adaptive learning of effective dynamics: Adaptive real-time, online modeling for complex systems
To the best of our knowledge, AdaLED is the first framework that couples a surrogate model with a computational solver to achieve online adaptive learning of effective dynamics.
GINA-3D: Learning to Generate Implicit Neural Assets in the Wild
Modeling the 3D world from sensor data for simulation is a scalable way of developing testing and validation environments for robotic learning problems such as autonomous driving.
JacobiNeRF: NeRF Shaping with Mutual Information Gradients
We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.
VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalization
We propose VDN-NeRF, a method to train neural radiance fields (NeRFs) for better geometry under non-Lambertian surface and dynamic lighting conditions that cause significant variation in the radiance of a point when viewed from different angles.
EFEM: Equivariant Neural Field Expectation Maximization for 3D Object Segmentation Without Scene Supervision
First, we introduce equivariant shape representations to this problem to eliminate the complexity induced by the variation in object configuration.
SCADE: NeRFs from Space Carving with Ambiguity-Aware Depth Estimates
To address this issue, we introduce SCADE, a novel technique that improves NeRF reconstruction quality on sparse, unconstrained input views for in-the-wild indoor scenes.
CC3D: Layout-Conditioned Generation of Compositional 3D Scenes
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.
PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision
Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.
Category-Level Multi-Part Multi-Joint 3D Shape Assembly
A successful joint-optimized assembly needs to satisfy the bilateral objectives of shape structure and joint alignment.
Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision
We address efficient and structure-aware 3D scene representation from images.
LEGO-Net: Learning Regular Rearrangements of Objects in Rooms
Humans universally dislike the task of cleaning up a messy room.
HTTE: A Hybrid Technique For Travel Time Estimation In Sparse Data Environments
Travel time estimation is a critical task, useful to many urban applications at the individual citizen and the stakeholder level.
Generating Part-Aware Editable 3D Shapes Without 3D Supervision
Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.
ShapeTalk: A Language Dataset and Framework for 3D Shape Edits and Deformations
In this work, we aim to facilitate the task of editing the geometry of 3D models through the use of natural language.
An Information-Theoretic Approach to Transferability in Task Transfer Learning
Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks.
MetaCLUE: Towards Comprehensive Visual Metaphors Research
Towards this goal, we introduce MetaCLUE, a set of vision tasks on visual metaphor.
LADIS: Language Disentanglement for 3D Shape Editing
Additionally, to measure edit locality, we define a new metric that we call part-wise edit precision.
ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction
One school of thought is to encode a latent vector for each point (point latents).
NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priors
Formulating single-view reconstruction as an image-conditioned 3D generation problem, we optimize the NeRF representations by minimizing a diffusion loss on its arbitrary view renderings with a pretrained image diffusion model under the input-view constraint.
SinGRAF: Learning a 3D Generative Radiance Field for a Single Scene
Generative models have shown great promise in synthesizing photorealistic 3D objects, but they require large amounts of training data.
Breaking the Symmetry: Resolving Symmetry Ambiguities in Equivariant Neural Networks
We present OAVNN: Orientation Aware Vector Neuron Network, an extension of the Vector Neuron Network.
Affection: Learning Affective Explanations for Real-World Visual Data
To embark on this journey, we introduce and share with the research community a large-scale dataset that contains emotional reactions and free-form textual explanations for 85, 007 publicly available images, analyzed by 6, 283 annotators who were asked to indicate and explain how and why they felt in a particular way when observing a specific image, producing a total of 526, 749 responses.
NeuForm: Adaptive Overfitting for Neural Shape Editing
Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis.
3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks.
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
Our model builds a panoptic radiance field representation of any scene from just color images.
Domain Adaptation on Point Clouds via Geometry-Aware Implicits
Here we propose a simple yet effective method for unsupervised domain adaptation on point clouds by employing a self-supervised task of learning geometry-aware implicits, which plays two critical roles in one shot.
Point2Cyl: Reverse Engineering 3D Objects from Point Clouds to Extrusion Cylinders
We propose Point2Cyl, a supervised network transforming a raw 3D point cloud to a set of extrusion cylinders.
Object Pursuit: Building a Space of Objects via Discriminative Weight Generation
We perform an extensive study of the key features of the proposed framework and analyze the characteristics of the learned representations.
Efficient Geometry-aware 3D Generative Adversarial Networks
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.
PartGlot: Learning Shape Part Segmentation from Language Reference Games
We introduce PartGlot, a neural framework and associated architectures for learning semantic part segmentation of 3D shape geometry, based solely on part referential language.
IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes
While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).
AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions
Perceiving and interacting with 3D articulated objects, such as cabinets, doors, and faucets, pose particular challenges for future home-assistant robots performing daily tasks in human environments.
Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks
Disobeying the classical wisdom of statistical learning theory, modern deep neural networks generalize well even though they typically contain millions of parameters.
Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation
Category-level object pose estimation aims to find 6D object poses of previously unseen object instances from known categories without access to object CAD models.
Projective Manifold Gradient Layer for Deep Rotation Regression
Regressing rotations on SO(3) manifold using deep neural networks is an important yet unsolved problem.
ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.
DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis
We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth.
O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning
Contrary to the vast literature in modeling, perceiving, and understanding agent-object (e. g., human-object, hand-object, robot-object) interaction in computer vision and robotics, very few past works have studied the task of object-object interaction, which also plays an important role in robotic manipulation and planning tasks.
VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects
In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals.
SketchGen: Generating Constrained CAD Sketches
Sketches can be represented as graphs, with the primitives as nodes and the constraints as edges.
Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds
To reduce the huge amount of pose annotations needed for category-level learning, we propose for the first time a self-supervised learning framework to estimate category-level 6D object pose from single 3D point clouds.
StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects
We present StrobeNet, a method for category-level 3D reconstruction of articulating objects from one or more unposed RGB images.
Vector Neurons: A General Framework for SO(3)-Equivariant Networks
Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds.
Learning Spectral Unions of Partial Deformable 3D Shapes
Spectral geometric methods have brought revolutionary changes to the field of geometry processing.
ArtEmis: Affective Language for Visual Art
We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language.
Joint Learning of 3D Shape Retrieval and Deformation
In fact, we use the embedding space to guide the shape pairs used to train the deformation module, so that it invests its capacity in learning deformations between meaningful shape pairs.
Quantum Permutation Synchronization
We present QuantumSync, the first quantum algorithm for solving a synchronization problem in the context of computer vision.
MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization
We present MultiBodySync, a novel, end-to-end trainable multi-body motion segmentation and rigid registration framework for multiple input 3D point clouds.
Where2Act: From Pixels to Actions for Articulated 3D Objects
One of the fundamental goals of visual perception is to allow agents to meaningfully interact with their environment.
Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation
For the former we contributed our own dataset composed of five indoor scenes where it is unavoidable to capture images corresponding to views that are hard to uniquely identify.
Towards Accurate Active Camera Localization
These approaches localize the camera in the discrete pose space and are agnostic to the localization-driven scene property, which restricts the camera pose accuracy in the coarse scale.
Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments
Localizing the camera in a known indoor environment is a key building block for scene mapping, robot navigation, AR, etc.
Generative 3D Part Assembly via Dynamic Graph Learning
Analogous to buying an IKEA furniture, given a set of 3D parts that can assemble a single shape, an intelligent agent needs to perceive the 3D part geometry, reason to propose pose estimations for the input parts, and finally call robotic planning and control routines for actuation.
ShapeFlow: Learnable Deformations Among 3D Shapes
We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
Robust Learning Through Cross-Task Consistency
Visual perception entails solving a wide set of tasks, e. g., object detection, depth estimation, etc.
MeshODE: A Robust and Scalable Framework for Mesh Deformation
Given a pair of shapes, our framework provides a novel shape feature-preserving mapping function that continuously deforms one model to the other by minimizing fitting and rigidity losses based on the non-rigid iterative-closest-point (ICP) algorithm.
Graphics Computational Geometry
6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference
We present a multimodal camera relocalization framework that captures ambiguities and uncertainties with continuous mixture models defined on the manifold of camera poses.
Deformation-Aware 3D Model Embedding and Retrieval
We introduce a new problem of retrieving 3D models that are deformable to a given query shape and present a novel deep deformation-aware embedding to solve this retrieval task.
Synchronizing Probability Measures on Rotations via Optimal Transport
We introduce a new paradigm, $\textit{measure synchronization}$, for synchronizing graphs with measure-valued edges.
Learning 3D Part Assembly from a Single Image
Autonomous assembly is a crucial capability for robots in many applications.
Adversarial Texture Optimization from RGB-D Scans
In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.
Representation Learning Through Latent Canonicalizations
We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision.
Continuous Geodesic Convolutions for Learning on 3D Shapes
In this work, we wish to challenge this practice and use a neural network to learn descriptors directly from the raw mesh.
The Whole Is Greater Than the Sum of Its Nonrigid Parts
Here, we claim that observing part of an object which was previously acquired as a whole, one could deal with both partial matching and shape completion in a holistic manner.
From Planes to Corners: Multi-Purpose Primitive Detection in Unorganized 3D Point Clouds
We propose a new method for segmentation-free joint estimation of orthogonal planes, their intersection lines, relationship graph and corners lying at the intersection of three orthogonal planes.
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
When training a neural network for a desired task, one may prefer to adapt a pre-trained network rather than starting from randomly initialized weights.
Quaternion Equivariant Capsule Networks for 3D Point Clouds
We present a 3D capsule module for processing point clouds that is equivariant to 3D rotations and translations, as well as invariant to permutations of the input points.
Category-Level Articulated Object Pose Estimation
We develop a deep network based on PointNet++ that predicts ANCSH from a single depth point cloud, including part segmentation, normalized coordinates, and joint parameters in the canonical object space.
Learning to Navigate Using Mid-Level Visual Priors
How much does having visual priors about the world (e. g. the fact that the world is 3D) assist in learning to perform downstream motor tasks (e. g. navigating a complex environment)?
Joint Embedding of 3D Scan and CAD Objects
We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.
Which Tasks Should Be Learned Together in Multi-task Learning?
Many computer vision applications require solving multiple tasks in real-time.
OperatorNet: Recovering 3D Shapes From Difference Operators
This paper proposes a learning-based framework for reconstructing 3D shapes from functional operators, compactly encoded as small-sized matrices.
FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image
In this work, we introduce the novel problem of identifying dense canonical 3D coordinate frames from a single RGB image.
AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss
While the part prior network can be trained with noisy and inconsistently segmented shapes, the final output of AdaCoSeg is a consistent part labeling for the input set, with each shape segmented into up to (a user-specified) K parts.
Learning Transformation Synchronization
Our approach alternates between transformation synchronization using weighted relative transformations and predicting new weights of the input relative transformations using a neural network.
Composite Shape Modeling via Latent Space Factorization
We present a novel neural network architecture, termed Decomposer-Composer, for semantic structure-aware 3D shape modeling.
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies
This skill set (hereafter mid-level perception) provides the policy with a more processed state of the world compared to raw images.
GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud
We introduce a novel 3D object proposal approach named Generative Shape Proposal Network (GSPN) for instance segmentation in point cloud data.
Ranked #25 on
3D Object Detection
on ScanNetV2
TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e. g., color texture maps).
Ranked #19 on
Semantic Segmentation
on ScanNet
Supervised Fitting of Geometric Primitives to 3D Point Clouds
Fitting geometric primitives to 3D point cloud data bridges a gap between low-level digitized 3D data and high-level structural information on the underlying 3D shapes.
Deep Part Induction from Articulated Object Pairs
In this paper, we explore how the observation of different articulation states provides evidence for part structure and motion of 3D objects.
Learning Fuzzy Set Representations of Partial Shapes on Dual Embedding Spaces
Modeling relations between components of 3D objects is essential for many geometry editing tasks.
Graphics I.3.5
PeerNets: Exploiting Peer Wisdom Against Adversarial Attacks
Deep learning systems have become ubiquitous in many aspects of our lives.
Deep Functional Dictionaries: Learning Consistent Semantic Structures on 3D Models from Functions
Even though our shapes have independent discretizations and no functional correspondences are provided, the network is able to generate latent bases, in a consistent order, that reflect the shared semantic structure among the shapes.
Taskonomy: Disentangling Task Transfer Learning
The product is a computational taxonomic map for task transfer learning.
Graph Matching with Anchor Nodes: A Learning Approach
In this paper, we consider the weighted graph matching problem with partially disclosed correspondences between a number of anchor nodes.
Robust Watertight Manifold Surface Generation Method for ShapeNet Models
In this paper, we describe a robust algorithm for 2-Manifold generation of various kinds of ShapeNet Models.
Computational Geometry
Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
We introduce a large-scale 3D shape understanding benchmark using data and annotation from ShapeNet 3D object database.
ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling
The combinatorial nature of part arrangements poses another challenge, since the retrieval network is not a function: several complements can be appropriate for the same input.
Graphics I.3.5
Learning Representations and Generative Models for 3D Point Clouds
Three-dimensional geometric data offer an excellent domain for studying representation learning and generative modeling.
GRASS: Generative Recursive Autoencoders for Shape Structures
We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures.
Beyond Holistic Object Recognition: Enriching Image Understanding with Part States
Important high-level vision tasks such as human-object interaction, image captioning and robotic manipulation require rich semantic descriptions of objects at part level.
SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation
To enable the prediction of vertex functions on them by convolutional neural networks, we resort to spectral CNN method that enables weight sharing by parameterizing kernels in the spectral domain spanned by graph laplacian eigenbases.
Ranked #54 on
3D Part Segmentation
on ShapeNet-Part
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
Our final solution is a conditional shape sampler, capable of predicting multiple plausible 3D point clouds from an input image.
Ranked #2 on
3D Reconstruction
on Data3D−R2N2
(using extra training data)
3D Object Reconstruction From A Single Image
3D Reconstruction
Distributable Consistent Multi-Object Matching
In this paper we propose an optimization-based framework to multiple object matching.
ShapeNet: An Information-Rich 3D Model Repository
We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects.
Deep Knowledge Tracing
Knowledge tracing---where a machine models the knowledge of a student as they interact with coursework---is a well established problem in computer supported education.
Ranked #1 on
Knowledge Tracing
on Assistments
Learning Program Embeddings to Propagate Feedback on Student Code
Providing feedback, both assessing final work and giving hints to stuck students, is difficult for open-ended assignments in massive online classes which can range from thousands to millions of students.
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
Object viewpoint estimation from 2D images is an essential task in computer vision.
3D-Assisted Image Feature Synthesis for Novel Views of an Object
In this paper, given a single input image of an object, we synthesize new features for other views of the same object.
Scalable Semidefinite Relaxation for Maximum A Posterior Estimation
Maximum a posteriori (MAP) inference over discrete Markov random fields is a fundamental task spanning a wide spectrum of real-world applications, which is known to be NP-hard for general graphs.
Stable and Informative Spectral Signatures for Graph Matching
We also introduce the pairwise heat kernel distance as a stable second order compatibility term; we justify its plausibility by showing that in a certain limiting case it converges to the classical adjacency matrix-based second order compatibility function.
