Search Results for author: Angela Dai

Found 59 papers, 32 papers with code

DrivAerNet: A Parametric Car Dataset for Data-Driven Aerodynamic Design and Graph-Based Drag Prediction

1 code implementation12 Mar 2024 Mohamed Elrefaie, Angela Dai, Faez Ahmed

This study introduces DrivAerNet, a large-scale high-fidelity CFD dataset of 3D industry-standard car shapes, and RegDGCNN, a dynamic graph convolutional neural network model, both aimed at aerodynamic car design through machine learning.

FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models

1 code implementation13 Dec 2023 Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner

We propose a new latent diffusion model for this task, operating in the expression space of neural parametric head models, to synthesize audio-driven realistic head sequences.

3D Face Animation Audio Synthesis +1

PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awareness

1 code implementation4 Dec 2023 Anh-Quan Cao, Angela Dai, Raoul de Charette

We propose the task of Panoptic Scene Completion (PSC) which extends the recently popular Semantic Scene Completion (SSC) task with instance-level information to produce a richer understanding of the 3D scene.

Autonomous Driving

DPHMs: Diffusion Parametric Head Models for Depth-based Tracking

no code implementations2 Dec 2023 Jiapeng Tang, Angela Dai, Yinyu Nie, Lev Markhasin, Justus Thies, Matthias Niessner

We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences.

DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image

no code implementations30 Nov 2023 Daoyi Gao, Dávid Rozenberszki, Stefan Leutenegger, Angela Dai

We formulate this as a conditional generative task, leveraging diffusion to learn implicit probabilistic models capturing the shape, pose, and scale of CAD objects in an image.

Retrieval

GenZI: Zero-Shot 3D Human-Scene Interaction Generation

no code implementations29 Nov 2023 Lei LI, Angela Dai

Given a natural language description and a coarse point location of the desired interaction in a 3D scene, we first leverage VLMs to imagine plausible 2D human interactions inpainted into multiple rendered views of the scene.

MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers

2 code implementations27 Nov 2023 Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, Matthias Nießner

We introduce MeshGPT, a new approach for generating triangle meshes that reflects the compactness typical of artist-created meshes, in contrast to dense triangle meshes extracted by iso-surfacing methods from neural fields.

CG-HOI: Contact-Guided 3D Human-Object Interaction Generation

no code implementations27 Nov 2023 Christian Diller, Angela Dai

Using this guidance to bridge human and object motion enables generating more realistic and physically plausible interaction sequences, where the human body and corresponding object move in a coherent manner.

Human-Object Interaction Detection Human-Object Interaction Generation +1

ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes

no code implementations ICCV 2023 Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, Angela Dai

Each scene is captured with a high-end laser scanner at sub-millimeter resolution, along with registered 33-megapixel images from a DSLR camera, and RGB-D streams from an iPhone.

Novel View Synthesis Scene Understanding

Mesh2Tex: Generating Mesh Textures from Image Queries

no code implementations ICCV 2023 Alexey Bokhovkin, Shubham Tulsiani, Angela Dai

The learned texture manifold enables effective navigation to generate an object texture for a given 3D object geometry that matches to an input RGB image, which maintains robustness even under challenging real-world scenarios where the mesh geometry approximates an inexact match to the underlying geometry in the RGB image.

Object

HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion

1 code implementation ICCV 2023 Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nießner, Angela Dai

HyperDiffusion operates directly on MLP weights and generates new neural implicit fields encoded by synthesized MLP parameters.

UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes

no code implementations25 Mar 2023 David Rozenberszki, Or Litany, Angela Dai

We propose UnScene3D, the first fully unsupervised 3D learning approach for class-agnostic 3D instance segmentation of indoor scans.

3D Instance Segmentation Segmentation +1

DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis

no code implementations24 Mar 2023 Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, Matthias Nießner

We introduce a diffusion network to synthesize a collection of 3D indoor objects by denoising a set of unordered object attributes.

Denoising Indoor Scene Synthesis +1

Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors

no code implementations CVPR 2023 Ji Hou, Xiaoliang Dai, Zijian He, Angela Dai, Matthias Nießner

Current popular backbones in computer vision, such as Vision Transformers (ViT) and ResNets are trained to perceive the world from 2D images.

Contrastive Learning Instance Segmentation +6

ClipFace: Text-guided Editing of Textured 3D Morphable Models

1 code implementation2 Dec 2022 Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner

Controllable editing and manipulation are given by language prompts to adapt texture and expression of the 3D morphable model.

Texture Synthesis

FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations

no code implementations25 Nov 2022 Christian Diller, Thomas Funkhouser, Angela Dai

Thus, we design our method to only require 2D RGB data while being able to generate 3D human motion sequences.

Pose Prediction

Learning 3D Scene Priors with 2D Supervision

no code implementations CVPR 2023 Yinyu Nie, Angela Dai, Xiaoguang Han, Matthias Nießner

Holistic 3D scene understanding entails estimation of both layout configuration and object geometry in a 3D environment.

Scene Understanding

Neural Poisson: Indicator Functions for Neural Fields

no code implementations25 Nov 2022 Angela Dai, Matthias Nießner

Implicit neural field generating signed distance field representations (SDFs) of 3D shapes have shown remarkable progress in 3D shape reconstruction and generation.

3D Shape Reconstruction Surface Reconstruction

PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories

1 code implementation10 Jun 2022 Yuchen Rao, Yinyu Nie, Angela Dai

While 3D shape representations enable powerful reasoning in many visual and perception applications, learning 3D shape priors tends to be constrained to the specific categories trained on, leading to an inefficient learning process, particularly for general applications with unseen categories.

Language-Grounded Indoor 3D Semantic Segmentation in the Wild

1 code implementation16 Apr 2022 David Rozenberszki, Or Litany, Angela Dai

Recent advances in 3D semantic segmentation with deep neural networks have shown remarkable success, with rapid performance increase on available datasets.

3D Semantic Segmentation Segmentation

Texturify: Generating Textures on 3D Shape Surfaces

no code implementations5 Apr 2022 Yawar Siddiqui, Justus Thies, Fangchang Ma, Qi Shan, Matthias Nießner, Angela Dai

Texture cues on 3D objects are key to compelling visual representations, with the possibility to create high visual fidelity with inherent spatial consistency across different views.

Weakly-Supervised End-to-End CAD Retrieval to Scan Objects

no code implementations24 Mar 2022 Tim Beyer, Angela Dai

CAD model retrieval to real-world scene observations has shown strong promise as a basis for 3D perception of objects and a clean, lightweight mesh-based scene representation; however, current approaches to retrieve CAD models to a query scan rely on expensive manual annotations of 1:1 associations of CAD-scan objects, which typically contain strong lower-level geometric differences.

object-detection Object Detection +1

Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scans

no code implementations CVPR 2023 Alexey Bokhovkin, Angela Dai

3D object recognition has seen significant advances in recent years, showing impressive performance on real-world 3D scan benchmarks, but lacking in object part reasoning, which is fundamental to higher-level scene understanding such as inter-object similarities or object functionality.

3D Object Recognition Object +1

SPAMs: Structured Implicit Parametric Models

no code implementations CVPR 2022 Pablo Palafox, Nikolaos Sarafianos, Tony Tung, Angela Dai

We observe that deformable object motion is often semantically structured, and thus propose to learn Structured-implicit PArametric Models (SPAMs) as a deformable object representation that structurally decomposes non-rigid object motion into part-based disentangled representations of shape and pose, with each being represented by deep implicit functions.

Object

Pose2Room: Understanding 3D Scenes from Human Activities

no code implementations1 Dec 2021 Yinyu Nie, Angela Dai, Xiaoguang Han, Matthias Nießner

To this end, we propose P2R-Net to learn a probabilistic 3D model of the objects in a scene characterized by their class categories and oriented 3D bounding boxes, based on an input observed human trajectory in the environment.

Object

Panoptic 3D Scene Reconstruction From a Single RGB Image

1 code implementation NeurIPS 2021 Manuel Dahnert, Ji Hou, Matthias Nießner, Angela Dai

Inspired by 2D panoptic segmentation, we propose to unify the tasks of geometric reconstruction, 3D semantic segmentation, and 3D instance segmentation into the task of panoptic 3D scene reconstruction - from a single RGB image, predicting the complete geometric reconstruction of the scene in the camera frustum of the image, along with semantic and instance segmentations.

3D Instance Segmentation 3D Scene Reconstruction +5

Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image

no code implementations ICCV 2021 Weicheng Kuo, Anelia Angelova, Tsung-Yi Lin, Angela Dai

3D perception of object shapes from RGB image input is fundamental towards semantic scene understanding, grounding image-based perception in our spatially 3-dimensional real-world environments.

Retrieval Scene Understanding

Pri3D: Can 3D Priors Help 2D Representation Learning?

1 code implementation ICCV 2021 Ji Hou, Saining Xie, Benjamin Graham, Angela Dai, Matthias Nießner

Inspired by these advances in geometric understanding, we aim to imbue image-based perception with representations learned under geometric constraints.

Contrastive Learning Instance Segmentation +5

NPMs: Neural Parametric Models for 3D Deformable Shapes

1 code implementation ICCV 2021 Pablo Palafox, Aljaž Božič, Justus Thies, Matthias Nießner, Angela Dai

Crucially, once learned, our neural parametric models of shape and pose enable optimization over the learned spaces to fit to new observations, similar to the fitting of a traditional parametric model, e. g., SMPL.

Pose Transfer

RetrievalFuse: Neural 3D Scene Reconstruction with a Database

1 code implementation ICCV 2021 Yawar Siddiqui, Justus Thies, Fangchang Ma, Qi Shan, Matthias Nießner, Angela Dai

3D reconstruction of large scenes is a challenging problem due to the high-complexity nature of the solution space, in particular for generative neural networks.

3D Reconstruction 3D Scene Reconstruction +3

Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences

no code implementations CVPR 2021 Norman Müller, Yu-Shiang Wong, Niloy J. Mitra, Angela Dai, Matthias Nießner

From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space.

3D Multi-Object Tracking Object

Towards Part-Based Understanding of RGB-D Scans

1 code implementation CVPR 2021 Alexey Bokhovkin, Vladislav Ishimtsev, Emil Bogomolov, Denis Zorin, Alexey Artemov, Evgeny Burnaev, Angela Dai

Recent advances in 3D semantic scene understanding have shown impressive progress in 3D instance segmentation, enabling object-level reasoning about 3D scenes; however, a finer-grained understanding is required to enable interactions with objects and their functional understanding.

3D Instance Segmentation Object +2

Forecasting Characteristic 3D Poses of Human Actions

no code implementations CVPR 2022 Christian Diller, Thomas Funkhouser, Angela Dai

To predict characteristic poses, we propose a probabilistic approach that models the possible multi-modality in the distribution of likely characteristic poses.

Human motion prediction motion prediction +1

Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve

no code implementations ECCV 2020 Wei-cheng Kuo, Anelia Angelova, Tsung-Yi Lin, Angela Dai

We propose to leverage existing large-scale datasets of 3D models to understand the underlying 3D structure of objects seen in an image by constructing a CAD-based representation of the objects and their poses.

Image to 3D Object +3

SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans

1 code implementation CVPR 2021 Angela Dai, Yawar Siddiqui, Justus Thies, Julien Valentin, Matthias Nießner

We present SPSG, a novel approach to generate high-quality, colored 3D models of scenes from RGB-D scan observations by learning to infer unobserved scene geometry and color in a self-supervised fashion.

3D Reconstruction Scene Generation

Neural Non-Rigid Tracking

1 code implementation NeurIPS 2020 Aljaž Božič, Pablo Palafox, Michael Zollhöfer, Angela Dai, Justus Thies, Matthias Nießner

We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction by a learned robust optimization.

SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans

no code implementations ECCV 2020 Armen Avetisyan, Tatiana Khanova, Christopher Choy, Denver Dash, Angela Dai, Matthias Nießner

We present a novel approach to reconstructing lightweight, CAD-based representations of scanned 3D environments from commodity RGB-D sensors.

Object

Adversarial Texture Optimization from RGB-D Scans

1 code implementation CVPR 2020 Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kundu, Chiyu Max Jiang, Leonidas Guibas, Matthias Nießner, Thomas Funkhouser

In this work, we present a novel approach for color texture generation using a conditional adversarial loss obtained from weakly-supervised views.

Surface Reconstruction Texture Synthesis

SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scans

2 code implementations CVPR 2020 Angela Dai, Christian Diller, Matthias Nießner

We present a novel approach that converts partial and noisy RGB-D scans into high-quality 3D scene reconstructions by inferring unobserved scene geometry.

3D Reconstruction

Joint Embedding of 3D Scan and CAD Objects

1 code implementation ICCV 2019 Manuel Dahnert, Angela Dai, Leonidas Guibas, Matthias Nießner

We propose a novel approach to learn a joint embedding space between scan and CAD geometry, where semantically similar objects from both domains lie close together.

Retrieval

End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans

1 code implementation ICCV 2019 Armen Avetisyan, Angela Dai, Matthias Nießner

We present a novel, end-to-end approach to align CAD models to an 3D scan of a scene, enabling transformation of a noisy, incomplete 3D scan to a compact, CAD reconstruction with clean, complete object geometry.

CAD Reconstruction Object +1

RevealNet: Seeing Behind Objects in RGB-D Scans

no code implementations CVPR 2020 Ji Hou, Angela Dai, Matthias Nießner

Thus, we introduce the task of semantic instance completion: from an incomplete RGB-D scan of a scene, we aim to detect the individual object instances and infer their complete object geometry.

3D Reconstruction 3D Semantic Instance Segmentation +2

Scan2CAD: Learning CAD Model Alignment in RGB-D Scans

2 code implementations CVPR 2019 Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X. Chang, Matthias Nießner

For a 3D reconstruction of an indoor scene, our method takes as input a set of CAD models, and predicts a 9DoF pose that aligns each model to the underlying scan geometry.

3D Reconstruction

Scan2Mesh: From Unstructured Range Scans to 3D Meshes

1 code implementation CVPR 2019 Angela Dai, Matthias Nießner

We introduce Scan2Mesh, a novel data-driven generative approach which transforms an unstructured and potentially incomplete range scan into a structured 3D mesh representation.

3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation

1 code implementation ECCV 2018 Angela Dai, Matthias Nießner

We present 3DMV, a novel method for 3D semantic scene segmentation of RGB-D scans in indoor environments using a joint 3D-multi-view prediction network.

3D Architecture Scene Segmentation +1

ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

no code implementations CVPR 2018 Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, Matthias Nießner

We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels.

Semantic Segmentation

Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis

2 code implementations CVPR 2017 Angela Dai, Charles Ruizhongtai Qi, Matthias Nießner

Although our 3D-EPN outperforms state-of-the-art completion method, the main contribution in our work lies in the combination of a data-driven shape predictor and analytic 3D shape synthesis.

3D Shape Generation

Volumetric and Multi-View CNNs for Object Classification on 3D Data

2 code implementations CVPR 2016 Charles R. Qi, Hao Su, Matthias Niessner, Angela Dai, Mengyuan Yan, Leonidas J. Guibas

Empirical results from these two types of CNNs exhibit a large gap, indicating that existing volumetric CNN architectures and approaches are unable to fully exploit the power of 3D representations.

3D Object Recognition 3D Point Cloud Classification +1

BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration

1 code implementation5 Apr 2016 Angela Dai, Matthias Nießner, Michael Zollhöfer, Shahram Izadi, Christian Theobalt

Our approach estimates globally optimized (i. e., bundle adjusted) poses in real-time, supports robust tracking with recovery from gross tracking failures (i. e., relocalization), and re-estimates the 3D model in real-time to ensure global consistency; all within a single framework.

3D Reconstruction Mixed Reality +1

Cannot find the paper you are looking for? You can Submit a new open access paper.