Search Results for author:
Found 70 papers, 30 papers with code
Points2Surf Learning Implicit Surfaces from Point Clouds
Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans.
ShapeLib: designing a library of procedural 3D shape abstractions with Large Language Models
The discovered shape functions in the library are not only expressive but also generalize beyond the seed set to a full family of shapes.
FlairGPT: Repurposing LLMs for Interior Designs
While we find that LLMs are not yet capable of generating complete layouts, they can be effectively leveraged in a structured manner, inspired by the workflow of interior designers.
GANFusion: Feed-Forward Text-to-3D with Diffusion in GAN Space
We observe that GAN- and diffusion-based generators have complementary qualities: GANs can be trained efficiently with 2D supervision to produce high-quality 3D objects but are hard to condition on text.
Probabilistic Inverse Cameras: Image to 3D via Multiview Geometry
We introduce a hierarchical probabilistic approach to go from a 2D image to multiview 3D: a diffusion "prior" models the unseen 3D geometry, which then conditions a diffusion "decoder" to generate novel views of the subject.
DiffCSG: Differentiable CSG via Rasterization
Our algorithm builds upon CSG rasterization, which displays the result of boolean operations between primitives without explicitly computing the resulting mesh and, as such, bypasses black-box mesh processing.
Multiple Importance Sampling for Stochastic Gradient Estimation
We introduce a theoretical and practical framework for efficient importance sampling of mini-batch samples for gradient estimation from single and multiple probability distributions.
Temporal Residual Jacobians For Rig-free Motion Transfer
We introduce Temporal Residual Jacobians as a novel representation to enable data-driven motion transfer.
Neural Geometry Processing via Spherical Neural Surfaces
In this work, we propose a spherical neural surface representation for genus-0 surfaces and demonstrate how to compute core geometric operators directly on this representation.
SuperGaussian: Repurposing Video Models for 3D Super Resolution
We demonstrate that it is possible to directly repurpose existing (pretrained) video models for 3D super-resolution and thus sidestep the problem of the shortage of large repositories of high-quality 3D training models.
Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos
We present a method to build animatable dog avatars from monocular videos.
Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D
Diffusion handles is a novel approach to enable 3D object edits on diffusion images requiring only existing pre-trained diffusion models depth estimation without any fine-tuning or 3D object retrieval.
GOEmbed: Gradient Origin Embeddings for Representation Agnostic 3D Feature Learning
Encoding information from 2D views of an object into a 3D representation is crucial for generalized 3D feature extraction.
LooseControl: Lifting ControlNet for Generalized Depth Conditioning
We present LooseControl to allow generalized depth conditioning for diffusion-based image generation.
Leveraging VLM-Based Pipelines to Annotate 3D Objects
With these supervised and unsupervised evaluations, we show how a VLM-based pipeline can be leveraged to produce reliable annotations for 764K objects from the Objaverse dataset.
Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features
We present Diff3F as a simple, robust, and class-agnostic feature descriptor that can be computed for untextured input shapes (meshes or point clouds).
Ranked #1 on
3D Dense Shape Correspondence
on SHREC'19
CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs
CAD programs are a popular way to compactly encode shapes as a sequence of operations that are easy to parametrically modify.
Online Importance Sampling for Stochastic Gradient Optimization
Then, we show how our approach can also be used for online data pruning by identifying and discarding data samples that contribute minimally towards the training loss.
ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context
We evaluate our setup on a variety of examples to demonstrate appearance and geometric edits and report 10-30x speedup over concurrent work focusing on text-guided NeRF editing.
Neural Semantic Surface Maps
We present an automated technique for computing a map between two genus-zero shapes, which matches semantically corresponding regions to one another.
BLiSS: Bootstrapped Linear Shape Space
Morphable models are fundamental to numerous human-centered processes as they offer a simple yet expressive shape space.
HoloFusion: Towards Photo-realistic 3D Generative Modeling
Diffusion-based image generators can now produce high-quality and diverse samples, but their success has yet to fully translate to 3D generation: existing diffusion methods can either generate low-resolution but 3D consistent outputs, or detailed 2D views of 3D objects but with potential structural defects and lacking view consistency or realism.
ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives
The discovered abstractions capture common patterns (both structural and parametric) across the dataset, so that programs rewritten with these abstractions are more compact, and expose fewer degrees of freedom.
Factored Neural Representation for Scene Understanding
A long-standing goal in scene understanding is to obtain interpretable and editable representations that can be directly constructed from a raw monocular RGB-D video, without requiring specialized hardware setup or priors.
Pix2Video: Video Editing using Image Diffusion
Our method works in two simple steps: first, we use a pre-trained structure-guided (e. g., depth) image diffusion model to perform text-guided edits on an anchor frame; then, in the key step, we progressively propagate the changes to the future frames via self-attention feature injection to adapt the core denoising step of the diffusion model.
Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Images
We investigate the problem in the context of dressed humans under the wind.
RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation
In this paper, we present RenderDiffusion, the first diffusion model for 3D generation and inference, trained using only monocular 2D supervision.
Search for Concepts: Discovering Visual Concepts Using Direct Optimization
Finding an unsupervised decomposition of an image into individual objects is a key step to leverage compositionality and to perform symbolic reasoning.
Motion Guided Deep Dynamic 3D Garments
Technically, we model garment dynamics, driven using the input character motion, by predicting per-frame local displacements in a canonical state of the garment that is enriched with frame-dependent skinning weights to bring the garment to the global space.
NeuForm: Adaptive Overfitting for Neural Shape Editing
Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis.
Video2StyleGAN: Disentangling Local and Global Variations in a Video
Image editing using a pretrained StyleGAN generator has emerged as a powerful paradigm for facial editing, providing disentangled controls over age, expression, illumination, etc.
ReLU Fields: The Little Non-linearity That Could
Although the MLPs are able to represent complex scenes with unprecedented quality and memory footprint, this expressive power of the MLPs, however, comes at the cost of long training and inference times.
Neural Convolutional Surfaces
Our pipeline and architecture are designed so that disentanglement of global geometry from local details is accomplished through optimization, in a completely unsupervised manner.
InsetGAN for Full-Body Image Generation
Instead of modeling this complex domain with a single GAN, we propose a novel method to combine multiple pretrained GANs, where one GAN generates a global canvas (e. g., human body) and a set of specialized GANs, or insets, focus on different parts (e. g., faces, shoes) that can be seamlessly inserted onto the global canvas.
ShapeFormer: Transformer-based Shape Completion via Sparse Representation
We present ShapeFormer, a transformer-based network that produces a distribution of object completions, conditioned on incomplete, and possibly noisy, point clouds.
CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions
The success of StyleGAN has enabled unprecedented semantic editing capabilities, on both synthesized and real images.
Dance In the Wild: Monocular Human Animation with Neural Dynamic Appearance Synthesis
Synthesizing dynamic appearances of humans in motion plays a central role in applications such as AR/VR and video editing.
CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds
We present Cascaded Primitive Fitting Networks (CPFN) that relies on an adaptive patch sampling network to assemble detection results of global and local primitive detection networks.
A Multi-Implicit Neural Representation for Fonts
Fonts are ubiquitous across documents and come in a variety of styles.
MoCo-Flow: Neural Motion Consensus Flow for Dynamic Humans in Stationary Monocular Cameras
In this paper, we introduce Neural Motion Consensus Flow (MoCo-Flow), a representation that models dynamic humans in stationary monocular cameras using a 4D continuous time-variant function.
Consistent Two-Flow Network for Tele-Registration of Point Clouds
In this paper, we present a learning-based technique that alleviates this problem, and allows registration between point clouds, presented in arbitrary poses, and having little or even no overlap, a setting that has been referred to as tele-registration.
Z2P: Instant Visualization of Point Clouds
We present a technique for visualizing point clouds using a neural network.
ShapeMOD: Macro Operation Discovery for 3D Shape Programs
In this paper, we present ShapeMOD, an algorithm for automatically discovering macros that are useful across large datasets of 3D shape programs.
Neural Surface Maps
Maps are arguably one of the most fundamental concepts used to define and operate on manifold surfaces in differentiable geometry.
Dynamic Neural Garments
A vital task of the wider digital human effort is the creation of realistic garments on digital avatars, both in the form of characteristic fold patterns and wrinkles in static frames as well as richness of garment dynamics under avatars' motion.
Generative Modelling of BRDF Textures from Flash Images
We learn a latent space for easy capture, consistent interpolation, and efficient reproduction of visual material appearance.
Im2Vec: Synthesizing Vector Graphics without Vector Supervision
The current alternative is to use specialized models that require explicit supervision on the vector graphics representation at training time.
Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences
From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space.
ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis
The program captures the subset of variability that is interpretable and editable.
DeformSyncNet: Deformation Transfer via Synchronized Shape Deformation Spaces
Shape deformation is an important component in any geometry processing toolbox.
Graphics
Points2Surf: Learning Implicit Surfaces from Point Cloud Patches
Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans.
Towards a Neural Graphics Pipeline for Controllable Image Generation
NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation.
Learning a Neural 3D Texture Space from 2D Exemplars
We propose a generative model of 2D and 3D natural textures with diversity, visual fidelity and at high computational efficiency.
Going Deeper with Lean Point Networks
By combining these blocks, we design wider and deeper point-based architectures.
Unsupervised Intuitive Physics from Past Experiences
We are interested in learning models of intuitive physics similar to the ones that animals use for navigation, manipulation and planning.
Unpaired Point Cloud Completion on Real Scans using Adversarial Training
As 3D scanning solutions become increasingly popular, several deep learning setups have been developed geared towards that task of scan completion, i. e., plausibly filling in regions there were missed in the raw scans.
PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds
Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers.
FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchronized GANs
The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods.
Learning a Shared Shape Space for Multimodal Garment Design
Designing real and virtual garments is becoming extremely demanding with rapidly changing fashion trends and increasing need for synthesizing realistic dressed digital humans for various applications.
Graphics
iMapper: Interaction-guided Joint Scene and Human Motion Mapping from Monocular Videos
A long-standing challenge in scene analysis is the recovery of scene arrangements under moderate to heavy occlusion, directly from monocular video.
FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs
The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods.
Graphics
Learning on the Edge: Explicit Boundary Handling in CNNs
Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding.
Joint Material and Illumination Estimation from Photo Sets in the Wild
To the other hand, methods that are automatic and work on 'in the wild' Internet images, often extract only low-frequency lighting or diffuse materials.
Graphics
PCPNET: Learning Local Shape Properties from Raw Point Clouds
In this paper, we propose PCPNet, a deep-learning based approach for estimating local 3D shape properties in point clouds.
Computational Geometry
DepthCut: Improved Depth Edge Estimation Using Multiple Unreliable Channels
In the context of scene understanding, a variety of methods exists to estimate different information channels from mono or stereo images, including disparity, depth, and normals.
Learning A Physical Long-term Predictor
Evolution has resulted in highly developed abilities in many natural intelligences to quickly and accurately predict mechanical phenomena.
SMASH: Physics-guided Reconstruction of Collisions from Videos
Authoring even two body collisions in the real world can be difficult, as one has to get timing and the object trajectories to be correctly synchronized.
RAID: A Relation-Augmented Image Descriptor
As humans, we regularly interpret images based on the relations between image regions.
Super 4PCS Fast Global Pointcloud Registration via Smart Indexing
We present Super 4PCS for global pointcloud registration that is optimal, i. e., runs in linear time (in the number of data points) and is also output sensitive in the complexity of the alignment problem based on the (unknown) overlap across scan pairs.
SmartAnnotator: An Interactive Tool for Annotating RGBD Indoor Images
Further, as more scenes are annotated, the system makes better suggestions based on structural and geometric priors learns from the previous annotation sessions.
