Search Results for author:
Found 114 papers, 49 papers with code
PatchRefiner V2: Fast and Lightweight Real-Domain High-Resolution Metric Depth Estimation
While current high-resolution depth estimation methods achieve strong results, they often suffer from computational inefficiencies due to reliance on heavyweight models and multiple inference steps, increasing inference time.
PrEditor3D: Fast and Precise 3D Shape Editing
The edited 3D mesh aligns well with the prompts, and remains identical for regions that are not intended to be altered.
ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models
Experimental evaluations demonstrate that our approach achieves competitive performance on standard benchmarks compared to supervised methods, despite not requiring any 3D keypoint annotations during training.
4Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusion
We propose a novel two-stream architecture.
Amodal Depth Anything: Amodal Depth Estimation in the Wild
Amodal depth estimation aims to predict the depth of occluded (invisible) parts of objects in a scene.
T-3DGS: Removing Transient Objects for 3D Scene Reconstruction
We propose a novel framework to remove transient objects from input videos for 3D scene reconstruction using Gaussian Splatting.
Geometry Distributions
Neural representations of 3D data have been widely adopted across various applications, particularly in recent work leveraging coordinate-based networks to model scalar or vector fields.
StereoCrafter-Zero: Zero-Shot Stereo Video Generation with Noisy Restart
Generating high-quality stereo videos that mimic human binocular vision requires maintaining consistent depth perception and temporal coherence across frames.
LaGeM: A Large Geometry Model for 3D Representation Learning and Diffusion
Different from previous approaches that only work on a regular image or volume grid, our hierarchical autoencoder operates on unordered sets of vectors.
ImmersePro: End-to-End Stereo Video Synthesis Via Implicit Disparity Learning
We introduce \textit{ImmersePro}, an innovative framework specifically designed to transform single-view videos into stereo videos.
Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation
Nevertheless, existing approaches for layout control are limited to 2D layouts, require the user to provide a static layout beforehand, and fail to preserve generated images under layout changes.
A3D: Does Diffusion Dream about 3D Alignment?
In order to achieve the alignment of the corresponding parts of the generated objects, we propose to embed these objects into a common latent space and optimize the continuous transitions between these objects.
VIA: Unified Spatiotemporal Video Adaptation Framework for Global and Local Video Editing
Video editing is a cornerstone of digital media, from entertainment and education to professional communication.
Vivid-ZOO: Multi-View Video Generation with Diffusion Model
To this end, we propose a novel diffusion-based pipeline that generates high-quality multi-view videos centered around a dynamic 3D object from text.
PatchRefiner: Leveraging Synthetic Data for Real-Domain High-Resolution Monocular Metric Depth Estimation
This paper introduces PatchRefiner, an advanced framework for metric single image depth estimation aimed at high-resolution real-domain inputs.
E$^3$-Net: Efficient E(3)-Equivariant Normal Estimation Network
To address this issue, we propose E3-Net to achieve equivariance for normal estimation.
Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Models
We introduce the first zero-shot approach for Video Semantic Segmentation (VSS) based on pre-trained diffusion models.
PS-CAD: Local Geometry Guidance via Prompting and Selection for CAD Reconstruction
At each step, we provide two forms of geometric guidance.
LASPA: Latent Spatial Alignment for Fast Training-free Single Image Editing
We present a novel, training-free approach for textual editing of real images using diffusion models.
AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning
To the best of our knowledge, our approach is the first to introduce multi-modal conditioning to 3D avatar generation and editing.
Deep Learning-based Image and Video Inpainting: A Survey
The goal of this paper is to comprehensively review the deep learning-based methods for image and video inpainting.
WinSyn: : A High Resolution Testbed for Synthetic Data
We evaluate a procedural model by training semantic segmentation networks on both synthetic and real images and then comparing their performances on a shared test set of real images.
EVP: Enhanced Visual Perception using Inverse Multi-Attentive Feature Refinement and Regularized Image-Text Alignment
Second, we propose a novel image-text alignment module for improved feature extraction of the Stable Diffusion backbone.
Ranked #1 on
Depth Estimation
on NYU-Depth V2
LatentMan: Generating Consistent Animated Characters using Image Diffusion Models
We strive to bridge this gap, and we introduce LatentMan, which leverages existing text-based motion diffusion models to generate diverse continuous motions to guide the T2I model.
VoxelKP: A Voxel-based Network Architecture for Human Keypoint Estimation in LiDAR Data
To the best of our knowledge, \textit{VoxelKP} is the first single-staged, fully sparse network that is specifically designed for addressing the challenging task of 3D keypoint estimation from LiDAR data, achieving state-of-the-art performances.
Ranked #1 on
3D Human Pose Estimation
on Waymo Open Dataset
NeuSD: Surface Completion with Multi-View Text-to-Image Diffusion
We present a novel method for 3D surface reconstruction from multiple images where only a part of the object of interest is captured.
LooseControl: Lifting ControlNet for Generalized Depth Conditioning
We present LooseControl to allow generalized depth conditioning for diffusion-based image generation.
PatchFusion: An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimation
Single image depth estimation is a foundational task in computer vision and generative modeling.
4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling
Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.
Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features
In this work, we propose to explore foundation models for the task of keypoint detection on 3D shapes.
Functional Diffusion
We propose a new class of generative diffusion models, called functional diffusion.
3DCoMPaT$^{++}$: An improved Large-scale 3D Vision Dataset for Compositional Recognition
In this work, we present 3DCoMPaT$^{++}$, a multimodal 2D/3D dataset with 160 million rendered views of more than 10 million stylized 3D shapes carefully annotated at the part-instance level, alongside matching RGB point clouds, 3D textured meshes, depth maps, and segmentation masks.
LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts
Diffusion-based generative models have significantly advanced text-to-image generation but encounter challenges when processing lengthy and intricate text prompts describing complex scenes with multiple objects.
State of the Art on Diffusion Models for Visual Computing
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.
WinSyn: A High Resolution Testbed for Synthetic Data
We evaluate a procedural model by training semantic segmentation networks on both synthetic and real images and then comparing their performances on a shared test set of real images.
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors.
Zero-Shot 3D Shape Correspondence
Instead, we propose to exploit the in-context learning capabilities of ChatGPT to generate two different sets of semantic regions for each shape and a semantic mapping between them.
InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions
In this work, we propose a framework termed InstructEdit that can do fine-grained editing based on user instructions.
Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects
We develop a method that recovers the surface, materials, and illumination of a scene from its posed multi-view images.
PET-NeuS: Positional Encoding Tri-Planes for Neural Surfaces
The first component is to borrow the tri-plane representation from EG3D and represent signed distance fields as a mixture of tri-planes and MLPs instead of representing it with MLPs only.
SATR: Zero-Shot Semantic Segmentation of 3D Shapes
We explore the task of zero-shot semantic segmentation of 3D shapes by using large-scale off-the-shelf 2D image recognition models.
MDP: A Generalized Framework for Text-Guided Image Editing by Manipulating the Diffusion Path
Image generation using diffusion can be controlled in multiple ways.
VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs
Our experiments demonstrate that VIVE3D generates high-fidelity face edits at consistent quality from a range of camera viewpoints which are composited with the original video in a temporally and spatially consistent manner.
BlobGAN-3D: A Spatially-Disentangled 3D-Aware Generative Model for Indoor Scenes
3D-aware image synthesis has attracted increasing interest as it models the 3D nature of our real world.
3D generation on ImageNet
Existing 3D-from-2D generators are typically designed for well-curated single-category datasets, where all the objects have (approximately) the same scale, 3D location, and orientation, and the camera always points to the center of the scene.
Dissolving Is Amplifying: Towards Fine-Grained Anomaly Detection
DIA is a fine-grained anomaly detection framework for medical images.
ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth
Finally, ZoeD-M12-NK is the first model that can jointly train on multiple datasets (NYU Depth v2 and KITTI) without a significant drop in performance and achieve unprecedented zero-shot generalization performance to eight unseen datasets from both indoor and outdoor domains.
Ranked #23 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models
We introduce 3DShape2VecSet, a novel shape representation for neural fields designed for generative diffusion models.
3DAvatarGAN: Bridging Domains for Personalized Editable Avatars
Finally, we propose a novel inversion method for 3D-GANs linking the latent spaces of the source and the target domains.
ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes
The two popular datasets ScanRefer [16] and ReferIt3D [3] connect natural language to real-world 3D data.
GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network
The major improvement over PSMNet comes from a novel Geometry-aware Panorama Registration Network or GPR-Net that effectively tackles the wide baseline registration problem by exploiting the layout geometry and computing fine-grained correspondences on the layout boundaries, instead of the global pixel-space.
Large-Scale Auto-Regressive Modeling Of Street Networks
We present a novel generative method for the creation of city-scale road layouts.
Learning to Construct 3D Building Wireframes from 3D Line Clouds
The network takes a line cloud as input , i. e., a nonstructural and unordered set of 3D line segments extracted from multi-view images, and outputs a 3D wireframe of the underlying building, which consists of a sparse set of 3D junctions connected by line segments.
Assesment of material layers in building walls using GeoRadar
Assessing the structure of a building with non-invasive methods is an important problem.
EpiGRAF: Rethinking training of 3D GANs
In this work, we show that it is possible to obtain a high-resolution 3D generator with SotA image quality by following a completely different route of simply training the model patch-wise.
HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details
We develop HF-NeuS, a novel method to improve the quality of surface reconstruction in neural rendering.
Gaussian Blue Noise
Among the various approaches for producing point distributions with blue noise spectrum, we argue for an optimization framework using Gaussian kernels.
RLSS: A Deep Reinforcement Learning Algorithm for Sequential Scene Generation
We present RLSS: a reinforcement learning algorithm for sequential scene generation.
COFS: Controllable Furniture layout Synthesis
Scalable generation of furniture layouts is essential for many applications in virtual reality, augmented reality, game development and synthetic data generation.
Video2StyleGAN: Disentangling Local and Global Variations in a Video
Image editing using a pretrained StyleGAN generator has emerged as a powerful paradigm for facial editing, providing disentangled controls over age, expression, illumination, etc.
3DILG: Irregular Latent Grids for 3D Generative Modeling
All probabilistic experiments confirm that we are able to generate detailed and high quality shapes to yield the new state of the art in generative 3D shape modeling.
Differentiable Electron Microscopy Simulation: Methods and Applications for Visualization
We propose a new microscopy simulation system that can depict atomistic models in a micrograph visual style, similar to results of physical electron microscopy imaging.
LocalBins: Improving Depth Estimation by Learning Local Distributions
We build on AdaBins which estimates a global distribution of depth values for the input image and evolve the architecture in two ways.
Ranked #43 on
Monocular Depth Estimation
on NYU-Depth V2
InsetGAN for Full-Body Image Generation
Instead of modeling this complex domain with a single GAN, we propose a novel method to combine multiple pretrained GANs, where one GAN generates a global canvas (e. g., human body) and a set of specialized GANs, or insets, focus on different parts (e. g., faces, shoes) that can be seamlessly inserted onto the global canvas.
On the Robustness of Quality Measures for GANs
Finally, we show that the FID can be robustified by simply replacing the standard Inception with a robust Inception.
CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions
The success of StyleGAN has enabled unprecedented semantic editing capabilities, on both synthesized and real images.
Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networks
The input to our method is trained GAN that can produce images in domain A and a single reference image I_B from domain B.
Flow-Guided Video Inpainting with Scene Templates
We consider the problem of filling in missing spatio-temporal regions of a video.
Fast Sinkhorn Filters: Using Matrix Scaling for Non-Rigid Shape Correspondence With Functional Maps
In this paper, we provide a theoretical foundation for pointwise map recovery from functional maps and highlight its relation to a range of shape correspondence methods based on spectral alignment.
SketchGen: Generating Constrained CAD Sketches
Sketches can be represented as graphs, with the primitives as nodes and the constraints as edges.
Self-Supervised Learning of Domain Invariant Features for Depth Estimation
Specifically, we extend self-supervised learning from traditional representation learning, which works on images from a single domain, to domain invariant representation learning, which works on images from two different domains by utilizing an image-to-image translation network.
Barbershop: GAN-based Image Compositing using Segmentation Masks
Seamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion which cause coupling between different parts of the image.
Finding Nano-Ötzi: Semi-Supervised Volume Visualization for Cryo-Electron Tomography
Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques.
Labels4Free: Unsupervised Segmentation using StyleGAN
We propose an unsupervised segmentation framework for StyleGAN generated objects.
IntraTomo: Self-Supervised Learning-Based Tomography via Sinogram Synthesis and Prediction
After getting estimated through the sinogram prediction module, the density field is consistently refined in the second module using local and non-local geometrical priors.
Ranked #6 on
Novel View Synthesis
on X3D
Computed Tomography (CT)
Low-Dose X-Ray Ct Reconstruction
+3
Improved StyleGAN Embedding: Where are the Good Latents?
First, we introduce a new normalized space to analyze the diversity and the quality of the reconstructed latent codes.
AdaBins: Depth Estimation using Adaptive Bins
We address the problem of estimating a high quality dense depth map from a single RGB input image.
Ranked #7 on
Depth Estimation
on NYU-Depth V2
Training Data Generating Networks: Shape Reconstruction via Bi-level Optimization
We combine training data generating networks with bi-level optimization algorithms to obtain a complete framework for which all components can be jointly trained.
Channel-Directed Gradients for Optimization of Convolutional Neural Networks
We introduce optimization methods for convolutional neural networks that can be used to improve existing gradient-based optimization in terms of generalization error.
StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows
We evaluate our method using the face and the car latent space of StyleGAN, and demonstrate fine-grained disentangled edits along various attributes on both real photographs and StyleGAN generated images.
Disentangled Image Generation Through Structured Noise Injection
We show that disentanglement in the first layer of the generator network leads to disentanglement in the generated image.
MGCN: Descriptor Learning using Multiscale GCNs
Second, we propose a new multiscale graph convolutional network (MGCN) to transform a non-learned feature to a more discriminative descriptor.
Point Cloud Instance Segmentation using Probabilistic Embeddings
In this paper we propose a new framework for point cloud instance segmentation.
Ranked #1 on
Instance Segmentation
on PartNet
SEAN: Image Synthesis with Semantic Region-Adaptive Normalization
Using SEAN normalization, we can build a network architecture that can control the style of each semantic region individually, e. g., we can specify one style reference image per region.
Image2StyleGAN++: How to Edit the Embedded Images?
We propose Image2StyleGAN++, a flexible image editing framework with many applications.
StructEdit: Learning Structural Shape Variations
Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation.
Manhattan Room Layout Reconstruction from a Single 360 image: A Comparative Study of State-of-the-art Methods
Recent approaches for predicting layouts from 360 panoramas produce excellent results.
Photorealistic Material Editing Through Direct Image Manipulation
In the proposed workflow, the user starts with an input image and applies a few intuitive transforms (e. g., colorization, image inpainting) within a 2D image editor of their choice, and in the next step, our technique produces a photorealistic result that approximates this target image.
StructureNet: Hierarchical Graph Networks for 3D Shape Generation
We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families; and (iii) can be used to generate a great diversity of realistic structured shape geometries.
TileGAN: Synthesis of Large-Scale Non-Homogeneous Textures
We tackle the problem of texture synthesis in the setting where many input images are given and a large-scale output is required.
ZoomOut: Spectral Upsampling for Efficient Shape Correspondence
Our main observation is that high quality maps can be obtained even if the input correspondences are noisy or are encoded by a small number of coefficients in a spectral basis.
Graphics
Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
We propose an efficient algorithm to embed a given image into the latent space of StyleGAN.
High Quality Monocular Depth Estimation via Transfer Learning
Accurate depth estimation from images is a fundamental task in many applications including scene understanding and reconstruction.
Ranked #49 on
Monocular Depth Estimation
on KITTI Eigen split
Latent Filter Scaling for Multimodal Unsupervised Image-to-Image Translation
In multimodal unsupervised image-to-image translation tasks, the goal is to translate an image from the source domain to many images in the target domain.
DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama
We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama.
How does Lipschitz Regularization Influence GAN Training?
In this work, we uncover an even more important effect of Lipschitz regularization by examining its impact on the loss function: It degenerates GAN loss functions to almost linear ones by restricting their domain and interval of attainable gradient values.
FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchronized GANs
The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods.
Super-Resolution and Sparse View CT Reconstruction
As a second, smaller contribution, we also show that when using such a proximal reconstruction framework, it is beneficial to employ the Simultaneous Algebraic Reconstruction Technique (SART) instead of the commonly used Conjugate Gradient (CG) method in the solution of the data term proximal operator.
FrankenGAN: Guided Detail Synthesis for Building Mass-Models Using Style-Synchonized GANs
The various GANs are synchronized to produce consistent style distributions over buildings and neighborhoods.
Graphics
Facade Segmentation in the Wild
Specifically, we propose a MULTIFACSEGNET architecture to assign multiple labels to each pixel, a SEPARABLE architecture as a low-rank formulation that encourages extraction of rectangular elements, and a COMPATIBILITY network that simultaneously seeks segmentation across facade element types allowing the network to 'see' intermediate output probabilities of the various facade element classes.
Gaussian Material Synthesis
Workflow timings against Disney's "principled" shader reveal that our system scales well with the number of sought materials, thus empowering even novice users to generate hundreds of high-quality material models without any expertise in material modeling.
Supervised Convolutional Sparse Coding
Convolutional Sparse Coding (CSC) is a well-established image representation model especially suited for image restoration tasks.
Integration of Absolute Orientation Measurements in the KinectFusion Reconstruction pipeline
We also present a technique to filter the pairs of 3D matched points based on the distribution of their distances.
PolyFit: Polygonal Surface Reconstruction From Point Clouds
We show that reconstruction from point clouds can be cast as a binary labeling problem.
Fast Convolutional Sparse Coding in the Dual Domain
Convolutional sparse coding (CSC) is an important building block of many computer vision applications ranging from image and video compression to deep learning.
TRex: A Tomography Reconstruction Proximal Framework for Robust Sparse View X-Ray Applications
We present TRex, a flexible and robust Tomographic Reconstruction framework using proximal algorithms.
Intrinsic Scene Decomposition From RGB-D images
In this paper, we address the problem of computing an intrinsic decomposition of the colors of a surface into an albedo and a shading term.
RAID: A Relation-Augmented Image Descriptor
As humans, we regularly interpret images based on the relations between image regions.
Scaling SVM and Least Absolute Deviations via Exact Data Reduction
Some appealing features of our screening method are: (1) DVI is safe in the sense that the vectors discarded by DVI are guaranteed to be non-support vectors; (2) the data set needs to be scanned only once to run the screening, whose computational cost is negligible compared to that of solving the SVM problem; (3) DVI is independent of the solvers and can be integrated with any existing efficient solvers.
A Safe Screening Rule for Sparse Logistic Regression
The l1-regularized logistic regression (or sparse logistic regression) is a widely used method for simultaneous classification and feature selection.
Lasso Screening Rules via Dual Polytope Projection
To improve the efficiency of solving large-scale Lasso problems, El Ghaoui and his colleagues have proposed the SAFE rules which are able to quickly identify the inactive predictors, i. e., predictors that have $0$ components in the solution vector.
Fused Multiple Graphical Lasso
We expect the two brain networks for NC and MCI to share common structures but not to be identical to each other; similarly for the two brain networks for MCI and AD.
Multi-Stage Dantzig Selector
We show that if $X$ obeys a certain condition, then with a large probability the difference between the solution $\hat\beta$ estimated by the proposed method and the true solution $\beta^*$ measured in terms of the $l_p$ norm ($p\geq 1$) is bounded as \begin{equation*} \|\hat\beta-\beta^*\|_p\leq \left(C(s-N)^{1/p}\sqrt{\log m}+\Delta\right)\sigma, \end{equation*} $C$ is a constant, $s$ is the number of nonzero entries in $\beta^*$, $\Delta$ is independent of $m$ and is much smaller than the first term, and $N$ is the number of entries of $\beta^*$ larger than a certain value in the order of $\mathcal{O}(\sigma\sqrt{\log m})$.
