Search Results for author:
Found 44 papers, 19 papers with code
Leverage Cross-Attention for End-to-End Open-Vocabulary Panoptic Reconstruction
Additionally, we ensure semantic-instance segmentation consistency by fusing query-based instance segmentation probabilities with semantic probabilities in a novel panoptic head supervised by a panoptic loss.
Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation
In this work, we introduce Prometheus, a 3D-aware latent diffusion model for text-to-3D generation at both object and scene levels in seconds.
HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Driving
In the past few decades, autonomous driving algorithms have made significant progress in perception, planning, and control.
UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulation
In this work, we introduce UrbanCAD, a framework that pushes the frontier of the photorealism-controllability trade-off by generating highly controllable and photorealistic 3D vehicle digital twins from a single urban image and a collection of free 3D CAD models and handcrafted materials.
E-3DGS: Gaussian Splatting with Exposure and Motion Events
We introduce a novel integration of 3DGS with exposure events for high-quality reconstruction of explicit scene representations.
General Intelligent Imaging and Uncertainty Quantification by Deterministic Diffusion Model
Computational imaging is crucial in many disciplines from autonomous driving to life sciences.
Efficient Depth-Guided Urban View Synthesis
Recent advances in implicit scene representation enable high-fidelity street view novel view synthesis.
PanopticRecon: Leverage Open-vocabulary Instance Segmentation for Zero-shot Panoptic Reconstruction
We tackle both challenges by propagating partial labels with the aid of dense generalized features and building a 3D instance graph for associating 2D instance IDs.
Learning Temporally Consistent Video Depth from Video Diffusion Priors
Moreover, we design an effective training strategy to provide context within a clip.
$ν$-DBA: Neural Implicit Dense Bundle Adjustment Enables Image-Only Driving Scene Reconstruction
The joint optimization of the sensor trajectory and 3D map is a crucial characteristic of bundle adjustment (BA), essential for autonomous driving.
Learning 3D-Aware GANs from Unposed Images with Template Feature Field
Collecting accurate camera poses of training images has been shown to well serve the learning of 3D-aware generative adversarial networks (GANs) yet can be quite expensive in practice.
NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation
The emergence of Neural Radiance Fields (NeRF) has greatly impacted 3D scene modeling and novel-view synthesis.
REFRAME: Reflective Surface Real-Time Rendering for Mobile Devices
This work tackles the challenging task of achieving real-time novel view synthesis for reflective surfaces across various scenes.
Recent Trends in 3D Reconstruction of General Non-Rigid Scenes
Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision.
HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting
Holistic understanding of urban scenes based on RGB images is a challenging yet important problem.
DORec: Decomposed Object Reconstruction and Segmentation Utilizing 2D Self-Supervised Features
Recovering 3D geometry and textures of individual objects is crucial for many robotics applications, such as manipulation, pose estimation, and autonomous driving.
PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes
Our key insight lies in exploiting the complementarity of 3D and 2D priors to mutually enhance geometry and semantics.
VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis
Unsupervised learning of 3D-aware generative adversarial networks has lately made much progress.
MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving
We expect this modular design to boost academic progress and industrial deployment of NeRF-based autonomous driving simulation.
UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields
Generating photorealistic images with controllable camera pose and scene contents is essential for many applications including AR/VR and simulation.
RICO: Regularizing the Unobservable for Indoor Compositional Reconstruction
Recently, neural implicit surfaces have become popular for multi-view reconstruction.
Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask
We introduce a novel approach that takes a single semantic mask as input to synthesize multi-view consistent color images of natural scenes, trained with a collection of single images from the Internet.
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set.
SteerNeRF: Accelerating NeRF Rendering via Smooth Viewpoint Trajectory
Neural Radiance Fields (NeRF) have demonstrated superior novel view synthesis performance but are slow at rendering.
Deep Generative Models on 3D Representations: A Survey
In this survey, we thoroughly review the ongoing developments of 3D generative models, including methods that employ 2D and 3D supervision.
RING++: Roto-translation Invariant Gram for Global Localization on a Sparse Scan Map
In addition, we derive sufficient conditions of feature extractors for the representation preserving the roto-translation invariance, making RING++ a framework applicable to generic multi-channel features.
VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids
State-of-the-art 3D-aware generative models rely on coordinate-based MLPs to parameterize 3D radiance fields.
DPCN++: Differentiable Phase Correlation Network for Versatile Pose Registration
Next, the rotation, scale, and translation are independently and efficiently estimated in the spectrum step-by-step using the DPC solver.
Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation
In this work, we present a novel 3D-to-2D label transfer method, Panoptic NeRF, which aims for obtaining per-pixel 2D semantic and instance labels from easy-to-obtain coarse 3D bounding primitives.
A Visual Navigation Perspective for Category-Level Object Pose Estimation
In this paper, we take a deeper look at the inference of analysis-by-synthesis from the perspective of visual navigation, and investigate what is a good navigation policy for this specific task.
On the Frequency Bias of Generative Models
2) Checkerboard artifacts introduced by upsampling cannot explain the spectral discrepancies alone as the generator is able to compensate for these artifacts.
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D
For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other.
Learning Interpretable BEV Based VIO without Deep Neural Networks
Specifically, we first adopt Unscented Kalman Filter as a differentiable layer to predict the pitch and roll, where the covariance matrices of noise are learned to filter out the noise of the IMU raw data.
Shape As Points: A Differentiable Poisson Solver
However, the implicit nature of neural implicit representations results in slow inference time and requires careful initialization.
SMD-Nets: Stereo Mixture Density Networks
Despite stereo matching accuracy has greatly improved by deep learning in the last few years, recovering sharp boundaries and high-resolution outputs efficiently remains challenging.
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
NeRF synthesizes novel views of a scene with unprecedented quality by fitting a neural radiance field to RGB images.
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
In contrast to voxel-based representations, radiance fields are not confined to a coarse discretization of the 3D space, yet allow for disentangling camera and scene properties while degrading gracefully in the presence of reconstruction ambiguity.
Ranked #2 on
Scene Generation
on VizDoom
Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis
We define the new task of 3D controllable image synthesis and propose an approach for solving it by reasoning both in 3D space and in the 2D image domain.
Deep Marching Cubes: Learning Explicit Surface Representations
Existing learning based solutions to 3D surface prediction cannot be trained end-to-end as they operate on intermediate representations (e. g., TSDF) from which 3D surface meshes must be extracted in a post-processing step (e. g., via the marching cubes algorithm).
On the Integration of Optical Flow and Action Recognition
Here we take a deeper look at the combination of flow and action recognition, and investigate why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better.
Parse Geometry from a Line: Monocular Depth Estimation with Partial Laser Observation
Many standard robotic platforms are equipped with at least a fixed 2D laser range finder and a monocular camera.
Understand Scene Categories by Objects: A Semantic Regularized Scene Classifier Using Convolutional Neural Networks
As scene images have larger diversity than the iconic object images, it is more challenging for deep learning methods to automatically learn features from scene images with less samples.
Place classification with a graph regularized deep neural network model
Furthermore, results show that the features automatically learned from the raw input range data can achieve competitive results to the features constructed based on statistical and geometrical information.
Image Representation Learning Using Graph Regularized Auto-Encoders
We consider the problem of image representation for the tasks of unsupervised learning and semi-supervised learning.
