Search Results for author:
Found 102 papers, 58 papers with code
PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes
Moreover, PanopticNeRF-360 enables omnidirectional rendering of high-fidelity, multi-view and spatiotemporally consistent appearance, semantic and instance labels.
On Offline Evaluation of 3D Object Detection for Autonomous Driving
Prior work in 3D object detection evaluates models using offline metrics like average precision since closed-loop online evaluation on the downstream driving task is costly.
End-to-end Autonomous Driving: Challenges and Frontiers
The autonomous driving community has witnessed a rapid growth in approaches that embrace an end-to-end algorithm framework, utilizing raw sensor input to generate vehicle motion plans, instead of concentrating on individual tasks such as detection and motion prediction.
Hidden Biases of End-to-End Driving Models
End-to-end driving systems have recently made rapid progress, in particular on CARLA.
Parting with Misconceptions about Learning-based Vehicle Motion Planning
The release of nuPlan marks a new era in vehicle motion planning research, offering the first large-scale real-world dataset and evaluation schemes requiring both precise short-term planning and long-horizon ego-forecasting.
Towards Scalable Multi-View Reconstruction of Geometry and Materials
In this paper, we propose a novel method for joint recovery of camera pose, object geometry and spatially-varying Bidirectional Reflectance Distribution Function (svBRDF) of 3D scenes that exceed object-scale and hence cannot be captured with stationary light stages.
AG3D: Learning to Generate 3D Avatars from 2D Image Collections
The key to progress is hence to learn generative models of 3D avatars from abundant unstructured 2D image collections.
MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes
We design a lossless procedure for baking the parameterization used during training into a model that achieves real-time rendering while still preserving the photorealistic view synthesis quality of a volumetric radiance field.
NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM
Neural implicit representations have recently become popular in simultaneous localization and mapping (SLAM), especially in dense visual SLAM.
Factor Fields: A Unified Framework for Neural Fields and Beyond
Our experiments show that DiF leads to improvements in approximation quality, compactness, and training time when compared to previous fast reconstruction methods.
StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis
Text-to-image synthesis has recently seen significant progress thanks to large pretrained language models, large-scale training data, and the introduction of scalable model families such as diffusion and autoregressive models.
Ranked #18 on
Text-to-Image Generation
on COCO
GOOD: Exploring Geometric Cues for Detecting Objects in an Open World
We address the task of open-world class-agnostic object detection, i. e., detecting every object in an image by learning from a limited number of base object classes.
Ranked #1 on
Open World Object Detection
on COCO VOC to non-VOC
Fast-SNARF: A Fast Deformer for Articulated Neural Fields
A key challenge in making such methods applicable to articulated objects, such as the human body, is to model the deformation of 3D locations between the rest pose (a canonical space) and the deformed space.
Unifying Flow, Stereo and Depth Estimation
We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images.
Ranked #1 on
Optical Flow Estimation
on Sintel-clean
NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields
Visually exploring in a real-world 4D spatiotemporal space freely in VR has been a long-term quest.
Deep Generative Models on 3D Representations: A Survey
In this survey, we thoroughly review the ongoing developments of 3D generative models, including methods that employ 2D and 3D supervision.
PlanT: Explainable Planning Transformers via Object-Level Representations
Planning an optimal route in a complex environment requires efficient reasoning about the surrounding scene.
ARAH: Animatable Volume Rendering of Articulated Human SDFs
We demonstrate that our proposed pipeline can generate clothed avatars with high-quality pose-dependent geometry and appearance from a sparse set of multi-view RGB videos.
VoxGRAF: Fast 3D-Aware Image Synthesis with Sparse Voxel Grids
State-of-the-art 3D-aware generative models rely on coordinate-based MLPs to parameterize 3D radiance fields.
MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction
Motivated by recent advances in the area of monocular geometry prediction, we systematically explore the utility these cues provide for improving neural implicit surface reconstruction.
TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving
At the time of submission, TransFuser outperforms all prior work on the CARLA leaderboard in terms of driving score by a large margin.
Ranked #4 on
Autonomous Driving
on CARLA Leaderboard
KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients
Simulators offer the possibility of safe, low-cost development of self-driving systems.
Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation
In this work, we present a novel 3D-to-2D label transfer method, Panoptic NeRF, which aims for obtaining per-pixel 2D semantic and instance labels from easy-to-obtain coarse 3D bounding primitives.
TensoRF: Tensorial Radiance Fields
We demonstrate that applying traditional CP decomposition -- that factorizes tensors into rank-one components with compact vectors -- in our framework leads to improvements over vanilla NeRF.
PINA: Learning a Personalized Implicit Neural Avatar from a Single RGB-D Video Sequence
We present a novel method to learn Personalized Implicit Neural Avatars (PINA) from a short RGB-D sequence.
StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets
StyleGAN in particular sets new standards for generative modeling regarding image quality and controllability.
Ranked #1 on
Image Generation
on ImageNet 512x512
gDNA: Towards Generative Detailed Neural Avatars
Furthermore, we show that our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.
RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs
We observe that the majority of artifacts in sparse input scenarios are caused by errors in the estimated scene geometry, and by divergent behavior at the start of training.
On the Frequency Bias of Generative Models
2) Checkerboard artifacts introduced by upsampling cannot explain the spectral discrepancies alone as the generator is able to compensate for these artifacts.
Projected GANs Converge Faster
Generative Adversarial Networks (GANs) produce high-quality images but are challenging to train.
Ranked #1 on
Image Generation
on Stanford Dogs
ATISS: Autoregressive Transformers for Indoor Scene Synthesis
The ability to synthesize realistic and diverse indoor furniture layouts automatically or based on partial input, unlocks many applications, from better interactive 3D tools to data synthesis for training and simulation.
KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D
For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other.
NEAT: Neural Attention Fields for End-to-End Autonomous Driving
Efficient reasoning about the semantic, spatial, and temporal structure of a scene is a crucial prerequisite for autonomous driving.
Ranked #10 on
Autonomous Driving
on CARLA Leaderboard
Learning Cascaded Detection Tasks with Weakly-Supervised Domain Adaptation
In order to handle the challenges of autonomous driving, deep learning has proven to be crucial in tackling increasingly complex tasks, such as 3D detection or instance segmentation.
MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
In contrast, we propose an approach that can quickly generate realistic clothed human avatars, represented as controllable neural SDFs, given only monocular depth images.
Shape As Points: A Differentiable Poisson Solver
However, the implicit nature of neural implicit representations results in slow inference time and requires careful initialization.
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
At the same time, neural radiance fields have revolutionized novel view synthesis.
Multi-Modal Fusion Transformer for End-to-End Autonomous Driving
How should representations from complementary sensors be integrated for autonomous driving?
Ranked #1 on
Autonomous Driving
on Town05 Short
Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration
We combine PTF with multi-class occupancy networks, obtaining a novel learning-based framework that learns to simultaneously predict shape and per-point correspondences between the posed space and the canonical space for clothed human.
SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes
However, this is problematic since the backward warp field is pose dependent and thus requires large amounts of data to learn.
SMD-Nets: Stereo Mixture Density Networks
Despite stereo matching accuracy has greatly improved by deep learning in the last few years, recovering sharp boundaries and high-resolution outputs efficiently remains challenging.
CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fields
At test time, our model generates images with explicit control over the camera as well as the shape and appearance of the scene.
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
NeRF synthesizes novel views of a scene with unprecedented quality by fitting a neural radiance field to RGB images.
Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks
The INN allows us to compute the inverse mapping of the homeomorphism, which in turn, enables the efficient computation of both the implicit surface function of a primitive and its mesh, without any additional post-processing.
STEP: Segmenting and Tracking Every Pixel
The task of assigning semantic classes and track identities to every pixel in a video is called video panoptic segmentation.
Counterfactual Generative Networks
Prior works on image classification show that instead of learning a connection to object shape, deep classifiers tend to exploit spurious correlations with low-level texture or the background for solving the classification task.
SLIM: Self-Supervised LiDAR Scene Flow and Motion Segmentation
Recently, several frameworks for self-supervised learning of 3D scene flow on point clouds have emerged.
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
While several recent works investigate how to disentangle underlying factors of variation in the data, most of them operate in 2D and hence ignore that our world is three-dimensional.
HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking
Multi-Object Tracking (MOT) has been notoriously difficult to evaluate.
Category Level Object Pose Estimation via Neural Analysis-by-Synthesis
Many object pose estimation algorithms rely on the analysis-by-synthesis framework which requires explicit representations of individual object instances.
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
In contrast to voxel-based representations, radiance fields are not confined to a coarse discretization of the 3D space, yet allow for disentangling camera and scene properties while degrading gracefully in the presence of reconstruction ambiguity.
Ranked #2 on
Scene Generation
on VizDoom
Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
Neural rendering techniques promise efficient photo-realistic image synthesis while at the same time providing rich control over scene parameters by learning the physical image formation process.
Benchmarking Unsupervised Object Representations for Video Sequences
Perceiving the world in terms of objects and tracking them through time is a crucial prerequisite for reasoning and scene understanding.
Learning Neural Light Transport
In recent years, deep generative models have gained significance due to their ability to synthesize natural-looking images with applications ranging from virtual reality to data augmentation for training computer vision models.
Label Efficient Visual Abstractions for Autonomous Driving
Beyond label efficiency, we find several additional training benefits when leveraging visual abstractions, such as a significant reduction in the variance of the learned policy when compared to state-of-the-art end-to-end driving models.
Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image
Humans perceive the 3D world as a set of distinct objects that are characterized by various low-level (geometry, reflectance) and high-level (connectivity, adjacency, symmetry) properties.
Learning Implicit Surface Light Fields
In this work, we propose a novel implicit representation for capturing the visual appearance of an object in terms of its surface light field.
Convolutional Occupancy Networks
Recently, implicit neural representations have gained popularity for learning-based 3D reconstruction.
Self-Supervised Linear Motion Deblurring
Motion blurry images challenge many computer vision algorithms, e. g, feature detection, motion estimation, or object recognition.
Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision
In this work, we propose a differentiable rendering formulation for implicit shape and texture representations.
Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis
We define the new task of 3D controllable image synthesis and propose an approach for solving it by reasoning both in 3D space and in the 2D image domain.
Attacking Optical Flow
In this paper, we extend adversarial patch attacks to optical flow networks and show that such attacks can compromise their performance.
Texture Fields: Learning Texture Representations in Function Space
A major reason for these limitations is that common representations of texture are inefficient or hard to interface for modern deep learning techniques.
Robust Dense Mapping for Large-Scale Dynamic Environments
We use both instance-aware semantic segmentation and sparse scene flow to classify objects as either background, moving, or potentially moving, thereby ensuring that the system is able to model objects with the potential to transition from static to dynamic, such as parked cars.
Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids
Abstracting complex 3D shapes with parsimonious part-based representations has been a long standing goal in computer vision.
MOTS: Multi-Object Tracking and Segmentation
This paper extends the popular task of multi-object tracking to multi-object tracking and segmentation (MOTS).
Ranked #6 on
Multi-Object Tracking
on MOTS20
Multi-Object Tracking
Multi-Object Tracking and Segmentation
+1
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
RayNet integrates a CNN that learns view-invariant feature representations with an MRF that explicitly encodes the physics of perspective projection and occlusion.
Taking a Deeper Look at the Inverse Compositional Algorithm
In this paper, we provide a modern synthesis of the classic inverse compositional algorithm for dense image alignment.
Occupancy Networks: Learning 3D Reconstruction in Function Space
With the advent of deep neural networks, learning-based approaches for 3D reconstruction have gained popularity.
Geometric Image Synthesis
The task of generating natural images from 3D scenes has been a long standing goal in computer graphics.
SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images
Omnidirectional cameras offer great benefits over classical cameras wherever a wide field of view is essential, such as in virtual reality applications or in autonomous robots.
Unsupervised Learning of Multi-Frame Optical Flow with Occlusions
In this paper, we propose a framework for unsupervised learning of optical flow and occlusions over multiple frames.
Learning Priors for Semantic 3D Reconstruction
In contrast to existing variational methods for semantic 3D reconstruction, our model is end-to-end trainable and captures more complex dependencies between the semantic labels and the 3D geometry.
Conditional Affordance Learning for Driving in Urban Environments
Most existing approaches to autonomous driving fall into one of two categories: modular pipelines, that build an extensive model of the environment, and imitation learning approaches, that map images directly to control outputs.
PointFlowNet: Learning Representations for Rigid Motion Estimation from Point Clouds
In this paper, we propose to estimate 3D motion from such unstructured point clouds using a deep neural network.
Deep Marching Cubes: Learning Explicit Surface Representations
Existing learning based solutions to 3D surface prediction cannot be trained end-to-end as they operate on intermediate representations (e. g., TSDF) from which 3D surface meshes must be extracted in a post-processing step (e. g., via the marching cubes algorithm).
Learning 3D Shape Completion From Laser Scan Data With Weak Supervision
Learning-based approaches, in contrast, avoid the expensive optimization step and instead directly predict the complete shape from the incomplete observations using deep neural networks.
Learning 3D Shape Completion under Weak Supervision
We address the problem of 3D shape completion from sparse and noisy point clouds, a fundamental problem in computer vision and robotics.
Which Training Methods for GANs do actually Converge?
In this paper, we show that the requirement of absolute continuity is necessary: we describe a simple yet prototypical counterexample showing that in the more realistic case of distributions that are not absolutely continuous, unregularized GAN training is not always convergent.
On the Integration of Optical Flow and Action Recognition
Here we take a deeper look at the combination of flow and action recognition, and investigate why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better.
Semantic Visual Localization
Robust visual localization under a wide range of viewing conditions is a fundamental problem in computer vision.
Bounding Boxes, Segmentations and Object Coordinates: How Important Is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios?
Existing methods for 3D scene flow estimation often fail in the presence of large displacement or local ambiguities, e. g., at texture-less or reflective surfaces.
Sparsity Invariant CNNs
In this paper, we consider convolutional neural networks operating on sparse inputs with an application to depth upsampling from sparse laser scan data.
Ranked #16 on
Depth Completion
on KITTI Depth Completion
Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes
Further, we demonstrate the utility of our approach on training standard deep models for semantic instance segmentation and object detection of cars in outdoor driving scenes.
Slow Flow: Exploiting High-Speed Cameras for Accurate and Diverse Optical Flow Reference Data
Existing optical flow datasets are limited in size and variability due to the difficulty of capturing dense ground truth.
Toroidal Constraints for Two-Point Localization Under High Outlier Ratios
Adding the knowledge of direction of triangulation, we are able to approximate the position of the camera from two matches alone.
A Multi-View Stereo Benchmark With High-Resolution Images and Multi-Camera Videos
Motivated by the limitations of existing multi-view stereo benchmarks, we present a novel dataset for this task.
Semantic Multi-View Stereo: Jointly Estimating Objects and Voxels
Due to its probabilistic nature, the approach is able to cope with the approximate geometry of the 3D models as well as input shapes that are not present in the scene.
The Numerics of GANs
In this paper, we analyze the numerics of common algorithms for training Generative Adversarial Networks (GANs).
Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
Towards this goal, we analyze the performance of the state of the art on several challenging benchmarking datasets, including KITTI, MOT, and Cityscapes.
OctNetFusion: Learning Depth Fusion from Data
In this paper, we present a learning based approach to depth fusion, i. e., dense 3D reconstruction from multiple depth images.
Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks
We show that in the nonparametric limit our method yields an exact maximum-likelihood assignment for the parameters of the generative model, as well as the exact posterior distribution over the latent variables given an observation.
Probabilistic Duality for Parallel Gibbs Sampling without Graph Coloring
We present a new notion of probabilistic duality for random variables involving mixture distributions.
OctNet: Learning Deep 3D Representations at High Resolutions
We present OctNet, a representation for deep learning with sparse 3D data.
Patches, Planes and Probabilities: A Non-Local Prior for Volumetric 3D Reconstruction
In this paper, we propose a non-local structured prior for volumetric multi-view 3D reconstruction.
Exploiting Object Similarity in 3D Reconstruction
Despite recent progress, reconstructing outdoor scenes in 3D from movable platforms remains a highly difficult endeavour.
Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer
Semantic annotations are vital for training models for object recognition, semantic segmentation or scene understanding.
Object Scene Flow for Autonomous Vehicles
We demonstrate the performance of our model on existing benchmarks as well as a novel realistic dataset with scene flow ground truth.
Displets: Resolving Stereo Ambiguities Using Object Knowledge
Stereo techniques have witnessed tremendous progress over the last decades, yet some aspects of the problem still remain challenging today.
FollowMe: Efficient Online Min-Cost Flow Tracking with Bounded Memory and Computation
One of the most popular approaches to multi-target tracking is tracking-by-detection.
Ranked #23 on
Multiple Object Tracking
on KITTI Tracking test
Lost! Leveraging the Crowd for Probabilistic Visual Self-Localization
In this paper we propose an affordable solution to selflocalization, which utilizes visual odometry and road maps as the only inputs.
Joint 3D Estimation of Objects and Scene Layout
We propose a novel generative model that is able to reason jointly about the 3D scene layout as well as the 3D location and orientation of objects in the scene.
