Search Results for author:
Found 72 papers, 22 papers with code
Controllable Weather Synthesis and Removal with Video Diffusion Models
Generating realistic and controllable weather effects in videos is valuable for many applications.
DRAWER: Digital Reconstruction and Articulation With Environment Realism
Creating virtual digital replicas from real-world data unlocks significant potential across domains like gaming and robotics.
Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video
This paper presents a unified approach to understanding dynamic scenes from casual videos.
PhysGen3D: Crafting a Miniature Interactive World from a Single Image
Envisioning physically plausible outcomes from a single image requires a deep understanding of the world's dynamics.
PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos
Creating a physical digital twin of a real-world object has immense potential in robotics, content creation, and XR.
GigaSLAM: Large-Scale Monocular SLAM with Hierachical Gaussian Splats
Consequently, GigaSLAM delivers high-precision tracking and visually faithful rendering on urban outdoor benchmarks, establishing a robust SLAM solution for large-scale, long-term scenarios, and significantly extending the applicability of Gaussian Splatting SLAM systems to unbounded outdoor environments.
Map Space Belief Prediction for Manipulation-Enhanced Mapping
To tackle this, we define a POMDP whose belief is summarized by a metric-semantic grid map and propose a novel framework that uses neural networks to perform map-space belief updates to reason efficiently and simultaneously about object geometries, locations, categories, occlusions, and manipulation physics.
LIFe-GoM: Generalizable Human Rendering with Learned Iterative Feedback Over Multi-Resolution Gaussians-on-Mesh
Generalizable rendering of an animatable human avatar from sparse inputs relies on data priors and inductive biases extracted from training on large data to avoid scene-specific optimization and to enable fast reconstruction.
CropCraft: Inverse Procedural Modeling for 3D Reconstruction of Crop Plants
The ability to automatically build 3D digital twins of plants from images has countless applications in agriculture, environmental science, robotics, and other fields.
AutoVFX: Physically Realistic Video Editing from Natural Language Instructions
Modern visual effects (VFX) software has made it possible for skilled artists to create imagery of virtually anything.
PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation
We present PhysGen, a novel image-to-video generation method that converts a single image and an input condition (e. g., force and torque applied to an object in the image) to produce a realistic, physically plausible, and temporally consistent video.
Plenoptic PNG: Real-Time Neural Radiance Fields in 150 KB
The goal of this paper is to encode a 3D scene into an extremely compact representation from 2D images and to enable its transmittance, decoding and rendering in real-time across various platforms.
Towards Energy-Efficiency by Navigating the Trilemma of Energy, Latency, and Accuracy
We identify a Pareto-optimal curve and show that the designs on the curve are achievable only through synergistic co-optimization of all three optimization classes and by considering the latency and accuracy needs of downstream scene reconstruction consumers.
EarthGen: Generating the World from Top-Down Views
In this work, we present a novel method for extensive multi-scale generative terrain modeling.
SuperGaussian: Repurposing Video Models for 3D Super Resolution
We demonstrate that it is possible to directly repurpose existing (pretrained) video models for 3D super-resolution and thus sidestep the problem of the shortage of large repositories of high-quality 3D training models.
Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video
Creating high-quality and interactive virtual environments, such as games and simulators, often involves complex and costly manual modeling processes.
MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance
The latest regularized Neural Radiance Field (NeRF) approaches produce poor geometry and view extrapolation for large scale sparse view scenes, such as ETH3D.
GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Mesh
We introduce GoMAvatar, a novel approach for real-time, memory-efficient, high-quality animatable human modeling.
Physical Property Understanding from Language-Embedded Feature Fields
Can computers perceive the physical properties of objects solely through vision?
LidarDM: Generative LiDAR Simulation in a Generated World
We present LidarDM, a novel LiDAR generative model capable of producing realistic, layout-aware, physically plausible, and temporally coherent LiDAR videos.
RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation
We introduce the novel task of interactive scene exploration, wherein robots autonomously explore environments and produce an action-conditioned scene graph (ACSG) that captures the structure of the underlying environment.
IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images
Our results show that IRIS effectively recovers HDR lighting, accurate material, and plausible camera response functions, supporting photorealistic relighting and object insertion.
Objects With Lighting: A Real-World Dataset for Evaluating Reconstruction and Rendering for Object Relighting
Reconstructing an object from photos and placing it virtually in a new environment goes beyond the standard novel view synthesis task as the appearance of the object has to not only adapt to the novel viewpoint but also to the new lighting conditions and yet evaluations of inverse rendering methods rely on novel view synthesis data or simplistic synthetic datasets for quantitative analysis.
Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects
Our world is full of identical objects (\emphe. g., cans of coke, cars of same model).
Video2Game: Real-time Interactive Realistic and Browser-Compatible Environment from a Single Video
Creating high-quality and interactive virtual environments such as games and simulators often involves complex and costly manual modeling processes.
On the Overconfidence Problem in Semantic 3D Mapping
Semantic 3D mapping, the process of fusing depth and image segmentation information between multiple views to build 3D maps annotated with object classes in real-time, is a recent topic of interest.
ContactGen: Generative Contact Modeling for Grasp Generation
This paper presents a novel object-centric contact representation ContactGen for hand-object interaction.
MapPrior: Bird's-Eye View Map Layout Estimation with Generative Models
Despite tremendous advancements in bird's-eye view (BEV) perception, existing models fall short in generating realistic and coherent semantic map layouts, and they fail to account for uncertainties arising from partial sensor information (such as occlusion or limited coverage).
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
We present UrbanIR (Urban Scene Inverse Rendering), a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video.
Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Clouds
We build rearticulable models for arbitrary everyday man-made objects containing an arbitrary number of parts that are connected together in arbitrary ways via 1 degree-of-freedom joints.
PEANUT: Predicting and Navigating to Unseen Targets
In this work, we present a straightforward method for learning these regularities by predicting the locations of unobserved objects from incomplete semantic maps.
QFF: Quantized Fourier Features for Neural Field Representations
Instead, we propose to encode features in bins of Fourier features that are commonly used for positional encoding.
ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field
Physical simulations produce excellent predictions of weather effects.
CASA: Category-agnostic Skeletal Animal Reconstruction
Recovering the skeletal shape of an animal from a monocular video is a longstanding challenge.
Learning to Generate Realistic LiDAR Point Clouds
We present LiDARGen, a novel, effective, and controllable generative model that produces realistic LiDAR point cloud sensory readings.
Virtual Correspondence: Humans as a Cue for Extreme-View Geometry
Similar to classic correspondences, VCs conform with epipolar geometry; unlike classic correspondences, VCs do not need to be co-visible across views.
NeurMiPs: Neural Mixture of Planar Experts for View Synthesis
We present Neural Mixtures of Planar Experts (NeurMiPs), a novel planar-based scene representation for modeling geometry and appearance.
Deep Feedback Inverse Problem Solver
Specifically, at each iteration, the neural network takes the feedback as input and outputs an update on the current estimation.
Non-parametric Memory for Spatio-Temporal Segmentation of Construction Zones for Self-Driving
In this paper, we introduce a non-parametric memory representation for spatio-temporal segmentation that captures the local space and time around an autonomous vehicle (AV).
Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wild
Reconstructing high-quality 3D objects from sparse, partial observations from a single view is of crucial importance for various applications in computer vision, robotics, and graphics.
Deep Parametric Continuous Convolutional Neural Networks
Standard convolutional neural networks assume a grid structured input is available and exploit discrete convolutions as their fundamental building blocks.
Ranked #2 on
Semantic Segmentation
on S3DIS Area5
(Number of params metric)
S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
Constructing and animating humans is an important component for building virtual worlds in a wide variety of applications such as virtual reality or robotics testing in simulation.
Asynchronous Multi-View SLAM
Existing multi-camera SLAM systems assume synchronized shutters for all cameras, which is often not the case in practice.
SceneGen: Learning to Generate Realistic Traffic Scenes
Existing methods typically insert actors into the scene according to a set of hand-crafted heuristics and are limited in their ability to model the true complexity and diversity of real traffic scenes, thus inducing a content gap between synthesized traffic scenes versus real ones.
GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Driving
Scalable sensor simulation is an important yet challenging open problem for safety-critical domains such as self-driving.
Pit30M: A Benchmark for Global Localization in the Age of Self-Driving Cars
We are interested in understanding whether retrieval-based localization approaches are good enough in the context of self-driving vehicles.
Convolutional Recurrent Network for Road Boundary Extraction
Creating high definition maps that contain precise information of static elements of the scene is of utmost importance for enabling self driving cars to drive safely.
Learning to Localize Using a LiDAR Intensity Map
In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars.
Learning to Localize Through Compressed Binary Maps
One of the main difficulties of scaling current localization systems to large environments is the on-board storage required for the maps.
Deep Continuous Fusion for Multi-Sensor 3D Object Detection
In this paper, we propose a novel 3D object detector that can exploit both LIDAR as well as cameras to perform very accurate localization.
MuSCLE: Multi Sweep Compression of LiDAR using Deep Entropy Models
Our model exploits spatio-temporal relationships across multiple LiDAR sweeps to reduce the bitrate of both geometry and intensity values.
DSDNet: Deep Structured self-Driving Network
In this paper, we propose the Deep Structured self-Driving Network (DSDNet), which performs object detection, motion prediction, and motion planning with a single neural network.
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World
We first utilize ray casting over the 3D scene and then use a deep neural network to produce deviations from the physics-based simulation, producing realistic LiDAR point clouds.
OctSqueeze: Octree-Structured Entropy Model for LiDAR Compression
We present a novel deep compression algorithm to reduce the memory footprint of LiDAR point clouds.
Identifying Unknown Instances for Autonomous Driving
In the past few years, we have seen great progress in perception algorithms, particular through the use of deep learning.
Efficient Graph Generation with Graph Recurrent Attention Networks
Our model generates graphs one block of nodes and associated edges at a time.
DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch
Our goal is to significantly speed up the runtime of current state-of-the-art stereo algorithms to enable real-time inference.
DSIC: Deep Stereo Image Compression
In this paper we tackle the problem of stereo image compression, and leverage the fact that the two images have overlapping fields of view to further compress the representations.
Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization
In this paper we propose a novel semantic localization algorithm that exploits multiple sensors and has precision on the order of a few centimeters.
Deep Multi-Sensor Lane Detection
Reliable and accurate lane detection has been a long-standing problem in the field of autonomous driving.
Deep Rigid Instance Scene Flow
In this paper we tackle the problem of scene flow estimation in the context of self-driving.
Proximal Deep Structured Models
Many problems in real-world applications involve predicting continuous-valued random variables that are statistically related.
TorontoCity: Seeing the World with a Million Eyes
In this paper we introduce the TorontoCity benchmark, which covers the full greater Toronto area (GTA) with 712. 5 $km^2$ of land, 8439 $km$ of road and around 400, 000 buildings.
AutoScaler: Scale-Attention Networks for Visual Correspondence
We propose AutoScaler, a scale-attention network to explicitly optimize this trade-off in visual correspondence tasks.
Find your Way by Observing the Sun and Other Semantic Cues
In this paper we present a robust, efficient and affordable approach to self-localization which does not require neither GPS nor knowledge about the appearance of the world.
HD Maps: Fine-Grained Road Segmentation by Parsing Ground and Aerial Images
In this paper we present an approach to enhance existing maps with fine grained segmentation categories such as parking spots and sidewalk, as well as the number and location of road lanes.
The Global Patch Collider
In contrast to conventional approaches that rely on pairwise distance computation, our algorithm isolates distinctive pixel pairs that hit the same leaf during traversal through multiple learned tree structures.
Enhancing Road Maps by Parsing Aerial Images Around the World
In recent years, contextual models that exploit maps have been shown to be very effective for many recognition and localization tasks.
Lost Shopping! Monocular Localization in Large Indoor Spaces
In this paper we propose a novel approach to localization in very large indoor spaces (i. e., 200+ store shopping malls) that takes a single image and a floor plan of the environment as input.
Holistic 3D Scene Understanding From a Single Geo-Tagged Image
In this paper we are interested in exploiting geographic priors to help outdoor scene understanding.
Efficient Inference of Continuous Markov Random Fields with Polynomial Potentials
In this paper, we prove that every multivariate polynomial with even degree can be decomposed into a sum of convex and concave polynomials.
