Search Results for author:
Found 42 papers, 10 papers with code
Coarse Correspondence Elicit 3D Spacetime Understanding in Multimodal Language Model
We introduce Coarse Correspondence, a simple, training-free, effective, and general-purpose visual prompting method to elicit 3D and temporal understanding in multimodal LLMs.
Preserving Identity with Variational Score for General-purpose 3D Editing
We present Piva (Preserving Identity with Variational Score Distillation), a novel optimization-based method for editing images and 3D models based on diffusion models.
ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Models
We propose ExtraNeRF, a novel method for extrapolating the range of views handled by a Neural Radiance Field (NeRF).
Multilingual Diversity Improves Vision-Language Representations
By translating all multilingual image-text pairs from a raw web crawl to English and re-filtering them, we increase the prevalence of (translated) multilingual data in the resulting training set.
BLINK: Multimodal Large Language Models Can See but Not Perceive
We introduce Blink, a new benchmark for multimodal language models (LLMs) that focuses on core visual perception abilities not found in other evaluations.
Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Video
Creating high-quality and interactive virtual environments, such as games and simulators, often involves complex and costly manual modeling processes.
Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects
Our world is full of identical objects (\emphe. g., cans of coke, cars of same model).
Video2Game: Real-time Interactive Realistic and Browser-Compatible Environment from a Single Video
Creating high-quality and interactive virtual environments such as games and simulators often involves complex and costly manual modeling processes.
LightSim: Neural Lighting Simulation for Urban Scenes
Different outdoor illumination conditions drastically alter the appearance of urban scenes, and they can harm the performance of image-based robot perception systems if not seen during training.
UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation
We show that by aligning the representation of a sparse point cloud to that of a dense point cloud, we can densify the sparse point clouds as if they were captured by a real high-density LiDAR, drastically reducing the cost.
CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation
To tackle these issues, we present CADSim, which combines part-aware object-class priors via a small set of CAD models with differentiable rendering to automatically reconstruct vehicle geometry, including articulated wheels, with high-quality appearance.
UniSim: A Neural Closed-Loop Sensor Simulator
Previously recorded driving logs provide a rich resource to build these new scenarios from, but for closed loop evaluation, we need to modify the sensor data based on the new scene configuration and the SDV's decisions, as actors might be added or removed and the trajectories of existing actors and the SDV will differ from the original log.
What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging
Inferring past human motion from RGB images is challenging due to the inherent uncertainty of the prediction problem.
VoxelFormer: Bird's-Eye-View Feature Generation based on Dual-view Attention for Multi-view 3D Object Detection
In recent years, transformer-based detectors have demonstrated remarkable performance in 2D visual perception tasks.
Learning Compact Representations for LiDAR Completion and Generation
We show that by aligning the representation of a sparse point cloud to that of a dense point cloud, we can densify the sparse point clouds as if they were captured by a real high-density LiDAR, drastically reducing the cost.
Virtual Correspondence: Humans as a Cue for Extreme-View Geometry
Similar to classic correspondences, VCs conform with epipolar geometry; unlike classic correspondences, VCs do not need to be co-visible across views.
NeurMiPs: Neural Mixture of Planar Experts for View Synthesis
We present Neural Mixtures of Planar Experts (NeurMiPs), a novel planar-based scene representation for modeling geometry and appearance.
BARF: Bundle-Adjusting Neural Radiance Fields
In this paper, we propose Bundle-Adjusting Neural Radiance Fields (BARF) for training NeRF from imperfect (or even unknown) camera poses -- the joint problem of learning neural 3D representations and registering camera frames.
Deep Feedback Inverse Problem Solver
Specifically, at each iteration, the neural network takes the feedback as input and outputs an update on the current estimation.
Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wild
Reconstructing high-quality 3D objects from sparse, partial observations from a single view is of crucial importance for various applications in computer vision, robotics, and graphics.
Deep Parametric Continuous Convolutional Neural Networks
Standard convolutional neural networks assume a grid structured input is available and exploit discrete convolutions as their fundamental building blocks.
Ranked #2 on
Semantic Segmentation
on S3DIS Area5
(Number of params metric)
S3: Neural Shape, Skeleton, and Skinning Fields for 3D Human Modeling
Constructing and animating humans is an important component for building virtual worlds in a wide variety of applications such as virtual reality or robotics testing in simulation.
VideoClick: Video Object Segmentation with a Single Click
Towards this goal, in this paper we propose a bottom up approach where given a single click for each object in a video, we obtain the segmentation masks of these objects in the full video.
Hierarchical Recurrent Attention Networks for Structured Online Maps
In this paper, we tackle the problem of online road network extraction from sparse 3D point clouds.
DAGMapper: Learning to Map by Discovering Lane Topology
One of the fundamental challenges to scale self-driving is being able to create accurate high definition maps (HD maps) with low cost.
Convolutional Recurrent Network for Road Boundary Extraction
Creating high definition maps that contain precise information of static elements of the scene is of utmost importance for enabling self driving cars to drive safely.
Recovering and Simulating Pedestrians in the Wild
We then incorporate the reconstructed pedestrian assets bank in a realistic LiDAR simulation system by performing motion retargeting, and show that the simulated LiDAR data can be used to significantly reduce the amount of annotated real-world data required for visual perception tasks.
Weakly-supervised 3D Shape Completion in the Wild
3D shape completion for real data is important but challenging, since partial point clouds acquired by real-world sensors are usually sparse, noisy and unaligned.
LevelSet R-CNN: A Deep Variational Method for Instance Segmentation
Obtaining precise instance segmentation masks is of high importance in many modern applications such as robotic manipulation and autonomous driving.
LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World
We first utilize ray casting over the 3D scene and then use a deep neural network to produce deviations from the physics-based simulation, producing realistic LiDAR point clouds.
PolyTransform: Deep Polygon Transformer for Instance Segmentation
In this paper, we propose PolyTransform, a novel instance segmentation algorithm that produces precise, geometry-preserving masks by combining the strengths of prevailing segmentation approaches and modern polygon-based methods.
Ranked #1000000000 on
Instance Segmentation
on Cityscapes test
(using extra training data)
DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch
Our goal is to significantly speed up the runtime of current state-of-the-art stereo algorithms to enable real-time inference.
Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization
In this paper we propose a novel semantic localization algorithm that exploits multiple sensors and has precision on the order of a few centimeters.
Deep Rigid Instance Scene Flow
In this paper we tackle the problem of scene flow estimation in the context of self-driving.
SurfConv: Bridging 3D and 2D Convolution for RGBD Images
On the other hand, 3D convolution wastes a large amount of memory on mostly unoccupied 3D space, which consists of only the surface visible to the sensor.
Single Image Intrinsic Decomposition without a Single Intrinsic Image
At inference time, our model can be easily reduced to a single stream module that performs intrinsic decomposition on a single input image.
Find your Way by Observing the Sun and Other Semantic Cues
In this paper we present a robust, efficient and affordable approach to self-localization which does not require neither GPS nor knowledge about the appearance of the world.
Forecasting Interactive Dynamics of Pedestrians with Fictitious Play
We develop predictive models of pedestrian dynamics by encoding the coupled nature of multi-pedestrian interaction using game theory, and deep learning-based visual analysis to estimate person-specific behavior parameters.
How Do We Use Our Hands? Discovering a Diverse Set of Common Grasps
Furthermore, we develop a hierarchical extension to the DPP clustering algorithm and show that it can be used to discover appearance-based grasp taxonomies.
