Search Results for author:
Found 78 papers, 31 papers with code
NASA Neural Articulated Shape Approximation
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
StochasticSplats: Stochastic Rasterization for Sorting-Free 3D Gaussian Splatting
3D Gaussian splatting (3DGS) is a popular radiance field method, with many application-specific extensions.
NoKSR: Kernel-Free Neural Surface Reconstruction via Point Cloud Serialization
We present a novel approach to large-scale point cloud surface reconstruction by developing an efficient framework that converts an irregular point cloud into a signed distance field (SDF).
RoMo: Robust Motion Segmentation Improves Structure from Motion
There has been extensive progress in the reconstruction and generation of 4D scenes from monocular casually-captured video.
AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers
This suggested us to limit the injection of camera conditioning to a subset of the architecture to prevent interference with other video features, leading to a 4x reduction of training parameters, improved training speed, and 10% higher visual quality.
Lagrangian Hashing for Compressed Neural Field Representations
We present Lagrangian Hashing, a representation for neural fields combining the characteristics of fast training NeRF methods that rely on Eulerian grids (i. e.~InstantNGP), with those that employ points equipped with features as a way to represent information (e. g. 3D Gaussian Splatting or PointNeRF).
VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
Recently, new methods demonstrate the ability to generate videos with controllable camera poses these techniques leverage pre-trained U-Net-based diffusion models that explicitly disentangle spatial and temporal generation.
Controlling Space and Time with Diffusion Models
We present 4DiM, a cascaded diffusion model for 4D novel view synthesis (NVS), supporting generation with arbitrary camera trajectories and timestamps, in natural scenes, conditioned on one or more images.
SpotlessSplats: Ignoring Distractors in 3D Gaussian Splatting
3D Gaussian Splatting (3DGS) is a promising technique for 3D reconstruction, offering efficient training and rendering speeds, making it suitable for real-time applications. However, current methods require highly controlled environments (no moving people or wind-blown elements, and consistent lighting) to meet the inter-view consistency assumption of 3DGS.
BANF: Band-limited Neural Fields for Levels of Detail Reconstruction
Largely due to their implicit nature, neural fields lack a direct mechanism for filtering, as Fourier analysis from discrete signal processing is not directly applicable to these representations.
Evaluating Alternatives to SFM Point Cloud Initialization for Gaussian Splatting
3D Gaussian Splatting has recently been embraced as a versatile and effective method for scene reconstruction and novel view synthesis, owing to its high-quality results and compatibility with hardware rasterization.
3D Gaussian Splatting as Markov Chain Monte Carlo
While 3D Gaussian Splatting has recently become popular for neural rendering, current methods rely on carefully engineered cloning and splitting strategies for placing Gaussians, which can lead to poor-quality renderings, and reliance on a good initialization.
TC4D: Trajectory-Conditioned Text-to-4D Generation
We learn local deformations that conform to the global trajectory using supervision from a text-to-video model.
Neural Fields as Distributions: Signal Processing Beyond Euclidean Space
We demonstrate how this framework can enable novel integrations of signal processing techniques for neural field applications on both Euclidean domains such as images and audio as well as non-Euclidean domains such as rotations and rays.
pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
We introduce pixelSplat, a feed-forward model that learns to reconstruct 3D radiance fields parameterized by 3D Gaussian primitives from pairs of images.
Ranked #2 on
Generalizable Novel View Synthesis
on ACID
Densify Your Labels: Unsupervised Clustering with Bipartite Matching for Weakly Supervised Point Cloud Segmentation
We propose a weakly supervised semantic segmentation method for point clouds that predicts "per-point" labels from just "whole-scene" annotations while achieving the performance of recent fully supervised approaches.
PointNeRF++: A multi-scale, point-based Neural Radiance Field
We overcome these problems with a simple representation that aggregates point clouds at multiple scale levels with sparse voxel grids at different resolutions.
Volumetric Rendering with Baked Quadrature Fields
We then perform ray-tracing and utilize the ray-tracing shader to obtain the final colour image.
Unsupervised Keypoints from Pretrained Diffusion Models
Unsupervised learning of keypoints and landmarks has seen significant progress with the help of modern neural network architectures, but performance is yet to match the supervised counterpart, making their practicability questionable.
Ranked #1 on
Unsupervised Human Pose Estimation
on Tai-Chi-HD
Accelerating Neural Field Training via Soft Mining
We present an approach to accelerate Neural Field training by efficiently selecting sampling locations.
4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling
Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.
Bayes' Rays: Uncertainty Quantification for Neural Radiance Fields
Neural Radiance Fields (NeRFs) have shown promise in applications like view synthesis and depth estimation, but learning from multiview images faces inherent uncertainties.
Unsupervised Semantic Correspondence Using Stable Diffusion
Text-to-image diffusion models are now capable of generating images that are often indistinguishable from real images.
Ranked #1 on
Semantic correspondence
on CUB-200-2011
BlendFields: Few-Shot Example-Driven Facial Modeling
Generating faithful visualizations of human faces requires capturing both coarse and fine-level details of the face geometry and appearance.
CC3D: Layout-Conditioned Generation of Compositional 3D Scenes
In this work, we introduce CC3D, a conditional generative model that synthesizes complex 3D scenes conditioned on 2D semantic scene layouts, trained using single-view images.
RobustNeRF: Ignoring Distractors with Robust Losses
To cope with distractors, we advocate a form of robust estimation for NeRF training, modeling distractors in training data as outliers of an optimization problem.
SparsePose: Sparse-View Camera Pose Regression and Refinement
Camera pose estimation is a key step in standard 3D reconstruction pipelines that operate on a dense set of images of a single object or scene.
OpenScene: 3D Scene Understanding with Open Vocabularies
Traditional 3D scene understanding approaches rely on labeled 3D datasets to train a model for a single task with supervision.
Ranked #7 on
3D Open-Vocabulary Instance Segmentation
on Replica
3D Open-Vocabulary Instance Segmentation
3D Semantic Segmentation
+1
NeuMap: Neural Coordinate Mapping by Auto-Transdecoder for Camera Localization
State-of-the-art feature matching methods require each scene to be stored as a 3D point cloud with per-point features, consuming several gigabytes of storage per scene.
nerf2nerf: Pairwise Registration of Neural Radiance Fields
We introduce a technique for pairwise registration of neural fields that extends classical optimization-based local registration (i. e. ICP) to operate on Neural Radiance Fields (NeRF) -- neural 3D scene representations trained from collections of calibrated images.
CUF: Continuous Upsampling Filters
Neural fields have rapidly been adopted for representing 3D signals, but their application to more classical 2D image-processing has been relatively limited.
Novel View Synthesis with Diffusion Models
We demonstrate that stochastic conditioning significantly improves the 3D consistency of a naive sampler for an image-to-image diffusion model, which involves conditioning on a single fixed view.
Attention Beats Concatenation for Conditioning Neural Fields
Neural fields model signals by mapping coordinate inputs to sampled values.
Volume Rendering Digest (for NeRF)
Neural Radiance Fields employ simple volume rendering as a way to overcome the challenges of differentiating through ray-triangle intersections by leveraging a probabilistic notion of visibility.
MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures
Neural Radiance Fields (NeRFs) have demonstrated amazing ability to synthesize images of 3D scenes from novel views.
Ranked #2 on
Novel View Synthesis
on LLFF
NeuralBF: Neural Bilateral Filtering for Top-down Instance Segmentation on Point Clouds
We introduce a method for instance proposal generation for 3D point clouds.
D$^2$NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Video
We introduce Decoupled Dynamic Neural Radiance Field (D$^2$NeRF), a self-supervised approach that takes a monocular video and learns a 3D scene representation which decouples moving objects, including their shadows, from the static background.
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
Our model builds a panoptic radiance field representation of any scene from just color images.
Neural Dual Contouring
We introduce neural dual contouring (NDC), a new data-driven approach to mesh reconstruction based on dual contouring (DC).
Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation
Our performance generalizes across both object instances and 6-DoF object poses, and significantly outperforms a recent baseline that relies on 2D descriptors.
CoNeRF: Controllable Neural Radiance Fields
We extend neural 3D representations to allow for intuitive and interpretable user control beyond novel view rendering (i. e. camera control).
Urban Radiance Fields
The goal of this work is to perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments (e. g., Street View).
VaxNeRF: Revisiting the Classic for Voxel-Accelerated Neural Radiance Field
We hope VaxNeRF -- a careful combination of a classic technique with a deep method (that arguably replaced it) -- can empower and accelerate new NeRF extensions and applications, with its simplicity, portability, and reliable performance gains.
NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes
We present NeSF, a method for producing 3D semantic fields from posed RGB images alone.
Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations
In this work, we propose the Scene Representation Transformer (SRT), a method which processes posed or unposed RGB images of a new area, infers a "set-latent scene representation", and synthesises novel views, all in a single feed-forward pass.
LOLNeRF: Learn from One Look
We present a method for learning a generative 3D model based on neural radiance fields, trained solely from data with only single views of each object.
MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision
In the era of deep learning, human pose estimation from multiple cameras with unknown calibration has received little attention to date.
Ranked #1 on
3D Human Pose Estimation
on SkiPose
Learning Mesh Representations via Binary Space Partitioning Tree Networks
The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built over a set of planes, where the planes and convexes are both defined by learned network weights.
MIST: Multiple Instance Spatial Transformer
We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.
Deep Medial Fields
Implicit representations of geometry, such as occupancy fields or signed distance fields (SDF), have recently re-gained popularity in encoding 3D solid shape in a functional form.
Vector Neurons: A General Framework for SO(3)-Equivariant Networks
Invariance and equivariance to the rotation group have been widely discussed in the 3D deep learning community for pointclouds.
COTR: Correspondence Transformer for Matching Across Images
We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other.
Ranked #1 on
Dense Pixel Correspondence Estimation
on KITTI 2012
Dense Pixel Correspondence Estimation
Optical Flow Estimation
Human 3D keypoints via spatial uncertainty modeling
We introduce a technique for 3D human keypoint estimation that directly models the notion of spatial uncertainty of a keypoint.
Canonical Capsules: Self-Supervised Capsules in Canonical Pose
We propose a self-supervised capsule architecture for 3D point clouds.
ShapeFlow: Learnable Deformation Flows Among 3D Shapes
Such a space naturally allows the disentanglement of geometric style (coming from the source) and structural pose (conforming to the target).
Unsupervised part representation by Flow Capsules
Capsule networks aim to parse images into a hierarchy of objects, parts and relations.
DeRF: Decomposed Radiance Fields
Moreover, we show that a Voronoi spatial decomposition is preferable for this purpose, as it is provably compatible with the Painter's Algorithm for efficient and GPU-friendly rendering.
Voronoi Convolutional Neural Networks
In this technical report, we investigate extending convolutional neural networks to the setting where functions are not sampled in a grid pattern.
PIE-NET: Parametric Inference of Point Cloud Edges
We introduce an end-to-end learnable technique to robustly identify feature edges in 3D point cloud data.
CoSE: Compositional Stroke Embeddings
We demonstrate qualitatively and quantitatively that our proposed approach is able to model the appearance of individual strokes, as well as the compositional structure of larger diagram drawings.
ShapeFlow: Learnable Deformations Among 3D Shapes
We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
Deep Implicit Volume Compression
We describe a novel approach for compressing truncated signed distance fields (TSDF) stored in 3D voxel grids, and their corresponding textures.
NiLBS: Neural Inverse Linear Blend Skinning
In this technical report, we investigate efficient representations of articulated objects (e. g. human bodies), which is an important problem in computer vision and graphics.
VoronoiNet: General Functional Approximators with Local Support
Voronoi diagrams are highly compact representations that are used in various Graphics applications.
NASA: Neural Articulated Shape Approximation
Efficient representation of articulated objects such as human bodies is an important problem in computer vision and graphics.
BSP-Net: Generating Compact Meshes via Binary Space Partitioning
The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built on a set of planes.
MIST: Multiple Instance Spatial Transformer Networks
We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.
CvxNet: Learnable Convex Decomposition
We introduce a network architecture to represent a low dimensional family of convexes.
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning
Many problems in computer vision require dealing with sparse, unordered data in the form of point clouds.
Volumetric Capture of Humans with a Single RGBD Camera via Semi-Parametric Learning
The key insight is to leverage previously seen "calibration" images of a given user to extrapolate what should be rendered in a novel viewpoint from the data available in the sensor.
Linearized Multi-Sampling for Differentiable Image Transformation
We propose a novel image sampling method for differentiable image transformation in deep neural networks.
MIST: Multiple Instance Spatial Transformer Network
We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.
Anomaly Detection In Surveillance Videos
Image Reconstruction
Espresso: Efficient Forward Propagation for Binary Deep Neural Networks
Binary Deep Neural Networks (BDNNs) have been shown to be an effective way of achieving this objective.
HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images
We propose an automatic method for generating high-quality annotations for depth-based hand segmentation, and introduce a large-scale hand segmentation dataset.
Low-Dimensionality Calibration Through Local Anisotropic Scaling for Robust Hand Model Personalization
We present a robust algorithm for personalizing a sphere-mesh tracking model to a user from a collection of depth measurements.
Espresso: Efficient Forward Propagation for BCNNs
In this paper, we show how Convolutional Neural Networks (CNNs) can be implemented using binary representations.
