Search Results for author:
Found 36 papers, 13 papers with code
Disentangling 3D from Large Vision-Language Models for Controlled Portrait Generation
We consider the problem of disentangling 3D from large vision-language models, which we show on generative 3D portraits.
MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learning
Specifically, (1) We developed a multimodal RLVR framework for multi-dataset post-training by curating a dataset that contains different verifiable vision-language problems and enabling multi-domain online RL learning with different verifiable rewards; (2) We proposed a data mixture strategy that learns to predict the RL fine-tuning outcome from the data mixture distribution, and consequently optimizes the best mixture.
Time of the Flight of the Gaussians: Optimizing Depth Indirectly in Dynamic Radiance Fields
Quickly achieving high-fidelity dynamic 3D reconstruction from a single viewpoint is a significant challenge in computer vision.
Zero-Shot Monocular Scene Flow Estimation in the Wild
Large models have shown generalization across datasets for many low-level vision tasks, like depth estimation, but no such general models exist for scene flow.
The GAN is dead; long live the GAN! A Modern GAN Baseline
There is a widely-spread claim that GANs are difficult to train, and GAN architectures in the literature are littered with empirical tricks.
Ranked #3 on
Image Generation
on ImageNet 32x32
Monocular Dynamic Gaussian Splatting is Fast and Brittle but Smooth Motion Helps
Gaussian splatting methods are emerging as a popular approach for converting multi-view image data into scene representations that allow view synthesis.
OmniSDF: Scene Reconstruction using Omnidirectional Signed Distance Functions and Adaptive Binoctrees
We present a method to reconstruct indoor and outdoor static scene geometry and appearance from an omnidirectional video moving in a small circular sweep.
GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis
Specifically, we propose a template set of 3D Gaussians residing in a canonical space, and a time-dependent forward-warping deformation field to model dynamic objects.
'Tax-free' 3DMM Conditional Face Generation
This results in a new model that effectively removes the quality tax between 3DMM conditioned face GANs and the unconditional StyleGAN.
Semantic Attention Flow Fields for Monocular Dynamic Scene Decomposition
From video, we reconstruct a neural volume that captures time-varying color, density, scene flow, semantics, and attention information.
Neural Fields for Structured Lighting
We present an image formation model and optimization procedure that combines the advantages of neural radiance fields and structured light imaging.
FloatingFusion: Depth from ToF and Image-stabilized Stereo Cameras
Fusing RGB stereo and ToF information is a promising direction to overcome these issues, but a key problem remains: to provide high-quality 2D RGB images, the main color sensor's lens is optically stabilized, resulting in an unknown pose for the floating lens that breaks the geometric relationships between the multimodal image sensors.
YouMVOS: An Actor-Centric Multi-Shot Video Object Segmentation Dataset
Many video understanding tasks require analyzing multi-shot videos, but existing datasets for video object segmentation (VOS) only consider single-shot videos.
Neural Fields in Visual Computing and Beyond
Recent advances in machine learning have created increasing interest in solving visual computing problems using a class of coordinate-based neural networks that parametrize physical properties of scenes or objects across space and time.
Testing using Privileged Information by Adapting Features with Statistical Dependence
Then, we empirically estimate and strengthen the statistical dependence between the initial noisy predictor and the additional features via manifold denoising.
TöRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis
Neural networks can represent and accurately reconstruct radiance fields for static 3D scenes (e. g., NeRF).
Dynamic Scene Novel View Synthesis via Deferred Spatio-temporal Consistency
Structure from motion (SfM) enables us to reconstruct a scene via casual capture from cameras at different viewpoints, and novel view synthesis (NVS) allows us to render a captured scene from a new viewpoint.
Edge-aware Bidirectional Diffusion for Dense Depth Estimation from Light Fields
We present an algorithm to estimate fast and accurate depth maps from light fields via a sparse set of depth edges and gradients.
GaussiGAN: Controllable Image Synthesis with 3D Gaussians from Unposed Silhouettes
We present an algorithm that learns a coarse 3D representation of objects from unposed multi-view 2D mask supervision, then uses it to generate detailed mask and image texture.
Differentiable Diffusion for Dense Depth Estimation from Multi-view Images
We present a method to estimate dense depth by optimizing a sparse set of points such that their diffusion into a depth map minimizes a multi-view reprojection error from RGB supervision.
Shape From Tracing: Towards Reconstructing 3D Object Geometry and SVBRDF Material from Images via Differentiable Path Tracing
In this paper, we explore how to use differentiable ray tracing to refine an initial coarse mesh and per-mesh-facet material representation.
Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation
To alleviate the problem, we propose to classify input features into intermediate shape codes and recover complete object shapes from them.
View-consistent 4D Light Field Depth Estimation
Previous light field depth estimation methods typically estimate a depth map only for the central sub-aperture view, and struggle with view consistent estimation.
Generating Handwriting via Decoupled Style Descriptors
Instead, we introduce the Decoupled Style Descriptor (DSD) model for handwriting, which factors both character- and writer-level styles and allows our model to represent an overall greater space of styles.
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images
Our approach is to simultaneously learn depth and disocclusions via a multi-sphere image representation, which can be rendered with correct 6DoF disparity and motion parallax in VR.
Generating Object Stamps
We present an algorithm to generate diverse foreground objects and composite them into background images using a GAN architecture.
Unsupervised Attention-guided Image-to-Image Translation
Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene.
Improving Shape Deformation in Unsupervised Image-to-Image Translation
Unsupervised image-to-image translation techniques are able to map local texture between two domains, but they are typically unsuccessful when the domains require larger shape change.
Unsupervised Attention-guided Image to Image Translation
Current unsupervised image-to-image translation techniques struggle to focus their attention on individual objects without altering the background or the way multiple objects interact within a scene.
High-Order Tensor Regularization With Application to Attribute Ranking
When learning functions on manifolds, we can improve performance by regularizing with respect to the intrinsic manifold geometry rather than the ambient space.
Predictor Combination at Test Time
We present an algorithm for test-time combination of a set of reference predictors with unknown parametric forms.
Criteria Sliders: Learning Continuous Database Criteria via Interactive Ranking
Large databases are often organized by hand-labeled metadata, or criteria, which are expensive to collect.
Guided Proofreading of Automatic Segmentations for Connectomics
Automatic cell image segmentation methods in connectomics produce merge and split errors, which require correction through proofreading.
Context-guided diffusion for label propagation on graphs
Existing approaches for diffusion on graphs, e. g., for label propagation, are mainly focused on isotropic diffusion, which is induced by the commonly-used graph Laplacian regularizer.
Semi-supervised Learning with Explicit Relationship Regularization
In many learning tasks, the structure of the target space of a function holds rich information about the relationships between evaluations of functions on different data points.
Local High-order Regularization on Data Manifolds
The iterated graph Laplacian enables high-order regularization, but it has a high computational complexity and so cannot be applied to large problems.
