Search Results for author:
Found 77 papers, 42 papers with code
Graph-Based Discriminative Learning for Location Recognition
For a query image, each database image is ranked according to these local distance functions in order to place the image in the right part of the graph.
Photometric Ambient Occlusion
We present a method for computing ambient occlusion (AO) for a stack of images of a scene from a fixed viewpoint.
Minimal Scene Descriptions from Structure from Motion Models
How much data do we need to describe a location?
Material Recognition in the Wild with the Materials in Context Database
In this paper, we introduce a new, large-scale, open dataset of materials in the wild, the Materials in Context Database (MINC), and combine this dataset with deep learning to achieve material recognition and segmentation of images in the wild.
DeepStereo: Learning to Predict New Views from the World's Imagery
To our knowledge, our work is the first to apply deep learning to the problem of new view synthesis from sets of real-world, natural imagery.
BubbLeNet: Foveated Imaging for Visual Discovery
We propose a new method for turning an Internet-scale corpus of categorized images into a small set of human-interpretable discriminative visual elements using powerful tools based on deep learning.
From A to Z: Supervised Transfer of Style and Content Using Deep Neural Network Generators
We propose a new neural network architecture for solving single-image analogies - the generation of an entire set of stylistically similar images from just a single input image.
Deep Feature Interpolation for Image Content Changes
We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation.
Unsupervised Learning of Depth and Ego-Motion from Video
We present an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences.
Shading Annotations in the Wild
We demonstrate the value of our data and network in an application to intrinsic images, where we can reduce decomposition artifacts produced by existing algorithms.
StreetStyle: Exploring world-wide clothing styles from millions of photos
Each day billions of photographs are uploaded to photo-sharing services and social media platforms.
Learning Intrinsic Image Decomposition from Watching the World
However, it is difficult to collect ground truth training data at scale for intrinsic images.
MegaDepth: Learning Single-View Depth Prediction from Internet Photos
We validate the use of large amounts of Internet data by showing that models trained on MegaDepth exhibit strong generalization-not only to novel scenes, but also to other diverse datasets including Make3D, KITTI, and DIW, even when no images from those datasets are seen during training.
Stereo Magnification: Learning View Synthesis using Multiplane Images
The view synthesis problem--generating novel views of a scene from known imagery--has garnered recent attention due in part to compelling applications in virtual and augmented reality.
Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning
We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object.
Layer-structured 3D Scene Inference via View Synthesis
We present an approach to infer a layer-structured 3D representation of a scene from a single input image.
CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering
Intrinsic image decomposition is a challenging, long-standing computer vision problem for which ground truth data is very difficult to acquire.
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task.
Ranked #10 on
Vision and Language Navigation
on Touchdown Dataset
Neural Rerendering in the Wild
Starting from internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud.
Learning the Depths of Moving People by Watching Frozen People
We present a method for predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving.
Pushing the Boundaries of View Extrapolation with Multiplane Images
We present a theoretical analysis showing how the range of views that can be rendered from an MPI increases linearly with the MPI disparity sampling frequency, as well as a novel MPI prediction procedure that theoretically enables view extrapolations of up to $4\times$ the lateral viewpoint movement allowed by prior work.
DeepView: View Synthesis with Learned Gradient Descent
We present a novel approach to view synthesis using multiplane images (MPIs).
UprightNet: Geometry-Aware Camera Orientation Estimation from Single Images
We introduce UprightNet, a learning-based approach for estimating 2DoF camera orientation from a single RGB image of an indoor scene.
GeoStyle: Discovering Fashion Trends and Events
Understanding fashion styles and trends is of great potential interest to retailers and consumers alike.
Leveraging Vision Reconstruction Pipelines for Satellite Imagery
Reconstructing 3D geometry from satellite imagery is an important topic of research.
Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination
We present a deep learning solution for estimating the incident illumination at any 3D location within a scene from an input narrow-baseline stereo image pair.
DualSDF: Semantic Shape Manipulation using a Two-Level Representation
We are seeing a Cambrian explosion of 3D shape representations for use in machine learning.
Depth Sensing Beyond LiDAR Range
Depth sensing is a critical component of autonomous driving technologies, but today's LiDAR- or stereo camera-based solutions have limited range.
Single-View View Synthesis with Multiplane Images
A recent strand of work in view synthesis uses deep learning to generate multiplane images (a camera-centric, layered 3D representation) given two or more input images at known viewpoints.
The Plant Pathology 2020 challenge dataset to classify foliar disease of apples
Appropriate and timely deployment of disease management depends on early disease detection.
Learning Feature Descriptors using Camera Pose Supervision
Recent research on learned visual descriptors has shown promising improvements in correspondence estimation, a key component of many 3D vision tasks.
MetaSDF: Meta-learning Signed Distance Functions
Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution.
An Analysis of SVD for Deep Rotation Estimation
Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$.
Crowdsampling the Plenoptic Function
These photos represent a sparse and unstructured sampling of the plenoptic function for a particular scene.
Learning to Factorize and Relight a City
We propose a learning-based framework for disentangling outdoor scenes into temporally-varying illumination and permanent scene factors.
Learning Gradient Fields for Shape Generation
Point cloud generation thus amounts to moving randomly sampled points to high-density areas.
Hidden Footprints: Learning Contextual Walkability from 3D Human Trails
Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis.
NeRF++: Analyzing and Improving Neural Radiance Fields
Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings, including 360 capture of bounded scenes and forward-facing capture of bounded and unbounded scenes.
Multi-Plane Program Induction with 3D Box Priors
We consider two important aspects in understanding and editing images: modeling regular, program-like texture or patterns in 2D planes, and 3D posing of these planes in the scene.
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
We present a method to perform novel view and time synthesis of dynamic scenes, requiring only a monocular video with known camera poses as input.
An Ethical Highlighter for People-Centric Dataset Creation
Important ethical concerns arising from computer vision datasets of people have been receiving significant attention, and a number of datasets have been withdrawn as a result.
Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image
We introduce the problem of perpetual view generation - long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image.
IBRNet: Learning Multi-View Image-Based Rendering
Unlike neural scene representation work that optimizes per-scene functions for rendering, we learn a generic view interpolation function that generalizes to novel scenes.
Repopulating Street Scenes
We present a framework for automatically reconfiguring images of street scenes by populating, depopulating, or repopulating them with objects such as pedestrians or vehicles.
PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting
We present PhySG, an end-to-end inverse rendering pipeline that includes a fully differentiable renderer and can reconstruct geometry, materials, and illumination from scratch from a set of RGB input images.
Ranked #5 on
Surface Normals Estimation
on Stanford-ORB
De-rendering the World's Revolutionary Artefacts
Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision.
KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control
We cast this as the problem of aligning a source 3D object to a target 3D object from the same object category.
Extreme Rotation Estimation using Dense Correlation Volumes
We present a technique for estimating the relative 3D rotation of an RGB image pair in an extreme setting, where the images have little or no overlap.
Wide-Baseline Relative Camera Pose Estimation with Directional Learning
Modern deep learning techniques that regress the relative camera pose between two images have difficulty dealing with challenging scenarios, such as large camera motions resulting in occlusions and significant changes in perspective that leave little overlap between images.
Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision
The abundance and richness of Internet photos of landmarks and cities has led to significant progress in 3D vision over the past two decades, including automated 3D reconstructions of the world's landmarks from tourist photos.
Who's Waldo? Linking People Across Text and Images
We present a task and benchmark dataset for person-centric visual grounding, the problem of linking between people named in a caption and people pictured in an image.
Ranked #1 on
Person-centric Visual Grounding
on Who’s Waldo
(using extra training data)
Dimensions of Motion: Monocular Prediction through Flow Subspaces
We introduce a way to learn to estimate a scene representation from a single image by predicting a low-dimensional subspace of optical flow for each training example, which encompasses the variety of possible camera and object movement.
Unsupervised Semantic Segmentation by Distilling Feature Correspondences
Unsupervised semantic segmentation aims to discover and localize semantically meaningful categories within image corpora without any form of annotation.
Ranked #4 on
Unsupervised Semantic Segmentation
on Potsdam-3
IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images
We propose a neural inverse rendering pipeline called IRON that operates on photometric images and outputs high-quality 3D content in the format of triangle meshes and material textures readily deployable in existing graphics pipelines.
Deformable Sprites for Unsupervised Video Decomposition
We describe a method to extract persistent elements of a dynamic scene from an input video.
3D Moments from Near-Duplicate Photos
As output, we produce a video that smoothly interpolates the scene motion from the first photo to the second, while also producing camera motion with parallax that gives a heightened sense of 3D.
Neural 3D Reconstruction in the Wild
We are witnessing an explosion of neural implicit representations in computer vision and graphics.
ARF: Artistic Radiance Fields
We present a method for transferring the artistic features of an arbitrary style image to a 3D scene.
InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images
We present a method for learning to generate unbounded flythrough videos of natural scenes starting from a single view, where this capability is learned from a collection of single photographs, without requiring camera poses or even multiple views of each scene.
im2nerf: Image to Neural Radiance Field in the Wild
We take a step towards addressing this shortcoming by introducing a model that encodes the input image into a disentangled object representation that contains a code for object shape, a code for object appearance, and an estimated camera pose from which the object image is captured.
What's in a Decade? Transforming Faces Through Time
How can one visually characterize people in a decade?
FactorMatte: Redefining Video Matting for Re-Composition Tasks
Based on this observation, we present a method for solving the factor matting problem that produces useful decompositions even for video with complex cross-layer interactions like splashes, shadows, and reflections.
DynIBaR: Neural Dynamic Image-Based Rendering
Our system retains the advantages of prior methods in its ability to model complex scenes and view-dependent effects, but also enables synthesizing photo-realistic novel views from long videos featuring complex scene dynamics with unconstrained camera trajectories.
Seeing a Rose in Five Thousand Ways
These instances all share the same intrinsics, but appear different due to a combination of variance within these intrinsics and differences in extrinsic factors, such as pose and illumination.
Omnimatte3D: Associating Objects and Their Effects in Unconstrained Monocular Video
Instead, our method applies recent progress in monocular camera pose and depth estimation to create a full, RGBD video layer for the background, along with a video layer for each foreground object.
Accidental Light Probes
Recovering lighting in a scene from a single image is a fundamental problem in computer vision.
Persistent Nature: A Generative Model of Unbounded 3D Worlds
Despite increasingly realistic image quality, recent 3D image generative models often operate on 3D volumes of fixed extent with limited camera motions.
Ranked #2 on
Scene Generation
on GoogleEarth
ASIC: Aligning Sparse in-the-wild Image Collections
We present a self-supervised technique that directly optimizes on a sparse collection of images of a particular object/object category to obtain consistent dense correspondences across the collection.
Neural Lens Modeling
Recent methods for 3D reconstruction and rendering increasingly benefit from end-to-end optimization of the entire image formation process.
Tracking Everything Everywhere All at Once
We present a new test-time optimization method for estimating dense and long-range motion from a video sequence.
Neural Scene Chronology
Specifically, we represent the scene as a space-time radiance field with a per-image illumination embedding, where temporally-varying scene changes are encoded using a set of learned step functions.
Doppelgangers: Learning to Disambiguate Images of Similar Structures
Our evaluation shows that our method can distinguish illusory matches in difficult cases, and can be integrated into SfM pipelines to produce correct, disambiguated 3D reconstructions.
WonderJourney: Going from Anywhere to Everywhere
We introduce WonderJourney, a modularized framework for perpetual 3D scene generation.
NeRFiller: Completing Scenes via Generative 3D Inpainting
In contrast to related works, we focus on completing scenes rather than deleting foreground objects, and our approach does not require tight 2D object masks or text.
PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation
Unlike unconditional or text-conditioned dynamics generation, action-conditioned dynamics requires perceiving the physical material properties of objects and grounding the 3D motion prediction on these properties, such as object stiffness.
