no code implementations • CVPR 2023 • Wenqi Xian, Aljaž Božič, Noah Snavely, Christoph Lassner
Recent methods for 3D reconstruction and rendering increasingly benefit from end-to-end optimization of the entire image formation process.
no code implementations • 28 Mar 2023 • Kamal Gupta, Varun Jampani, Carlos Esteves, Abhinav Shrivastava, Ameesh Makadia, Noah Snavely, Abhishek Kar
We present a self-supervised technique that directly optimizes on a sparse collection of images of a particular object/object category to obtain consistent dense correspondences across the collection.
no code implementations • CVPR 2023 • Lucy Chai, Richard Tucker, Zhengqi Li, Phillip Isola, Noah Snavely
Despite increasingly realistic image quality, recent 3D image generative models often operate on 3D volumes of fixed extent with limited camera motions.
no code implementations • CVPR 2023 • Hong-Xing Yu, Samir Agarwala, Charles Herrmann, Richard Szeliski, Noah Snavely, Jiajun Wu, Deqing Sun
Recovering lighting in a scene from a single image is a fundamental problem in computer vision.
no code implementations • CVPR 2023 • Haotong Lin, Qianqian Wang, Ruojin Cai, Sida Peng, Hadar Averbuch-Elor, Xiaowei Zhou, Noah Snavely
To tackle these problems, we propose a new scene representation equipped with a novel temporal step function encoding method that can model discrete scene-level content changes as piece-wise constant functions over time.
no code implementations • CVPR 2023 • Mohammed Suhail, Erika Lu, Zhengqi Li, Noah Snavely, Leonid Sigal, Forrester Cole
Instead, our method applies recent progress in monocular camera pose and depth estimation to create a full, RGBD video layer for the background, along with a video layer for each foreground object.
no code implementations • CVPR 2023 • Yunzhi Zhang, Shangzhe Wu, Noah Snavely, Jiajun Wu
These instances all share the same intrinsics, but appear different due to a combination of variance within these intrinsics and differences in extrinsic factors, such as pose and illumination.
no code implementations • CVPR 2023 • Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, Noah Snavely
Our system retains the advantages of prior methods in its ability to model complex scenes and view-dependent effects, but also enables synthesizing photo-realistic novel views from long videos featuring complex scene dynamics with unconstrained camera trajectories.
no code implementations • 3 Nov 2022 • Zeqi Gu, Wenqi Xian, Noah Snavely, Abe Davis
Based on this observation, we present a method for solving the factor matting problem that produces useful decompositions even for video with complex cross-layer interactions like splashes, shadows, and reflections.
no code implementations • 13 Oct 2022 • Eric Ming Chen, Jin Sun, Apoorv Khandelwal, Dani Lischinski, Noah Snavely, Hadar Averbuch-Elor
How can one visually characterize people in a decade?
no code implementations • 8 Sep 2022 • Lu Mi, Abhijit Kundu, David Ross, Frank Dellaert, Noah Snavely, Alireza Fathi
We take a step towards addressing this shortcoming by introducing a model that encodes the input image into a disentangled object representation that contains a code for object shape, a code for object appearance, and an estimated camera pose from which the object image is captured.
1 code implementation • 22 Jul 2022 • Zhengqi Li, Qianqian Wang, Noah Snavely, Angjoo Kanazawa
We present a method for learning to generate unbounded flythrough videos of natural scenes starting from a single view, where this capability is learned from a collection of single photographs, without requiring camera poses or even multiple views of each scene.
1 code implementation • 13 Jun 2022 • Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, Noah Snavely
We present a method for transferring the artistic features of an arbitrary style image to a 3D scene.
no code implementations • 25 May 2022 • Jiaming Sun, Xi Chen, Qianqian Wang, Zhengqi Li, Hadar Averbuch-Elor, Xiaowei Zhou, Noah Snavely
We are witnessing an explosion of neural implicit representations in computer vision and graphics.
no code implementations • CVPR 2022 • Qianqian Wang, Zhengqi Li, David Salesin, Noah Snavely, Brian Curless, Janne Kontkanen
As output, we produce a video that smoothly interpolates the scene motion from the first photo to the second, while also producing camera motion with parallax that gives a heightened sense of 3D.
no code implementations • CVPR 2022 • Vickie Ye, Zhengqi Li, Richard Tucker, Angjoo Kanazawa, Noah Snavely
We describe a method to extract persistent elements of a dynamic scene from an input video.
no code implementations • CVPR 2022 • Kai Zhang, Fujun Luan, Zhengqi Li, Noah Snavely
We propose a neural inverse rendering pipeline called IRON that operates on photometric images and outputs high-quality 3D content in the format of triangle meshes and material textures readily deployable in existing graphics pipelines.
2 code implementations • ICLR 2022 • Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, William T. Freeman
Unsupervised semantic segmentation aims to discover and localize semantically meaningful categories within image corpora without any form of annotation.
Ranked #1 on Unsupervised Semantic Segmentation on COCO-Stuff
no code implementations • 2 Dec 2021 • Richard Strong Bowen, Richard Tucker, Ramin Zabih, Noah Snavely
We introduce a way to learn to estimate a scene representation from a single image by predicting a low-dimensional subspace of optical flow for each training example, which encompasses the variety of possible camera and object movement.
1 code implementation • ICCV 2021 • Claire Yuqing Cui, Apoorv Khandelwal, Yoav Artzi, Noah Snavely, Hadar Averbuch-Elor
We present a task and benchmark dataset for person-centric visual grounding, the problem of linking between people named in a caption and people pictured in an image.
Ranked #1 on Person-centric Visual Grounding on Who’s Waldo (using extra training data)
1 code implementation • ICCV 2021 • Xiaoshi Wu, Hadar Averbuch-Elor, Jin Sun, Noah Snavely
The abundance and richness of Internet photos of landmarks and cities has led to significant progress in 3D vision over the past two decades, including automated 3D reconstructions of the world's landmarks from tourist photos.
no code implementations • CVPR 2021 • Kefan Chen, Noah Snavely, Ameesh Makadia
Modern deep learning techniques that regress the relative camera pose between two images have difficulty dealing with challenging scenarios, such as large camera motions resulting in occlusions and significant changes in perspective that leave little overlap between images.
1 code implementation • CVPR 2021 • Ruojin Cai, Bharath Hariharan, Noah Snavely, Hadar Averbuch-Elor
We present a technique for estimating the relative 3D rotation of an RGB image pair in an extreme setting, where the images have little or no overlap.
no code implementations • CVPR 2021 • Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa
We cast this as the problem of aligning a source 3D object to a target 3D object from the same object category.
1 code implementation • CVPR 2021 • Shangzhe Wu, Ameesh Makadia, Jiajun Wu, Noah Snavely, Richard Tucker, Angjoo Kanazawa
Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision.
no code implementations • CVPR 2021 • Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, Noah Snavely
We present PhySG, an end-to-end inverse rendering pipeline that includes a fully differentiable renderer and can reconstruct geometry, materials, and illumination from scratch from a set of RGB input images.
no code implementations • CVPR 2021 • Yifan Wang, Andrew Liu, Richard Tucker, Jiajun Wu, Brian L. Curless, Steven M. Seitz, Noah Snavely
We present a framework for automatically reconfiguring images of street scenes by populating, depopulating, or repopulating them with objects such as pedestrians or vehicles.
1 code implementation • CVPR 2021 • Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, Thomas Funkhouser
Unlike neural scene representation work that optimizes per-scene functions for rendering, we learn a generic view interpolation function that generalizes to novel scenes.
1 code implementation • ICCV 2021 • Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, Angjoo Kanazawa
We introduce the problem of perpetual view generation - long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image.
no code implementations • 27 Nov 2020 • Margot Hanley, Apoorv Khandelwal, Hadar Averbuch-Elor, Noah Snavely, Helen Nissenbaum
Important ethical concerns arising from computer vision datasets of people have been receiving significant attention, and a number of datasets have been withdrawn as a result.
3 code implementations • CVPR 2021 • Zhengqi Li, Simon Niklaus, Noah Snavely, Oliver Wang
We present a method to perform novel view and time synthesis of dynamic scenes, requiring only a monocular video with known camera poses as input.
no code implementations • NeurIPS 2020 • Yikai Li, Jiayuan Mao, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu
We consider two important aspects in understanding and editing images: modeling regular, program-like texture or patterns in 2D planes, and 3D posing of these planes in the scene.
5 code implementations • 15 Oct 2020 • Kai Zhang, Gernot Riegler, Noah Snavely, Vladlen Koltun
Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings, including 360 capture of bounded scenes and forward-facing capture of bounded and unbounded scenes.
no code implementations • ECCV 2020 • Jin Sun, Hadar Averbuch-Elor, Qianqian Wang, Noah Snavely
Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis.
1 code implementation • ECCV 2020 • Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, Bharath Hariharan
Point cloud generation thus amounts to moving randomly sampled points to high-density areas.
no code implementations • ECCV 2020 • Andrew Liu, Shiry Ginosar, Tinghui Zhou, Alexei A. Efros, Noah Snavely
We propose a learning-based framework for disentangling outdoor scenes into temporally-varying illumination and permanent scene factors.
1 code implementation • ECCV 2020 • Zhengqi Li, Wenqi Xian, Abe Davis, Noah Snavely
These photos represent a sparse and unstructured sampling of the plenoptic function for a particular scene.
2 code implementations • NeurIPS 2020 • Jake Levinson, Carlos Esteves, Kefan Chen, Noah Snavely, Angjoo Kanazawa, Afshin Rostamizadeh, Ameesh Makadia
Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$.
2 code implementations • NeurIPS 2020 • Vincent Sitzmann, Eric R. Chan, Richard Tucker, Noah Snavely, Gordon Wetzstein
Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution.
1 code implementation • CVPR 2020 • Zhiqiu Lin, Jin Sun, Abe Davis, Noah Snavely
How can we tell whether an image has been mirrored?
1 code implementation • ECCV 2020 • Qianqian Wang, Xiaowei Zhou, Bharath Hariharan, Noah Snavely
Recent research on learned visual descriptors has shown promising improvements in correspondence estimation, a key component of many 3D vision tasks.
5 code implementations • 24 Apr 2020 • Ranjita Thapa, Noah Snavely, Serge Belongie, Awais Khan
Appropriate and timely deployment of disease management depends on early disease detection.
no code implementations • CVPR 2020 • Richard Tucker, Noah Snavely
A recent strand of work in view synthesis uses deep learning to generate multiplane images (a camera-centric, layered 3D representation) given two or more input images at known viewpoints.
no code implementations • CVPR 2020 • Kai Zhang, Jiaxin Xie, Noah Snavely, Qifeng Chen
Depth sensing is a critical component of autonomous driving technologies, but today's LiDAR- or stereo camera-based solutions have limited range.
1 code implementation • CVPR 2020 • Zekun Hao, Hadar Averbuch-Elor, Noah Snavely, Serge Belongie
We are seeing a Cambrian explosion of 3D shape representations for use in machine learning.
1 code implementation • CVPR 2020 • Pratul P. Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T. Barron, Richard Tucker, Noah Snavely
We present a deep learning solution for estimating the incident illumination at any 3D location within a scene from an input narrow-baseline stereo image pair.
no code implementations • 7 Oct 2019 • Kai Zhang, Jin Sun, Noah Snavely
Reconstructing 3D geometry from satellite imagery is an important topic of research.
1 code implementation • ICCV 2019 • Utkarsh Mall, Kevin Matzen, Bharath Hariharan, Noah Snavely, Kavita Bala
Understanding fashion styles and trends is of great potential interest to retailers and consumers alike.
no code implementations • ICCV 2019 • Wenqi Xian, Zhengqi Li, Matthew Fisher, Jonathan Eisenmann, Eli Shechtman, Noah Snavely
We introduce UprightNet, a learning-based approach for estimating 2DoF camera orientation from a single RGB image of an indoor scene.
no code implementations • CVPR 2019 • John Flynn, Michael Broxton, Paul Debevec, Matthew DuVall, Graham Fyffe, Ryan Overbeck, Noah Snavely, Richard Tucker
We present a novel approach to view synthesis using multiplane images (MPIs).
1 code implementation • CVPR 2019 • Pratul P. Srinivasan, Richard Tucker, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng, Noah Snavely
We present a theoretical analysis showing how the range of views that can be rendered from an MPI increases linearly with the MPI disparity sampling frequency, as well as a novel MPI prediction procedure that theoretically enables view extrapolations of up to $4\times$ the lateral viewpoint movement allowed by prior work.
no code implementations • CVPR 2019 • Zhengqi Li, Tali Dekel, Forrester Cole, Richard Tucker, Noah Snavely, Ce Liu, William T. Freeman
We present a method for predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving.
no code implementations • CVPR 2019 • Moustafa Meshry, Dan B. Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, Ricardo Martin-Brualla
Starting from internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud.
4 code implementations • CVPR 2019 • Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, Yoav Artzi
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task.
Ranked #10 on Vision and Language Navigation on Touchdown Dataset
no code implementations • ECCV 2018 • Zhengqi Li, Noah Snavely
Intrinsic image decomposition is a challenging, long-standing computer vision problem for which ground truth data is very difficult to acquire.
1 code implementation • ECCV 2018 • Shubham Tulsiani, Richard Tucker, Noah Snavely
We present an approach to infer a layer-structured 3D representation of a scene from a single input image.
1 code implementation • NeurIPS 2018 • Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi
We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object.
1 code implementation • 24 May 2018 • Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, Noah Snavely
The view synthesis problem--generating novel views of a scene from known imagery--has garnered recent attention due in part to compelling applications in virtual and augmented reality.
1 code implementation • CVPR 2018 • Zhengqi Li, Noah Snavely
We validate the use of large amounts of Internet data by showing that models trained on MegaDepth exhibit strong generalization-not only to novel scenes, but also to other diverse datasets including Make3D, KITTI, and DIW, even when no images from those datasets are seen during training.
1 code implementation • CVPR 2018 • Zhengqi Li, Noah Snavely
However, it is difficult to collect ground truth training data at scale for intrinsic images.
2 code implementations • 6 Jun 2017 • Kevin Matzen, Kavita Bala, Noah Snavely
Each day billions of photographs are uploaded to photo-sharing services and social media platforms.
no code implementations • CVPR 2017 • Balazs Kovacs, Sean Bell, Noah Snavely, Kavita Bala
We demonstrate the value of our data and network in an application to intrinsic images, where we can reduce decomposition artifacts produced by existing algorithms.
2 code implementations • CVPR 2017 • Tinghui Zhou, Matthew Brown, Noah Snavely, David G. Lowe
We present an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences.
2 code implementations • CVPR 2017 • Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, Kilian Weinberger
We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation.
no code implementations • 7 Mar 2016 • Paul Upchurch, Noah Snavely, Kavita Bala
We propose a new neural network architecture for solving single-image analogies - the generation of an entire set of stylistically similar images from just a single input image.
no code implementations • ICCV 2015 • Kevin Matzen, Noah Snavely
We propose a new method for turning an Internet-scale corpus of categorized images into a small set of human-interpretable discriminative visual elements using powerful tools based on deep learning.
1 code implementation • CVPR 2016 • John Flynn, Ivan Neulander, James Philbin, Noah Snavely
To our knowledge, our work is the first to apply deep learning to the problem of new view synthesis from sets of real-world, natural imagery.
no code implementations • CVPR 2015 • Sean Bell, Paul Upchurch, Noah Snavely, Kavita Bala
In this paper, we introduce a new, large-scale, open dataset of materials in the wild, the Materials in Context Database (MINC), and combine this dataset with deep learning to achieve material recognition and segmentation of images in the wild.
no code implementations • CVPR 2014 • Song Cao, Noah Snavely
How much data do we need to describe a location?
no code implementations • CVPR 2013 • Daniel Hauagge, Scott Wehrwein, Kavita Bala, Noah Snavely
We present a method for computing ambient occlusion (AO) for a stack of images of a scene from a fixed viewpoint.
no code implementations • CVPR 2013 • Song Cao, Noah Snavely
For a query image, each database image is ranked according to these local distance functions in order to place the image in the right part of the graph.