Search Results for author: Noah Snavely

Found 52 papers, 27 papers with code

Who's Waldo? Linking People Across Text and Images

no code implementations ICCV 2021 Claire Yuqing Cui, Apoorv Khandelwal, Yoav Artzi, Noah Snavely, Hadar Averbuch-Elor

We present a task and benchmark dataset for person-centric visual grounding, the problem of linking between people named in a caption and people pictured in an image.

 Ranked #1 on Person-centric Visual Grounding on Who’s Waldo (using extra training data)

Person-centric Visual Grounding

Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision

1 code implementation ICCV 2021 Xiaoshi Wu, Hadar Averbuch-Elor, Jin Sun, Noah Snavely

The abundance and richness of Internet photos of landmarks and cities has led to significant progress in 3D vision over the past two decades, including automated 3D reconstructions of the world's landmarks from tourist photos.

Image Captioning

Wide-Baseline Relative Camera Pose Estimation with Directional Learning

no code implementations CVPR 2021 Kefan Chen, Noah Snavely, Ameesh Makadia

Modern deep learning techniques that regress the relative camera pose between two images have difficulty dealing with challenging scenarios, such as large camera motions resulting in occlusions and significant changes in perspective that leave little overlap between images.

Pose Estimation

Extreme Rotation Estimation using Dense Correlation Volumes

1 code implementation CVPR 2021 Ruojin Cai, Bharath Hariharan, Noah Snavely, Hadar Averbuch-Elor

We present a technique for estimating the relative 3D rotation of an RGB image pair in an extreme setting, where the images have little or no overlap.

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

no code implementations CVPR 2021 Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa

We cast this as the problem of aligning a source 3D object to a target 3D object from the same object category.

De-rendering the World's Revolutionary Artefacts

1 code implementation CVPR 2021 Shangzhe Wu, Ameesh Makadia, Jiajun Wu, Noah Snavely, Richard Tucker, Angjoo Kanazawa

Recent works have shown exciting results in unsupervised image de-rendering -- learning to decompose 3D shape, appearance, and lighting from single-image collections without explicit supervision.

PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting

no code implementations CVPR 2021 Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, Noah Snavely

We present PhySG, an end-to-end inverse rendering pipeline that includes a fully differentiable renderer and can reconstruct geometry, materials, and illumination from scratch from a set of RGB input images.

Repopulating Street Scenes

no code implementations CVPR 2021 Yifan Wang, Andrew Liu, Richard Tucker, Jiajun Wu, Brian L. Curless, Steven M. Seitz, Noah Snavely

We present a framework for automatically reconfiguring images of street scenes by populating, depopulating, or repopulating them with objects such as pedestrians or vehicles.

Autonomous Driving

IBRNet: Learning Multi-View Image-Based Rendering

no code implementations CVPR 2021 Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, Thomas Funkhouser

Unlike neural scene representation work that optimizes per-scene functions for rendering, we learn a generic view interpolation function that generalizes to novel scenes.

Neural Rendering Novel View Synthesis

Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image

1 code implementation ICCV 2021 Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, Angjoo Kanazawa

We introduce the problem of perpetual view generation - long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image.

Image Generation Video Generation

An Ethical Highlighter for People-Centric Dataset Creation

no code implementations27 Nov 2020 Margot Hanley, Apoorv Khandelwal, Hadar Averbuch-Elor, Noah Snavely, Helen Nissenbaum

Important ethical concerns arising from computer vision datasets of people have been receiving significant attention, and a number of datasets have been withdrawn as a result.

Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes

2 code implementations CVPR 2021 Zhengqi Li, Simon Niklaus, Noah Snavely, Oliver Wang

We present a method to perform novel view and time synthesis of dynamic scenes, requiring only a monocular video with known camera poses as input.

Multi-Plane Program Induction with 3D Box Priors

no code implementations NeurIPS 2020 Yikai Li, Jiayuan Mao, Xiuming Zhang, William T. Freeman, Joshua B. Tenenbaum, Noah Snavely, Jiajun Wu

We consider two important aspects in understanding and editing images: modeling regular, program-like texture or patterns in 2D planes, and 3D posing of these planes in the scene.

Program induction Program Synthesis

NeRF++: Analyzing and Improving Neural Radiance Fields

1 code implementation15 Oct 2020 Kai Zhang, Gernot Riegler, Noah Snavely, Vladlen Koltun

Neural Radiance Fields (NeRF) achieve impressive view synthesis results for a variety of capture settings, including 360 capture of bounded scenes and forward-facing capture of bounded and unbounded scenes.

Hidden Footprints: Learning Contextual Walkability from 3D Human Trails

no code implementations ECCV 2020 Jin Sun, Hadar Averbuch-Elor, Qianqian Wang, Noah Snavely

Predicting where people can walk in a scene is important for many tasks, including autonomous driving systems and human behavior analysis.

Autonomous Driving

Learning to Factorize and Relight a City

no code implementations ECCV 2020 Andrew Liu, Shiry Ginosar, Tinghui Zhou, Alexei A. Efros, Noah Snavely

We propose a learning-based framework for disentangling outdoor scenes into temporally-varying illumination and permanent scene factors.

Intrinsic Image Decomposition

Crowdsampling the Plenoptic Function

1 code implementation ECCV 2020 Zhengqi Li, Wenqi Xian, Abe Davis, Noah Snavely

These photos represent a sparse and unstructured sampling of the plenoptic function for a particular scene.

Neural Rendering Novel View Synthesis

An Analysis of SVD for Deep Rotation Estimation

2 code implementations NeurIPS 2020 Jake Levinson, Carlos Esteves, Kefan Chen, Noah Snavely, Angjoo Kanazawa, Afshin Rostamizadeh, Ameesh Makadia

Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$.

3D Pose Estimation 3D Rotation Estimation

MetaSDF: Meta-learning Signed Distance Functions

1 code implementation NeurIPS 2020 Vincent Sitzmann, Eric R. Chan, Richard Tucker, Noah Snavely, Gordon Wetzstein

Neural implicit shape representations are an emerging paradigm that offers many potential benefits over conventional discrete representations, including memory efficiency at a high spatial resolution.

Meta-Learning

Learning Feature Descriptors using Camera Pose Supervision

1 code implementation ECCV 2020 Qianqian Wang, Xiaowei Zhou, Bharath Hariharan, Noah Snavely

Recent research on learned visual descriptors has shown promising improvements in correspondence estimation, a key component of many 3D vision tasks.

Single-View View Synthesis with Multiplane Images

no code implementations CVPR 2020 Richard Tucker, Noah Snavely

A recent strand of work in view synthesis uses deep learning to generate multiplane images (a camera-centric, layered 3D representation) given two or more input images at known viewpoints.

Depth Sensing Beyond LiDAR Range

no code implementations CVPR 2020 Kai Zhang, Jiaxin Xie, Noah Snavely, Qifeng Chen

Depth sensing is a critical component of autonomous driving technologies, but today's LiDAR- or stereo camera-based solutions have limited range.

Autonomous Driving

Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination

1 code implementation CVPR 2020 Pratul P. Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T. Barron, Richard Tucker, Noah Snavely

We present a deep learning solution for estimating the incident illumination at any 3D location within a scene from an input narrow-baseline stereo image pair.

GeoStyle: Discovering Fashion Trends and Events

1 code implementation ICCV 2019 Utkarsh Mall, Kevin Matzen, Bharath Hariharan, Noah Snavely, Kavita Bala

Understanding fashion styles and trends is of great potential interest to retailers and consumers alike.

UprightNet: Geometry-Aware Camera Orientation Estimation from Single Images

no code implementations ICCV 2019 Wenqi Xian, Zhengqi Li, Matthew Fisher, Jonathan Eisenmann, Eli Shechtman, Noah Snavely

We introduce UprightNet, a learning-based approach for estimating 2DoF camera orientation from a single RGB image of an indoor scene.

Pushing the Boundaries of View Extrapolation with Multiplane Images

1 code implementation CVPR 2019 Pratul P. Srinivasan, Richard Tucker, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng, Noah Snavely

We present a theoretical analysis showing how the range of views that can be rendered from an MPI increases linearly with the MPI disparity sampling frequency, as well as a novel MPI prediction procedure that theoretically enables view extrapolations of up to $4\times$ the lateral viewpoint movement allowed by prior work.

Learning the Depths of Moving People by Watching Frozen People

no code implementations CVPR 2019 Zhengqi Li, Tali Dekel, Forrester Cole, Richard Tucker, Noah Snavely, Ce Liu, William T. Freeman

We present a method for predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving.

Depth Estimation

Neural Rerendering in the Wild

no code implementations CVPR 2019 Moustafa Meshry, Dan B. Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, Ricardo Martin-Brualla

Starting from internet photos of a tourist landmark, we apply traditional 3D reconstruction to register the photos and approximate the scene as a point cloud.

3D Reconstruction

Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments

4 code implementations CVPR 2019 Howard Chen, Alane Suhr, Dipendra Misra, Noah Snavely, Yoav Artzi

We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task.

CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering

no code implementations ECCV 2018 Zhengqi Li, Noah Snavely

Intrinsic image decomposition is a challenging, long-standing computer vision problem for which ground truth data is very difficult to acquire.

Intrinsic Image Decomposition

Layer-structured 3D Scene Inference via View Synthesis

1 code implementation ECCV 2018 Shubham Tulsiani, Richard Tucker, Noah Snavely

We present an approach to infer a layer-structured 3D representation of a scene from a single input image.

Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning

1 code implementation NeurIPS 2018 Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi

We demonstrate this framework on 3D pose estimation by proposing a differentiable objective that seeks the optimal set of keypoints for recovering the relative pose between two views of an object.

3D Pose Estimation

Stereo Magnification: Learning View Synthesis using Multiplane Images

1 code implementation24 May 2018 Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, Noah Snavely

The view synthesis problem--generating novel views of a scene from known imagery--has garnered recent attention due in part to compelling applications in virtual and augmented reality.

Novel View Synthesis

MegaDepth: Learning Single-View Depth Prediction from Internet Photos

1 code implementation CVPR 2018 Zhengqi Li, Noah Snavely

We validate the use of large amounts of Internet data by showing that models trained on MegaDepth exhibit strong generalization-not only to novel scenes, but also to other diverse datasets including Make3D, KITTI, and DIW, even when no images from those datasets are seen during training.

Depth Estimation Semantic Segmentation +1

StreetStyle: Exploring world-wide clothing styles from millions of photos

2 code implementations6 Jun 2017 Kevin Matzen, Kavita Bala, Noah Snavely

Each day billions of photographs are uploaded to photo-sharing services and social media platforms.

Shading Annotations in the Wild

no code implementations CVPR 2017 Balazs Kovacs, Sean Bell, Noah Snavely, Kavita Bala

We demonstrate the value of our data and network in an application to intrinsic images, where we can reduce decomposition artifacts produced by existing algorithms.

Image Relighting Intrinsic Image Decomposition +1

Unsupervised Learning of Depth and Ego-Motion from Video

2 code implementations CVPR 2017 Tinghui Zhou, Matthew Brown, Noah Snavely, David G. Lowe

We present an unsupervised learning framework for the task of monocular depth and camera motion estimation from unstructured video sequences.

Depth And Camera Motion Motion Estimation +1

Deep Feature Interpolation for Image Content Changes

2 code implementations CVPR 2017 Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, Kilian Weinberger

We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation.

From A to Z: Supervised Transfer of Style and Content Using Deep Neural Network Generators

no code implementations7 Mar 2016 Paul Upchurch, Noah Snavely, Kavita Bala

We propose a new neural network architecture for solving single-image analogies - the generation of an entire set of stylistically similar images from just a single input image.

BubbLeNet: Foveated Imaging for Visual Discovery

no code implementations ICCV 2015 Kevin Matzen, Noah Snavely

We propose a new method for turning an Internet-scale corpus of categorized images into a small set of human-interpretable discriminative visual elements using powerful tools based on deep learning.

DeepStereo: Learning to Predict New Views from the World's Imagery

1 code implementation CVPR 2016 John Flynn, Ivan Neulander, James Philbin, Noah Snavely

To our knowledge, our work is the first to apply deep learning to the problem of new view synthesis from sets of real-world, natural imagery.

Material Recognition in the Wild with the Materials in Context Database

no code implementations CVPR 2015 Sean Bell, Paul Upchurch, Noah Snavely, Kavita Bala

In this paper, we introduce a new, large-scale, open dataset of materials in the wild, the Materials in Context Database (MINC), and combine this dataset with deep learning to achieve material recognition and segmentation of images in the wild.

Material Recognition

Graph-Based Discriminative Learning for Location Recognition

no code implementations CVPR 2013 Song Cao, Noah Snavely

For a query image, each database image is ranked according to these local distance functions in order to place the image in the right part of the graph.

Photometric Ambient Occlusion

no code implementations CVPR 2013 Daniel Hauagge, Scott Wehrwein, Kavita Bala, Noah Snavely

We present a method for computing ambient occlusion (AO) for a stack of images of a scene from a fixed viewpoint.

Cannot find the paper you are looking for? You can Submit a new open access paper.