Search Results for author: Jan-Michael Frahm

Found 38 papers, 11 papers with code

A Surface-normal Based Neural Framework for Colonoscopy Reconstruction

no code implementations13 Mar 2023 Shuxian Wang, Yubo Zhang, Sarah K. McGill, Julian G. Rosenman, Jan-Michael Frahm, Soumyadip Sengupta, Stephen M. Pizer

Reconstructing a 3D surface from colonoscopy video is challenging due to illumination and reflectivity variation in the video frame that can cause defective shape predictions.

A Practical Stereo Depth System for Smart Glasses

no code implementations CVPR 2023 Jialiang Wang, Daniel Scharstein, Akash Bapat, Kevin Blackburn-Matzen, Matthew Yu, Jonathan Lehman, Suhib Alsisan, Yanghan Wang, Sam Tsai, Jan-Michael Frahm, Zijian He, Peter Vajda, Michael F. Cohen, Matt Uyttendaele

We present the design of a productionized end-to-end stereo depth sensing system that does pre-processing, online stereo rectification, and stereo depth estimation with a fallback to monocular depth estimation when rectification is unreliable.

Monocular Depth Estimation Stereo Depth Estimation

Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search

no code implementations4 Oct 2022 Thanh Vu, Yanqi Zhou, Chunfeng Wen, Yueqi Li, Jan-Michael Frahm

Our first key insight is that MultiTask Learning (MTL) and hardware-aware Neural Architecture Search (NAS) can work in synergy to greatly benefit on-device Dense Predictions (DP).

Hardware Aware Neural Architecture Search Neural Architecture Search +2

Leveraging Disentangled Representations to Improve Vision-Based Keystroke Inference Attacks Under Low Data

1 code implementation5 Apr 2022 John Lim, Jan-Michael Frahm, Fabian Monrose

We evaluate our method on real-life data using a variety of metrics to quantify the amount of information an attacker is able to recover.

Data Augmentation Domain Adaptation

VPFusion: Joint 3D Volume and Pixel-Aligned Feature Fusion for Single and Multi-view 3D Reconstruction

no code implementations14 Mar 2022 Jisan Mahmud, Jan-Michael Frahm

VPFusion attains high-quality reconstruction using both - 3D feature volume to capture 3D-structure-aware context, and pixel-aligned image features to capture fine local detail.

3D Reconstruction 3D Shape Reconstruction +1

ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction

no code implementations19 Nov 2021 Yubo Zhang, Jan-Michael Frahm, Samuel Ehrenstein, Sarah K. McGill, Julian G. Rosenman, Shuxian Wang, Stephen M. Pizer

Aiming to fundamentally improve the depth estimation quality for colonoscopy 3D reconstruction, in this work we have designed a set of training losses to deal with the special challenges of colonoscopy data.

3D Reconstruction Depth Estimation +1

Any-Width Networks

1 code implementation6 Dec 2020 Thanh Vu, Marc Eder, True Price, Jan-Michael Frahm

To address these constraints, we propose the Any-Width Network (AWN), an adjustable-width CNN architecture and associated training routine that allow for fine-grained control over speed and accuracy during inference.

Revisiting the Threat Space for Vision-based Keystroke Inference Attacks

1 code implementation12 Sep 2020 John Lim, True Price, Fabian Monrose, Jan-Michael Frahm

This indicates that these models are able to learn rich, meaningful representations from our synthetic data and that training on the synthetic data can help overcome the issue of having small, real-life datasets for vision-based key stroke inference attacks.

Domain Adaptation Inference Attack +1

One Shot 3D Photography

1 code implementation27 Aug 2020 Johannes Kopf, Kevin Matzen, Suhib Alsisan, Ocean Quigley, Francis Ge, Yangming Chong, Josh Patterson, Jan-Michael Frahm, Shu Wu, Matthew Yu, Peizhao Zhang, Zijian He, Peter Vajda, Ayush Saraf, Michael Cohen

3D photos are static in time, like traditional photos, but are displayed with interactive parallax on mobile or desktop screens, as well as on Virtual Reality devices, where viewing it also includes stereo.

Monocular Depth Estimation

Reducing Drift in Structure From Motion Using Extended Features

no code implementations27 Aug 2020 Aleksander Holynski, David Geraghty, Jan-Michael Frahm, Chris Sweeney, Richard Szeliski

Low-frequency long-range errors (drift) are an endemic problem in 3D structure from motion, and can often hamper reasonable reconstructions of the scene.

Tangent Images for Mitigating Spherical Distortion

1 code implementation CVPR 2020 Marc Eder, Mykhailo Shvets, John Lim, Jan-Michael Frahm

In this work, we propose "tangent images," a spherical image representation that facilitates transferable and scalable $360^\circ$ computer vision.

Semantic Segmentation

ViewSynth: Learning Local Features from Depth using View Synthesis

1 code implementation22 Nov 2019 Jisan Mahmud, Rajat Vikram Singh, Peri Akiva, Spondon Kundu, Kuan-Chuan Peng, Jan-Michael Frahm

By learning view synthesis, we explicitly encourage the feature extractor to encode information about not only the visible, but also the occluded parts of the scene.

Camera Localization Keypoint Detection

Mapped Convolutions

1 code implementation26 Jun 2019 Marc Eder, True Price, Thanh Vu, Akash Bapat, Jan-Michael Frahm

We present a versatile formulation of the convolution operation that we term a "mapped convolution."

Depth Estimation

Convolutions on Spherical Images

no code implementations21 May 2019 Marc Eder, Jan-Michael Frahm

Applying convolutional neural networks to spherical images requires particular considerations.

Semantic Segmentation

Deep Blending for Free-Viewpoint Image-Based-Rendering

1 code implementation SIGGRAPH Asia 2018 2018 Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, Gabriel Brostow

We present a new deep learning approach to blending for IBR, in which we use held-out real image data to learn blending weights to combine input photo contributions.

Novel View Synthesis

Rolling Shutter and Radial Distortion Are Features for High Frame Rate Multi-Camera Tracking

no code implementations CVPR 2018 Akash Bapat, True Price, Jan-Michael Frahm

In this paper, we introduce a novel multi-camera tracking approach that for the first time jointly leverages the information introduced by rolling shutter and radial distortion as a feature to achieve superior performance with respect to high-frequency camera pose estimation.

Motion Estimation Pose Estimation

Augmenting Crowd-Sourced 3D Reconstructions Using Semantic Detections

no code implementations CVPR 2018 True Price, Johannes L. Schönberger, Zhen Wei, Marc Pollefeys, Jan-Michael Frahm

Image-based 3D reconstruction for Internet photo collections has become a robust technology to produce impressive virtual representations of real-world scenes.

3D Reconstruction

Recurrent Neural Network for Learning DenseDepth and Ego-Motion from Video

no code implementations17 May 2018 Rui Wang, Jan-Michael Frahm, Stephen M. Pizer

Our method produces superior results to the state-of-the-art learning-based, single- or two-view depth estimation methods on both indoor and outdoor benchmark datasets.

3D Reconstruction Depth Estimation +1

The Domain Transform Solver

no code implementations CVPR 2019 Akash Bapat, Jan-Michael Frahm

We present a framework for edge-aware optimization that is an order of magnitude faster than the state of the art while having comparable performance.

Super-Resolution

The Misty Three Point Algorithm for Relative Pose

no code implementations CVPR 2017 Tobias Palmer, Kalle Astrom, Jan-Michael Frahm

There is a significant interest in scene reconstruction from underwater images given its utility for oceanic research and for recreational image manipulation.

Image Manipulation Motion Estimation

Learned Contextual Feature Reweighting for Image Geo-Localization

no code implementations CVPR 2017 Hyo Jin Kim, Enrique Dunn, Jan-Michael Frahm

We address the problem of large scale image geo-localization where the location of an image is estimated by identifying geo-tagged reference images depicting the same place.

Structure-From-Motion Revisited

1 code implementation CVPR 2016 Johannes L. Schonberger, Jan-Michael Frahm

Incremental Structure-from-Motion is a prevalent strategy for 3D reconstruction from unordered image collections.

3D Reconstruction Point Clouds

From Dusk Till Dawn: Modeling in the Dark

no code implementations CVPR 2016 Filip Radenovic, Johannes L. Schonberger, Dinghuang Ji, Jan-Michael Frahm, Ondrej Chum, Jiri Matas

We present an algorithm that leverages the appearance variety to obtain more complete and accurate scene geometry along with consistent multi-illumination appearance information.

Self-expressive Dictionary Learning for Dynamic 3D Reconstruction

no code implementations22 May 2016 Enliang Zheng, Dinghuang Ji, Enrique Dunn, Jan-Michael Frahm

Given the smooth motion of dynamic objects, we observe any element in the dictionary can be well approximated by a sparse linear combination of other elements in the same dictionary (i. e. self-expression).

3D Reconstruction Dictionary Learning

Sparse Dynamic 3D Reconstruction From Unsynchronized Videos

no code implementations ICCV 2015 Enliang Zheng, Dinghuang Ji, Enrique Dunn, Jan-Michael Frahm

We target the sparse 3D reconstruction of dynamic objects observed by multiple unsynchronized video cameras with unknown temporal overlap.

3D Reconstruction Dictionary Learning

Predicting Good Features for Image Geo-Localization Using Per-Bundle VLAD

no code implementations ICCV 2015 Hyo Jin Kim, Enrique Dunn, Jan-Michael Frahm

We address the problem of recognizing a place depicted in a query image by using a large database of geo-tagged images at a city-scale.

Minimal Solvers for 3D Geometry From Satellite Imagery

no code implementations ICCV 2015 Enliang Zheng, Ke Wang, Enrique Dunn, Jan-Michael Frahm

We propose two novel minimal solvers which advance the state of the art in satellite imagery processing.

Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset)

no code implementations CVPR 2015 Jared Heinly, Johannes L. Schonberger, Enrique Dunn, Jan-Michael Frahm

We propose a novel, large-scale, structure-from-motion framework that advances the state of the art in data scalability from city-scale modeling (millions of images) to world-scale modeling (several tens of millions of images) using just a single computer.

Clustering Image Clustering

Adaptive Eye-Camera Calibration for Head-Worn Devices

no code implementations CVPR 2015 David Perra, Rohit Kumar Gupta, Jan-Michael Frahm

Our calibration scheme allows a head-worn device to calculate a locally optimal eye-device transformation on demand by computing an optimal model from a local window of previous frames.

Camera Calibration

PAIGE: PAirwise Image Geometry Encoding for Improved Efficiency in Structure-From-Motion

no code implementations CVPR 2015 Johannes L. Schonberger, Alexander C. Berg, Jan-Michael Frahm

Based on the insights of this evaluation, we propose a learning-based approach, the PAirwise Image Geometry Encoding (PAIGE), to efficiently identify image pairs with scene overlap without the need to perform exhaustive putative matching and geometric verification.

PatchMatch Based Joint View Selection and Depthmap Estimation

no code implementations CVPR 2014 Enliang Zheng, Enrique Dunn, Vladimir Jojic, Jan-Michael Frahm

We propose a multi-view depthmap estimation approach aimed at adaptively ascertaining the pixel level data associations between a reference image and all the elements of a source image set.

Cannot find the paper you are looking for? You can Submit a new open access paper.