Search Results for author: Kwang Moo Yi

Found 46 papers, 21 papers with code

INVE: Interactive Neural Video Editing

no code implementations15 Jul 2023 Jiahui Huang, Leonid Sigal, Kwang Moo Yi, Oliver Wang, Joon-Young Lee

We present Interactive Neural Video Editing (INVE), a real-time video editing solution, which can assist the video editing process by consistently propagating sparse frame edits to the entire video clip.

Video Editing

Unsupervised Semantic Correspondence Using Stable Diffusion

no code implementations24 May 2023 Eric Hedlin, Gopal Sharma, Shweta Mahajan, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi

Text-to-image diffusion models are now capable of generating images that are often indistinguishable from real images.

Semantic correspondence

PPDONet: Deep Operator Networks for Fast Prediction of Steady-State Solutions in Disk-Planet Systems

1 code implementation18 May 2023 Shunyuan Mao, Ruobing Dong, Lu Lu, Kwang Moo Yi, Sifan Wang, Paris Perdikaris

We develop a tool, which we name Protoplanetary Disk Operator Network (PPDONet), that can predict the solution of disk-planet interactions in protoplanetary disks in real-time.

BlendFields: Few-Shot Example-Driven Facial Modeling

no code implementations CVPR 2023 Kacper Kania, Stephan J. Garbin, Andrea Tagliasacchi, Virginia Estellers, Kwang Moo Yi, Julien Valentin, Tomasz Trzciński, Marek Kowalski

Generating faithful visualizations of human faces requires capturing both coarse and fine-level details of the face geometry and appearance.

Pointersect: Neural Rendering with Cloud-Ray Intersection

no code implementations CVPR 2023 Jen-Hao Rick Chang, Wei-Yu Chen, Anurag Ranjan, Kwang Moo Yi, Oncel Tuzel

Specifically, we train a set transformer that, given a small number of local neighbor points along a light ray, provides the intersection point, the surface normal, and the material blending weights, which are used to render the outcome of this light ray.

Inverse Rendering Neural Rendering +2

CN-DHF: Compact Neural Double Height-Field Representations of 3D Shapes

no code implementations29 Mar 2023 Eric Hedlin, Jinfan Yang, Nicholas Vining, Kwang Moo Yi, Alla Sheffer

We introduce CN-DHF (Compact Neural Double-Height-Field), a novel hybrid neural implicit 3D shape representation that is dramatically more compact than the current state of the art.

3D Shape Representation

FaceLit: Neural 3D Relightable Faces

no code implementations CVPR 2023 Anurag Ranjan, Kwang Moo Yi, Jen-Hao Rick Chang, Oncel Tuzel

We propose a generative framework, FaceLit, capable of generating a 3D face that can be rendered at various user-defined lighting conditions and views, learned purely from 2D images in-the-wild without any manual annotation.

Neural Fourier Filter Bank

1 code implementation CVPR 2023 Zhijie Wu, Yuhe Jin, Kwang Moo Yi

We present a novel method to provide efficient and highly detailed reconstructions.

3D Shape Reconstruction

Estimating Visual Information From Audio Through Manifold Learning

1 code implementation3 Aug 2022 Fabrizio Pedersoli, Dryden Wiebe, Amin Banitalebi, Yong Zhang, George Tzanetakis, Kwang Moo Yi

Therefore, audio-based methods can be useful even for applications in which only visual information is of interest Our framework is based on Manifold Learning and consists of two steps.

Semantic Segmentation

TUSK: Task-Agnostic Unsupervised Keypoints

no code implementations16 Jun 2022 Yuhe Jin, Weiwei Sun, Jan Hosang, Eduard Trulls, Kwang Moo Yi

Existing unsupervised methods for keypoint learning rely heavily on the assumption that a specific keypoint type (e. g. elbow, digit, abstract geometric shape) appears only once in an image.

Object Discovery Unsupervised Keypoints

FlowNet-PET: Unsupervised Learning to Perform Respiratory Motion Correction in PET Imaging

1 code implementation27 May 2022 Teaghan O'Briain, Carlos Uribe, Kwang Moo Yi, Jonas Teuwen, Ioannis Sechopoulos, Magdalena Bazalova-Carter

To correct for respiratory motion in PET imaging, an interpretable and unsupervised deep learning technique, FlowNet-PET, was constructed.

Optical Flow Estimation

A Simple Method to Boost Human Pose Estimation Accuracy by Correcting the Joint Regressor for the Human3.6m Dataset

2 code implementations29 Apr 2022 Eric Hedlin, Helge Rhodin, Kwang Moo Yi

While the quality of this pseudo-ground-truth is challenging to assess due to the lack of actual ground-truth SMPL, with the Human 3. 6m dataset, we qualitatively show that our joint locations are more accurate and that our regressor leads to improved pose estimations results on the test set without any need for retraining.

Pose Estimation

NeuMan: Neural Human Radiance Field from a Single Video

1 code implementation23 Mar 2022 Wei Jiang, Kwang Moo Yi, Golnoosh Samei, Oncel Tuzel, Anurag Ranjan

Photorealistic rendering and reposing of humans is important for enabling augmented reality experiences.

CoNeRF: Controllable Neural Radiance Fields

1 code implementation CVPR 2022 Kacper Kania, Kwang Moo Yi, Marek Kowalski, Tomasz Trzciński, Andrea Tagliasacchi

We extend neural 3D representations to allow for intuitive and interpretable user control beyond novel view rendering (i. e. camera control).

3D Face Modelling 3D Reconstruction +1

Layered Controllable Video Generation

no code implementations24 Nov 2021 Jiahui Huang, Yuhe Jin, Kwang Moo Yi, Leonid Sigal

In the first stage, with the rich set of losses and dynamic foreground size prior, we learn how to separate the frame into foreground and background layers and, conditioned on these layers, how to generate the next frame using VQ-VAE generator.

Video Generation

LOLNeRF: Learn from One Look

no code implementations CVPR 2022 Daniel Rebain, Mark Matthews, Kwang Moo Yi, Dmitry Lagun, Andrea Tagliasacchi

We present a method for learning a generative 3D model based on neural radiance fields, trained solely from data with only single views of each object.

Depth Estimation Depth Prediction +1

MIST: Multiple Instance Spatial Transformer

1 code implementation CVPR 2021 Baptiste Angles, Yuhe Jin, Simon Kornblith, Andrea Tagliasacchi, Kwang Moo Yi

We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.

Image Reconstruction

Deep Medial Fields

no code implementations7 Jun 2021 Daniel Rebain, Ke Li, Vincent Sitzmann, Soroosh Yazdani, Kwang Moo Yi, Andrea Tagliasacchi

Implicit representations of geometry, such as occupancy fields or signed distance fields (SDF), have recently re-gained popularity in encoding 3D solid shape in a functional form.

COTR: Correspondence Transformer for Matching Across Images

1 code implementation ICCV 2021 Wei Jiang, Eduard Trulls, Jan Hosang, Andrea Tagliasacchi, Kwang Moo Yi

We propose a novel framework for finding correspondences in images based on a deep neural network that, given two images and a query point in one of them, finds its correspondence in the other.

Dense Pixel Correspondence Estimation Optical Flow Estimation

DeRF: Decomposed Radiance Fields

no code implementations CVPR 2021 Daniel Rebain, Wei Jiang, Soroosh Yazdani, Ke Li, Kwang Moo Yi, Andrea Tagliasacchi

Moreover, we show that a Voronoi spatial decomposition is preferable for this purpose, as it is provably compatible with the Painter's Algorithm for efficient and GPU-friendly rendering.

Eigendecomposition-Free Training of Deep Networks for Linear Least-Square Problems

no code implementations15 Apr 2020 Zheng Dang, Kwang Moo Yi, Yinlin Hu, Fei Wang, Pascal Fua, Mathieu Salzmann

In this paper, we introduce an eigendecomposition-free approach to training a deep network whose loss depends on the eigenvector corresponding to a zero eigenvalue of a matrix predicted by the network.

Denoising Pose Estimation

VaB-AL: Incorporating Class Imbalance and Difficulty with Variational Bayes for Active Learning

no code implementations CVPR 2021 Jongwon Choi, Kwang Moo Yi, Ji-Hoon Kim, Jinho Choo, Byoungjip Kim, Jin-Yeop Chang, Youngjune Gwon, Hyung Jin Chang

We show that our method can be applied to classification tasks on multiple different datasets -- including one that is a real-world dataset with heavy data imbalance -- significantly outperforming the state of the art.

Active Learning

Image Matching across Wide Baselines: From Paper to Practice

5 code implementations3 Mar 2020 Yuhe Jin, Dmytro Mishkin, Anastasiia Mishchuk, Jiri Matas, Pascal Fua, Kwang Moo Yi, Eduard Trulls

We introduce a comprehensive benchmark for local features and robust estimation algorithms, focusing on the downstream task -- the accuracy of the reconstructed camera pose -- as our primary metric.


VoronoiNet: General Functional Approximators with Local Support

no code implementations8 Dec 2019 Francis Williams, Daniele Panozzo, Kwang Moo Yi, Andrea Tagliasacchi

Voronoi diagrams are highly compact representations that are used in various Graphics applications.

Reducing the Human Effort in Developing PET-CT Registration

no code implementations25 Nov 2019 Teaghan O'Briain, Kyong Hwan Jin, Hongyoon Choi, Erika Chin, Magdalena Bazalova-Carter, Kwang Moo Yi

We aim to reduce the tedious nature of developing and evaluating methods for aligning PET-CT scans from multiple patient visits.

MIST: Multiple Instance Spatial Transformer Networks

no code implementations25 Sep 2019 Baptiste Angles, Simon Kornblith, Shahram Izadi, Andrea Tagliasacchi, Kwang Moo Yi

We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.

Image Reconstruction

Optimizing Through Learned Errors for Accurate Sports Field Registration

1 code implementation17 Sep 2019 Wei Jiang, Juan Camilo Gamboa Higuera, Baptiste Angles, Weiwei Sun, Mehrsan Javan, Kwang Moo Yi

We propose an optimization-based framework to register sports field templates onto broadcast videos.

Beyond Cartesian Representations for Local Descriptors

1 code implementation ICCV 2019 Patrick Ebel, Anastasiia Mishchuk, Kwang Moo Yi, Pascal Fua, Eduard Trulls

We demonstrate that this representation is particularly amenable to learning descriptors with deep networks.

Self-Supervised Deep Active Accelerated MRI

no code implementations14 Jan 2019 Kyong Hwan Jin, Michael Unser, Kwang Moo Yi

The reconstruction network is trained to give the highest reconstruction quality, given the MCTS sampling pattern.

MIST: Multiple Instance Spatial Transformer Network

1 code implementation26 Nov 2018 Baptiste Angles, Yuhe Jin, Simon Kornblith, Andrea Tagliasacchi, Kwang Moo Yi

We propose a deep network that can be trained to tackle image reconstruction and classification problems that involve detection of multiple object instances, without any supervision regarding their whereabouts.

Anomaly Detection In Surveillance Videos Image Reconstruction

LF-Net: Learning Local Features from Images

4 code implementations NeurIPS 2018 Yuki Ono, Eduard Trulls, Pascal Fua, Kwang Moo Yi

We present a novel deep architecture and a training strategy to learn a local feature pipeline from scratch, using collections of images without the need for human supervision.

Eigendecomposition-free Training of Deep Networks with Zero Eigenvalue-based Losses

no code implementations ECCV 2018 Zheng Dang, Kwang Moo Yi, Yinlin Hu, Fei Wang, Pascal Fua, Mathieu Salzmann

Many classical Computer Vision problems, such as essential matrix computation and pose estimation from 3D to 2D correspondences, can be solved by finding the eigenvector corresponding to the smallest, or zero, eigenvalue of a matrix representing a linear system.

3D Pose Estimation

Learning to Find Good Correspondences

3 code implementations CVPR 2018 Kwang Moo Yi, Eduard Trulls, Yuki Ono, Vincent Lepetit, Mathieu Salzmann, Pascal Fua

We develop a deep architecture to learn to find good correspondences for wide-baseline stereo.

LIFT: Learned Invariant Feature Transform

1 code implementation30 Mar 2016 Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, Pascal Fua

We introduce a novel Deep Network architecture that implements the full feature point handling pipeline, that is, detection, orientation estimation, and feature description.

Learning to Assign Orientations to Feature Points

no code implementations CVPR 2016 Kwang Moo Yi, Yannick Verdie, Pascal Fua, Vincent Lepetit

We show how to train a Convolutional Neural Network to assign a canonical orientation to feature points given an image patch centered on the feature point.

TILDE: A Temporally Invariant Learned DEtector

no code implementations CVPR 2015 Yannick Verdie, Kwang Moo Yi, Pascal Fua, Vincent Lepetit

We introduce a learning-based approach to detect repeatable keypoints under drastic imaging changes of weather and lighting conditions to which state-of-the-art keypoint detectors are surprisingly sensitive.

Cannot find the paper you are looking for? You can Submit a new open access paper.