Search Results for author: Hyun Soo Park

Found 39 papers, 8 papers with code

Ego4D: Around the World in 3,000 Hours of Egocentric Video

5 code implementations CVPR 2022 Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei HUANG, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Ziwei Zhao, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Christian Fuegen, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, Jitendra Malik

We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.

De-identification Ethics

Self-supervised 3D Representation Learning of Dressed Humans from Social Media Videos

1 code implementation CVPR 2021 Yasamin Jafarian, Hyun Soo Park

A key challenge of learning a visual representation for the 3D high fidelity geometry of dressed humans lies in the limited availability of the ground truth data (e. g., 3D scanned models), which results in the performance degradation of 3D human reconstruction when applying to real-world imagery.

3D Human Reconstruction Depth Estimation +2

HUMBI: A Large Multiview Dataset of Human Body Expressions

1 code implementation CVPR 2020 Zhixuan Yu, Jae Shin Yoon, In Kyu Lee, Prashanth Venkatesh, Jaesik Park, Jihun Yu, Hyun Soo Park

This paper presents a new large multiview dataset called HUMBI for human body expressions with natural clothing.

Surface Normal Estimation of Tilted Images via Spatial Rectifier

1 code implementation ECCV 2020 Tien Do, Khiem Vuong, Stergios I. Roumeliotis, Hyun Soo Park

Our two main hypotheses are: (1) visual scene layout is indicative of the gravity direction; and (2) not all surfaces are equally represented by a learned estimator due to the structured distribution of the training data, thus, there exists a transformation for each tilted image that is more responsive to the learned estimator than others.

Data Augmentation Surface Normal Estimation

MONET: Multiview Semi-supervised Keypoint Detection via Epipolar Divergence

1 code implementation ICCV 2019 Yuan Yao, Yasamin Jafarian, Hyun Soo Park

While multiview geometry can be used to self-supervise the unlabeled data, integrating the geometry into learning a keypoint detector is challenging due to representation mismatch.

Data Augmentation Keypoint Detection

Egocentric Scene Understanding via Multimodal Spatial Rectifier

1 code implementation CVPR 2022 Tien Do, Khiem Vuong, Hyun Soo Park

We present a multimodal spatial rectifier that stabilizes the egocentric images to a set of reference directions, which allows learning a coherent visual representation.

Scene Understanding Surface Normal Estimation

Multiview Supervision By Registration

1 code implementation27 Nov 2018 Yilun Zhang, Hyun Soo Park

This paper presents a semi-supervised learning framework to train a keypoint detector using multiview image streams given the limited labeled data (typically $<$4\%).

3D Reconstruction Keypoint Detection +2

Unsupervised Learning of Important Objects from First-Person Videos

1 code implementation ICCV 2017 Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi

In this work, we show that we can detect important objects in first-person images without the supervision by the camera wearer or even third-person labelers.

Object Segmentation +1

3D Semantic Trajectory Reconstruction from 3D Pixel Continuum

no code implementations CVPR 2018 Jae Shin Yoon, Ziwei Li, Hyun Soo Park

This paper presents a method to reconstruct dense semantic trajectory stream of human interactions in 3D from synchronized multiple videos.

Am I a Baller? Basketball Performance Assessment from First-Person Videos

no code implementations ICCV 2017 Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi

Finally, we use this feature to learn a basketball assessment model from pairs of labeled first-person basketball videos, for which a basketball expert indicates, which of the two players is better.

First Person Action-Object Detection with EgoNet

no code implementations15 Mar 2016 Gedas Bertasius, Hyun Soo Park, Stella X. Yu, Jianbo Shi

Unlike traditional third-person cameras mounted on robots, a first-person camera, captures a person's visual sensorimotor object interactions from up close.

Human-Object Interaction Detection Object +2

Customizing First Person Image Through Desired Actions

no code implementations1 Apr 2017 Shan Su, Jianbo Shi, Hyun Soo Park

Our conjecture is that the spatial arrangement of a first person visual scene is deployed to afford an action, and therefore, the action can be inversely used to synthesize a new scene such that the action is feasible.

Generative Adversarial Network

Social Behavior Prediction from First Person Videos

no code implementations29 Nov 2016 Shan Su, Jung Pyo Hong, Jianbo Shi, Hyun Soo Park

This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos.

3D Reconstruction

Exploiting Egocentric Object Prior for 3D Saliency Detection

no code implementations9 Nov 2015 Gedas Bertasius, Hyun Soo Park, Jianbo Shi

We empirically show that this representation can accurately characterize the egocentric object prior by testing it on an egocentric RGBD dataset for three tasks: the 3D saliency detection, future saliency prediction, and interaction classification.

Object Saliency Prediction

Future Localization from an Egocentric Depth Image

no code implementations7 Sep 2015 Hyun Soo Park, Yedong Niu, Jianbo Shi

As a byproduct of the predicted trajectories of ego-motion, we discover in the image the empty space occluded by foreground objects.

object-detection Object Detection

ECO: Egocentric Cognitive Mapping

no code implementations2 Dec 2018 Jayant Sharma, Zixing Wang, Alberto Speranzon, Vijay Venkataraman, Hyun Soo Park

We present a new method to localize a camera within a previously unseen environment perceived from an egocentric point of view.

Domain Adaptation Navigate

Multiview Cross-supervision for Semantic Segmentation

no code implementations4 Dec 2018 Yuan Yao, Hyun Soo Park

We hypothesize that it is possible to leverage multiview image streams that are linked through the underlying 3D geometry, which can provide an additional supervisionary signal to train a segmentation model.

3D Reconstruction Camera Calibration +2

MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction

no code implementations CVPR 2014 Hanbyul Joo, Hyun Soo Park, Yaser Sheikh

Many traditional challenges in reconstructing 3D motion, such as matching across wide baselines and handling occlusion, reduce in significance as the number of unique viewpoints increases.

3D Reconstruction

Social Saliency Prediction

no code implementations CVPR 2015 Hyun Soo Park, Jianbo Shi

An ensemble classifier is trained to learn the geometric relationship.

Saliency Prediction

Force From Motion: Decoding Physical Sensation in a First Person Video

no code implementations CVPR 2016 Hyun Soo Park, Jyh-Jing Hwang, Jianbo Shi

In this paper, we focus on a problem of Force from Motion---decoding the sensation of 1) passive forces such as the gravity, 2) the physical scale of the motion (speed) and space, and 3) active forces exerted by the observer such as pedaling a bike or banking on a ski turn.

Action Recognition Friction +2

Egocentric Future Localization

no code implementations CVPR 2016 Hyun Soo Park, Jyh-Jing Hwang, Yedong Niu, Jianbo Shi

We refine them by minimizing a cost function that describes compatibility between the obstacles in the EgoRetinal map and trajectories.

Motion Planning

Predicting Behaviors of Basketball Players From First Person Videos

no code implementations CVPR 2017 Shan Su, Jung Pyo Hong, Jianbo Shi, Hyun Soo Park

This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos.

3D Reconstruction

Self-Supervised Adaptation of High-Fidelity Face Models for Monocular Performance Tracking

no code implementations CVPR 2019 Jae Shin Yoon, Takaaki Shiratori, Shoou-I Yu, Hyun Soo Park

In this paper, we propose a self-supervised domain adaptation approach to enable the animation of high-fidelity face models from a commodity camera.

Domain Adaptation Face Model

Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera

no code implementations CVPR 2020 Jae Shin Yoon, Kihwan Kim, Orazio Gallo, Hyun Soo Park, Jan Kautz

Our insight is that although its scale and quality are inconsistent with other views, the depth estimation from a single view can be used to reason about the globally coherent geometry of dynamic contents.

Depth Estimation Novel View Synthesis

Pose-Guided Human Animation from a Single Image in the Wild

no code implementations CVPR 2021 Jae Shin Yoon, Lingjie Liu, Vladislav Golyanik, Kripasindhu Sarkar, Hyun Soo Park, Christian Theobalt

We present a new pose transfer method for synthesizing a human animation from a single image of a person controlled by a sequence of body poses.

Pose Transfer

Neural 3D Clothes Retargeting from a Single Image

no code implementations29 Jan 2021 Jae Shin Yoon, Kihwan Kim, Jan Kautz, Hyun Soo Park

In this paper, we present a method of clothes retargeting; generating the potential poses and deformations of a given 3D clothing template model to fit onto a person in a single RGB image.

Inverse Simulation: Reconstructing Dynamic Geometry of Clothed Humans via Optimal Control

no code implementations CVPR 2021 Jingfan Guo, Jie Li, Rahul Narain, Hyun Soo Park

Inspired by the theory of optimal control, we optimize the body states such that the simulated cloth motion is matched to the point cloud measurements, and the analytic gradient of the simulator is back-propagated to update the body states.

Friction

Automated Tracking of Primate Behavior

no code implementations30 Aug 2021 Benjamin Hayden, Hyun Soo Park, Jan Zimmermann

The availability of such data has in turn spurred developments in data analysis techniques.

Pose Tracking

Semi-supervised Dense Keypoints Using Unlabeled Multiview Images

no code implementations20 Sep 2021 Zhixuan Yu, Haozheng Yu, Long Sha, Sujoy Ganguly, Hyun Soo Park

(2) Geometric consistency: every point in the continuous correspondence fields must satisfy the multiview consistency collectively.

3D Reconstruction Keypoint Detection

Self-supervised Secondary Landmark Detection via 3D Representation Learning

no code implementations1 Oct 2021 Praneet C. Bala, Jan Zimmermann, Hyun Soo Park, Benjamin Y. Hayden

We hypothesize that there exists a shared representation between the primary and secondary landmarks because the range of motion of the secondary landmarks can be approximately spanned by that of the primary landmarks.

Representation Learning

HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge

no code implementations30 Sep 2021 Jae Shin Yoon, Zhixuan Yu, Jaesik Park, Hyun Soo Park

We demonstrate that HUMBI is highly effective in learning and reconstructing a complete human model and is complementary to the existing datasets of human body expressions with limited views and subjects such as MPII-Gaze, Multi-PIE, Human3. 6M, and Panoptic Studio datasets.

PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound

no code implementations CVPR 2022 Zhijian Yang, Xiaoran Fan, Volkan Isler, Hyun Soo Park

Based on this insight, we introduce a time-invariant transfer function called pose kernel -- the impulse response of audio signals induced by the body pose.

regression

Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera

no code implementations CVPR 2022 Jae Shin Yoon, Duygu Ceylan, Tuanfeng Y. Wang, Jingwan Lu, Jimei Yang, Zhixin Shu, Hyun Soo Park

Appearance of dressed humans undergoes a complex geometric transformation induced not only by the static pose but also by its dynamics, i. e., there exists a number of cloth geometric configurations given a pose depending on the way it has moved.

Learning To Detect Scene Landmarks for Camera Localization

no code implementations CVPR 2022 Tien Do, Ondrej Miksik, Joseph DeGol, Hyun Soo Park, Sudipta N. Sinha

Our key idea is to implicitly encode the appearance of a sparse yet salient set of 3D scene points into a convolutional neural network (CNN) that can detect these scene points in query images whenever they are visible.

Camera Localization Image Retrieval +2

Self-supervised Wide Baseline Visual Servoing via 3D Equivariance

no code implementations12 Sep 2022 Jinwook Huh, Jungseok Hong, Suveer Garg, Hyun Soo Park, Volkan Isler

Existing approaches that regress absolute camera pose with respect to an object require 3D ground truth data of the object in the forms of 3D bounding boxes or meshes.

Object

Normal-guided Garment UV Prediction for Human Re-texturing

no code implementations CVPR 2023 Yasamin Jafarian, Tuanfeng Y. Wang, Duygu Ceylan, Jimei Yang, Nathan Carr, Yi Zhou, Hyun Soo Park

To edit human videos in a physically plausible way, a texture map must take into account not only the garment transformation induced by the body movements and clothes fitting, but also its 3D fine-grained surface geometry.

3D Reconstruction

Diffusion Shape Prior for Wrinkle-Accurate Cloth Registration

no code implementations10 Nov 2023 Jingfan Guo, Fabian Prada, Donglai Xiang, Javier Romero, Chenglei Wu, Hyun Soo Park, Takaaki Shiratori, Shunsuke Saito

Registering clothes from 4D scans with vertex-accurate correspondence is challenging, yet important for dynamic appearance modeling and physics parameter estimation from real-world data.

Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives

no code implementations30 Nov 2023 Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, Eugene Byrne, Zach Chavis, Joya Chen, Feng Cheng, Fu-Jen Chu, Sean Crane, Avijit Dasgupta, Jing Dong, Maria Escobar, Cristhian Forigua, Abrham Gebreselasie, Sanjay Haresh, Jing Huang, Md Mohaiminul Islam, Suyog Jain, Rawal Khirodkar, Devansh Kukreja, Kevin J Liang, Jia-Wei Liu, Sagnik Majumder, Yongsen Mao, Miguel Martin, Effrosyni Mavroudi, Tushar Nagarajan, Francesco Ragusa, Santhosh Kumar Ramakrishnan, Luigi Seminara, Arjun Somayazulu, Yale Song, Shan Su, Zihui Xue, Edward Zhang, Jinxu Zhang, Angela Castillo, Changan Chen, Xinzhu Fu, Ryosuke Furuta, Cristina Gonzalez, Prince Gupta, Jiabo Hu, Yifei HUANG, Yiming Huang, Weslie Khoo, Anush Kumar, Robert Kuo, Sach Lakhavani, Miao Liu, Mi Luo, Zhengyi Luo, Brighid Meredith, Austin Miller, Oluwatumininu Oguntola, Xiaqing Pan, Penny Peng, Shraman Pramanick, Merey Ramazanova, Fiona Ryan, Wei Shan, Kiran Somasundaram, Chenan Song, Audrey Southerland, Masatoshi Tateno, Huiyu Wang, Yuchen Wang, Takuma Yagi, Mingfei Yan, Xitong Yang, Zecheng Yu, Shengxin Cindy Zha, Chen Zhao, Ziwei Zhao, Zhifan Zhu, Jeff Zhuo, Pablo Arbelaez, Gedas Bertasius, David Crandall, Dima Damen, Jakob Engel, Giovanni Maria Farinella, Antonino Furnari, Bernard Ghanem, Judy Hoffman, C. V. Jawahar, Richard Newcombe, Hyun Soo Park, James M. Rehg, Yoichi Sato, Manolis Savva, Jianbo Shi, Mike Zheng Shou, Michael Wray

We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.

Video Understanding

One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation

no code implementations19 Feb 2024 Zhixuan Yu, Ziqian Bai, Abhimitra Meka, Feitong Tan, Qiangeng Xu, Rohit Pandey, Sean Fanello, Hyun Soo Park, yinda zhang

Traditional methods for constructing high-quality, personalized head avatars from monocular videos demand extensive face captures and training time, posing a significant challenge for scalability.

Camera Calibration

Cannot find the paper you are looking for? You can Submit a new open access paper.