1 code implementation • 28 Jan 2025 • Yoojin Jang, Junsu Kim, Hayeon Kim, Eun-ki Lee, Eun-Sol Kim, Seungryul Baek, Jaejun Yoo
Human-object interaction (HOI) is an essential problem in artificial intelligence (AI) which aims to understand the visual world that involves complex relationships between humans and objects.
no code implementations • 14 Jan 2025 • Hansoo Park, Chanwoo Kim, Jihyeon Kim, Hoseong Cho, Nhat Nguyen Bao Truong, Taehwan Kim, Seungryul Baek
RGB-based 3D pose estimation methods have been successful with the development of deep learning and the emergence of high-quality 3D pose datasets.
no code implementations • 18 Dec 2024 • Junuk Cha, Mengwei Ren, Krishna Kumar Singh, He Zhang, Yannick Hold-Geoffroy, Seunghyun Yoon, HyunJoon Jung, Jae Shin Yoon, Seungryul Baek
As a condition of the lighting images, we perform image-based relighting for both foreground and background using a single portrait image or a set of OLAT (One-Light-at-A-Time) images captured from lightstage system.
no code implementations • 28 Sep 2024 • Jeongwan On, Kyeonghwan Gwak, Gunyoung Kang, Hyein Hwang, Soohyun Hwang, Junuk Cha, Jaewook Han, Seungryul Baek
In this challenge, we address the task of bimanual category-agnostic hand-object interaction reconstruction, which aims to generate 3D reconstructions of both hands and the object from a monocular video, without relying on predefined templates.
1 code implementation • CVPR 2024 • Junuk Cha, Jihyeon Kim, Jae Shin Yoon, Seungryul Baek
For contact generation, a VAE-based network takes as input a text and an object mesh, and generates the probability of contacts between the surfaces of hands and the object during the interaction.
2 code implementations • 25 Mar 2024 • Zicong Fan, Takehiko Ohkawa, Linlin Yang, Nie Lin, Zhishan Zhou, Shihao Zhou, Jiajun Liang, Zhong Gao, Xuanyang Zhang, Xue Zhang, Fei Li, Zheng Liu, Feng Lu, Karim Abou Zeid, Bastian Leibe, Jeongwan On, Seungryul Baek, Aditya Prakash, Saurabh Gupta, Kun He, Yoichi Sato, Otmar Hilliges, Hyung Jin Chang, Angela Yao
A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation.
no code implementations • CVPR 2024 • Jongwook Choi, TaeHoon Kim, Yonghyun Jeong, Seungryul Baek, Jongwon Choi
This paper presents a new approach for the detection of fake videos, based on the analysis of style latent vectors and their abnormal behavior in temporal changes in the generated videos.
no code implementations • 8 Mar 2024 • Junsu Kim, Yunhoe Ku, Jihyeon Kim, Junuk Cha, Seungryul Baek
This technique uses Vision-Language Model (VLM) to verify the correctness of pseudo ground-truths (GTs) without requiring additional model training.
no code implementations • CVPR 2024 • Junsu Kim, Hoseong Cho, Jihyeon Kim, Yihalem Yimolal Tiruneh, Seungryul Baek
In the field of class incremental learning (CIL), generative replay has become increasingly prominent as a method to mitigate the catastrophic forgetting, alongside the continuous improvements in generative models.
no code implementations • 12 Jan 2024 • Junuk Cha, Hansol Lee, Jaewon Kim, Nhat Nguyen Bao Truong, Jae Shin Yoon, Seungryul Baek
This paper introduces a novel pipeline to reconstruct the geometry of interacting multi-person in clothing on a globally coherent scene space from a single image.
no code implementations • 28 Dec 2023 • Hansol Lee, Junuk Cha, Yunhoe Ku, Jae Shin Yoon, Seungryul Baek
For implicit modeling, an implicit network combines the appearance and 3D motion features to decode high-fidelity clothed 3D human avatars with motion-dependent geometry and texture.
no code implementations • 14 Dec 2023 • Junsu Kim, Sumin Hong, Chanwoo Kim, Jihyeon Kim, Yihalem Yimolal Tiruneh, Jeongwan On, Jihyun Song, Sunhwa Choi, Seungryul Baek
In this paper, we introduce an effective buffer training strategy (eBTS) that creates the optimized replay buffer on object detection.
1 code implementation • 25 Sep 2023 • Uyoung Jeong, Seungryul Baek, Hyung Jin Chang, Kwang In Kim
Our new instance embedding loss provides a learning signal on the entire area of the image with bounding box annotations, achieving globally consistent and disentangled instance representation.
no code implementations • 8 Jun 2023 • Hansol Lee, Yunhoe Ku, Eunseo Kim, Seungryul Baek
We proposed IFaceUV, a fully differentiable pipeline that properly combines 2D and 3D information to conduct the facial reenactment task.
1 code implementation • CVPR 2023 • Hoseong Cho, Chanwoo Kim, Jihyeon Kim, Seongyeong Lee, Elkhan Ismayilzada, Seungryul Baek
In our framework, we insert the whole image depicting two hands, an object and their interactions as input and jointly estimate 3 information from each frame: poses of two hands, pose of an object and object types.
Ranked #4 on
Action Recognition
on H2O (2 Hands and Objects)
no code implementations • 11 Nov 2022 • Changhwa Lee, Junuk Cha, Hansol Lee, Seongyeong Lee, Donguk Kim, Seungryul Baek
At the same time, to obtain high-quality 2D images from 3D space, well-designed 3D-to-2D projection and image refinement are required.
no code implementations • 30 Oct 2022 • Seongyeong Lee, Hansoo Park, Dong Uk Kim, Jihyeon Kim, Muhammadjon Boboev, Seungryul Baek
The manipulated image features are then exploited to train the hand pose estimation network via the contrastive learning framework.
1 code implementation • 24 Oct 2022 • Junuk Cha, Muhammad Saqlain, GeonU Kim, Mingyu Shin, Seungryul Baek
To tackle the challenges, we propose a coarse-to-fine pipeline that benefits from 1) inverse kinematics from the occlusion-robust 3D skeleton estimation and 2) Transformer-based relation-aware refinement techniques.
Ranked #1 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
(using extra training data)
no code implementations • 20 Oct 2022 • Hoseong Cho, Donguk Kim, Chanwoo Kim, Seongyeong Lee, Seungryul Baek
In this challenge, we aim to estimate global 3D hand poses from the input image where two hands and an object are interacting on the egocentric viewpoint.
no code implementations • 20 Oct 2022 • Hoseong Cho, Seungryul Baek
This report describes the 2nd place solution to the ECCV 2022 Human Body, Hands, and Activities (HBHA) from Egocentric and Multi-view Cameras Challenge: Action Recognition.
no code implementations • ICCV 2021 • Dong Uk Kim, Kwang In Kim, Seungryul Baek
Three dimensional hand pose estimation has reached a level of maturity, enabling real-world applications for single-hand cases.
no code implementations • CVPR 2020 • Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Via the domain adaption in image space, not only 3D HPE accuracy is improved, but also HOI input images are translated to segmented and de-occluded hand-only images.
no code implementations • ECCV 2020 • Anil Armagan, Guillermo Garcia-Hernando, Seungryul Baek, Shreyas Hampali, Mahdi Rad, Zhaohui Zhang, Shipeng Xie, Mingxiu Chen, Boshen Zhang, Fu Xiong, Yang Xiao, Zhiguo Cao, Junsong Yuan, Pengfei Ren, Weiting Huang, Haifeng Sun, Marek Hrúz, Jakub Kanis, Zdeněk Krňoul, Qingfu Wan, Shile Li, Linlin Yang, Dongheui Lee, Angela Yao, Weiguo Zhou, Sijia Mei, Yun-hui Liu, Adrian Spurr, Umar Iqbal, Pavlo Molchanov, Philippe Weinzaepfel, Romain Brégier, Grégory Rogez, Vincent Lepetit, Tae-Kyun Kim
To address these issues, we designed a public challenge (HANDS'19) to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set.
no code implementations • 10 Sep 2019 • Binod Bhattarai, Seungryul Baek, Rumeysa Bodur, Tae-Kyun Kim
Unlike previous studies of randomly augmenting the synthetic data with real data, we present our simple, effective and easy to implement synthetic data sampling methods to train deep CNNs more efficiently and accurately.
no code implementations • CVPR 2019 • Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Once the model is successfully fitted to input RGB images, its meshes i. e. shapes and articulations, are realistic, and we augment view-points on top of estimated dense hand poses.
no code implementations • CVPR 2018 • Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
By training the HPG and HPE in a single unified optimization framework enforcing that 1) the HPE agrees with the paired depth and skeleton entries; and 2) the HPG-HPE combination satisfies the cyclic consistency (both the input and the output of HPG-HPE are skeletons) observed via the newly generated unpaired skeletons, our algorithm constructs a HPE which is robust to variations that go beyond the coverage of the existing database.
no code implementations • 6 Jun 2017 • Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Each response map-or node-in both the convolutional and fully-connected layers selectively respond to class labels s. t.
Ranked #176 on
Image Classification
on CIFAR-100
(using extra training data)
1 code implementation • CVPR 2018 • Guillermo Garcia-Hernando, Shanxin Yuan, Seungryul Baek, Tae-Kyun Kim
Our dataset and experiments can be of interest to communities of 3D hand pose estimation, 6D object pose, and robotics as well as action recognition.
no code implementations • 28 Oct 2016 • Seungryul Baek, Kwang In Kim, Tae-Kyun Kim
Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed.
no code implementations • 23 Jul 2016 • Seungryul Baek, Zhiyuan Shi, Masato Kawade, Tae-Kyun Kim
In this paper, we tackle the problem of 24 hours-monitoring patient actions in a ward such as "stretching an arm out of the bed", "falling out of the bed", where temporal movements are subtle or significant.