Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture

1 code implementation • 1 Sep 2023 • Shaohua Pan, Qi Ma, Xinyu Yi, Weifeng Hu, Xiong Wang, Xingkang Zhou, Jijunnan Li, Feng Xu

We believe that the combination is complementary and able to solve the inherent difficulties of using one modality input, including occlusions, extreme lighting/texture, and out-of-view for visual mocap and global drifts for inertial mocap.

3D Human Pose Estimation

EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors

no code implementations • 2 May 2023 • Xinyu Yi, Yuxiao Zhou, Marc Habermann, Vladislav Golyanik, Shaohua Pan, Christian Theobalt, Feng Xu

We integrate the two techniques together in EgoLocate, a system that simultaneously performs human motion capture (mocap), localization, and mapping in real time from sparse body-mounted sensors, including 6 inertial measurement units (IMUs) and a monocular phone camera.

Simultaneous Localization and Mapping

Physical Interaction: Reconstructing Hand-object Interactions with Physics

1 code implementation • 22 Sep 2022 • Haoyu Hu, Xinyu Yi, Hao Zhang, Jun-Hai Yong, Feng Xu

Single view-based reconstruction of hand-object interaction is challenging due to the severe observation missing caused by occlusions.

TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensors

1 code implementation • 10 May 2021 • Xinyu Yi, Yuxiao Zhou, Feng Xu

For global translation estimation, we propose a supporting-foot-based method and an RNN-based method to robustly solve for the global translations with a confidence-based fusion technique.

Pose Estimation Translation