SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding

no code implementations • 17 Jan 2024 • Baoxiong Jia, Yixin Chen, Huangyue Yu, Yan Wang, Xuesong Niu, Tengyu Liu, Qing Li, Siyuan Huang

In comparison to recent advancements in the 2D domain, grounding language in 3D scenes faces several significant challenges: (i) the inherent complexity of 3D scenes due to the diverse object configurations, their rich attributes, and intricate relationships; (ii) the scarcity of paired 3D vision-language data to support grounded learning; and (iii) the absence of a unified learning framework to distill knowledge from grounded 3D data.

Scene Understanding Visual Grounding

What I See Is What You See: Joint Attention Learning for First and Third Person Video Co-analysis

no code implementations • 16 Apr 2019 • Huangyue Yu, Minjie Cai, Yunfei Liu, Feng Lu

However, techniques for analyzing the first-person video can be fundamentally different from those for the third-person video, and it is even more difficult to explore the shared information from both viewpoints.

Self-Supervised Learning