ActAnywhere: Subject-Aware Video Background Generation

no code implementations • 19 Jan 2024 • Boxiao Pan, Zhan Xu, Chun-Hao Paul Huang, Krishna Kumar Singh, Yang Zhou, Leonidas J. Guibas, Jimei Yang

Generating video background that tailors to foreground subject motion is an important problem for the movie industry and visual effects community.

JacobiNeRF: NeRF Shaping with Mutual Information Gradients

1 code implementation • CVPR 2023 • Xiaomeng Xu, Yanchao Yang, Kaichun Mo, Boxiao Pan, Li Yi, Leonidas Guibas

We propose a method that trains a neural radiance field (NeRF) to encode not only the appearance of the scene but also semantic correlations between scene points, regions, or entities -- aiming to capture their mutual co-variation patterns.

Instance Segmentation Semantic Segmentation

PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision

no code implementations • 16 Mar 2023 • Konstantinos Tertikas, Despoina Paschalidou, Boxiao Pan, Jeong Joon Park, Mikaela Angelina Uy, Ioannis Emiris, Yannis Avrithis, Leonidas Guibas

Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.

Generating Part-Aware Editable 3D Shapes Without 3D Supervision

1 code implementation • CVPR 2023 • Konstantinos Tertikas, Despoina Paschalidou, Boxiao Pan, Jeong Joon Park, Mikaela Angelina Uy, Ioannis Emiris, Yannis Avrithis, Leonidas Guibas

Evaluations on various ShapeNet categories demonstrate the ability of our model to generate editable 3D objects of improved fidelity, compared to previous part-based generative approaches that require 3D supervision or models relying on NeRFs.

COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos

no code implementations • ICCV 2023 • Boxiao Pan, Bokui Shen, Davis Rempe, Despoina Paschalidou, Kaichun Mo, Yanchao Yang, Leonidas J. Guibas

In this work, we introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras.

Collision Avoidance Synthetic Data Generation

Efficient Geometry-aware 3D Generative Adversarial Networks

2 code implementations • CVPR 2022 • Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein

Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge.

Computational Efficiency Neural Rendering

Spatio-Temporal Graph for Video Captioning with Knowledge Distillation

no code implementations • CVPR 2020 • Boxiao Pan, Haoye Cai, De-An Huang, Kuan-Hui Lee, Adrien Gaidon, Ehsan Adeli, Juan Carlos Niebles

In this paper, we propose a novel spatio-temporal graph model for video captioning that exploits object interactions in space and time.

Knowledge Distillation Object +2

Adversarial Cross-Domain Action Recognition with Co-Attention

no code implementations • 22 Dec 2019 • Boxiao Pan, Zhangjie Cao, Ehsan Adeli, Juan Carlos Niebles

Action recognition has been a widely studied topic with a heavy focus on supervised learning involving sufficient labeled videos.

Action Recognition