Search Results for author:
Found 14 papers, 4 papers with code
Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos
We investigate exocentric-to-egocentric cross-view translation, which aims to generate a first-person (egocentric) view of an actor based on a video recording that captures the actor from a third-person (exocentric) perspective.
Detours for Navigating Instructional Videos
We introduce the video detours problem for navigating instructional videos.
Learning Object State Changes in Videos: An Open-World Perspective
Object State Changes (OSCs) are pivotal for video understanding.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
Egocentric Video Task Translation @ Ego4D Challenge 2022
With no modification to the baseline architectures, our proposed approach achieves competitive performance on two Ego4D challenges, ranking the 1st in the talking to me challenge and the 3rd in the PNR keyframe localization challenge.
Egocentric Video Task Translation
Different video understanding tasks are typically treated in isolation, and even with distinct types of curated data (e. g., classifying sports in one dataset, tracking animals in another).
The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation
Crossmodal knowledge distillation (KD) extends traditional knowledge distillation to the area of multimodal learning and demonstrates great success in various applications.
Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularization
Multimodal fusion emerges as an appealing technique to improve model performances on many tasks.
Dynamic Multimodal Fusion
In this work, we propose dynamic multimodal fusion (DynMM), a new approach that adaptively fuses multimodal data and generates data-dependent forward paths during inference.
Ranked #43 on
Semantic Segmentation
on NYU Depth v2
SUGAR: Efficient Subgraph-level Training via Resource-aware Graph Partitioning
Graph Neural Networks (GNNs) have demonstrated a great potential in a variety of graph-based applications, such as recommender systems, drug discovery, and object recognition.
Co-advise: Cross Inductive Bias Distillation
Transformers recently are adapted from the community of natural language processing as a promising substitute of convolution-based neural networks for visual learning tasks.
What Makes Multi-modal Learning Better than Single (Provably)
The world provides us with data of multiple modalities.
On Feature Decorrelation in Self-Supervised Learning
In self-supervised representation learning, a common idea behind most of the state-of-the-art approaches is to enforce the robustness of the representations to predefined augmentations.
Multimodal Knowledge Expansion
The popularity of multimodal sensors and the accessibility of the Internet have brought us a massive amount of unlabeled multimodal data.
Ranked #63 on
Semantic Segmentation
on NYU Depth v2
