Search Results for author:
Found 18 papers, 10 papers with code
Graph-Based Multimodal and Multi-view Alignment for Keystep Recognition
Egocentric videos capture scenes from a wearer's viewpoint, resulting in dynamic backgrounds, frequent motion, and occlusions, posing challenges to accurate keystep recognition.
Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation
To address the data scarcity associated with 3D assets, 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines.
Ego-VPA: Egocentric Video Understanding with Parameter-efficient Adaptation
Video understanding typically requires fine-tuning the large backbone when adapting to new domains.
SViTT-Ego: A Sparse Video-Text Transformer for Egocentric Video
Pretraining egocentric vision-language models has become essential to improving downstream egocentric video-text tasks.
Contrastive Language Video Time Pre-training
We introduce LAVITI, a novel approach to learning language, video, and temporal representations in long-form videos via contrastive learning.
R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model
In the evolving landscape of text-to-image (T2I) diffusion models, the remarkable capability to generate high-quality images from textual descriptions faces challenges with the potential misuse of reproducing sensitive content.
Action Scene Graphs for Long-Form Understanding of Egocentric Videos
We present Egocentric Action Scene Graphs (EASGs), a new representation for long-form understanding of egocentric videos.
STHG: Spatial-Temporal Heterogeneous Graph Learning for Advanced Audio-Visual Diarization
This report introduces our novel method named STHG for the Audio-Visual Diarization task of the Ego4D Challenge 2023.
WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models
This paper introduces a novel approach to model fingerprinting that assigns responsibility for the generated images, thereby serving as a potential countermeasure to model misuse.
SViTT: Temporal Learning of Sparse Video-Text Transformers
Do video-text transformers learn to model temporal relationships across frames?
Ranked #4 on
Video Question Answering
on AGQA 2.0 balanced
(Average Accuracy metric)
Unbiased Scene Graph Generation in Videos
The task of dynamic scene graph generation (SGG) from videos is complicated and challenging due to the inherent dynamics of a scene, temporal fluctuation of model predictions, and the long-tailed distribution of the visual relationships in addition to the already existing challenges in image-based SGG.
Intel Labs at Ego4D Challenge 2022: A Better Baseline for Audio-Visual Diarization
This report describes our approach for the Audio-Visual Diarization (AVD) task of the Ego4D Challenge 2022.
Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection
Active speaker detection (ASD) in videos with multiple speakers is a challenging task as it requires learning effective audiovisual features and spatial-temporal correlations over long temporal windows.
Ranked #1 on
Node Classification
on AVA
Active Speaker Detection
Audio-Visual Active Speaker Detection
+2
Learning Spatial-Temporal Graphs for Active Speaker Detection
We address the problem of active speaker detection through a new framework, called SPELL, that learns long-range multimodal graphs to encode the inter-modal relationship between audio and visual data.
Active Speaker Detection
Audio-Visual Active Speaker Detection
+1
Integrating Human Gaze into Attention for Egocentric Activity Recognition
In addition, we model the distribution of gaze fixations using a variational method.
Ranked #2 on
Egocentric Activity Recognition
on EGTEA
Adversarial Background-Aware Loss for Weakly-supervised Temporal Activity Localization
Two triplets of the feature space are considered in our approach: one triplet is used to learn discriminative features for each activity class, and the other one is used to distinguish the features where no activity occurs (i. e. background features) from activity-related features for each video.
TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detection
It consists of two building blocks: first, the encoder network extracts low-resolution spatiotemporal features from an input clip of several consecutive frames, and then the following prediction network decodes the encoded features spatially while aggregating all the temporal information.
Hierarchical Novelty Detection for Visual Object Recognition
The essential ingredients of our methods are confidence-calibrated classifiers, data relabeling, and the leave-one-out strategy for modeling novel classes under the hierarchical taxonomy.
