Search Results for author:
Found 40 papers, 21 papers with code
Eyes Wide Unshut: Unsupervised Mistake Detection in Egocentric Procedural Video by Detecting Unpredictable Gaze
In this paper, we address the challenge of unsupervised mistake detection in egocentric procedural video through the analysis of gaze signals.
AFF-ttention! Affordances and Attention models for Short-Term Object Interaction Anticipation
Short-Term object-interaction Anticipation consists of detecting the location of the next-active objects, the noun and verb categories of the interaction, and the time to contact from the observation of egocentric video.
Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos
Task graphs learned with our approach are also shown to significantly enhance online mistake detection in procedural egocentric videos, achieving notable gains of +19. 8% and +7. 5% on the Assembly101 and EPIC-Tent datasets.
PREGO: online mistake detection in PRocedural EGOcentric videos
We propose PREGO, the first online one-class classification model for mistake detection in PRocedural EGOcentric videos.
Action Scene Graphs for Long-Form Understanding of Egocentric Videos
We present Egocentric Action Scene Graphs (EASGs), a new representation for long-form understanding of egocentric videos.
Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairs
Instead, we propose a novel methodology which performs the adaptation leveraging existing labeled exocentric videos and a new set of unlabeled, synchronized exocentric-egocentric video pairs, for which temporal action segmentation annotations do not need to be collected.
Are Synthetic Data Useful for Egocentric Hand-Object Interaction Detection?
In this study, we investigate the effectiveness of synthetic data in enhancing egocentric hand-object interaction detection.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarios
ENIGMA-51 is a new egocentric dataset acquired in an industrial scenario by 19 subjects who followed instructions to complete the repair of electrical boards using industrial tools (e. g., electric screwdriver) and equipments (e. g., oscilloscope).
Streaming egocentric action anticipation: An evaluation scheme and approach
We propose a streaming egocentric action evaluation scheme which assumes that predictions are performed online and made available only after the model has processed the current input segment, which depends on its runtime.
Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenario
In this paper, we tackle the problem of Egocentric Human-Object Interaction (EHOI) detection in an industrial setting.
StillFast: An End-to-End Approach for Short-Term Object Interaction Anticipation
Anticipation problem has been studied considering different aspects such as predicting humans' locations, predicting hands and objects trajectories, and forecasting actions and human-object interactions.
A Multi Camera Unsupervised Domain Adaptation Pipeline for Object Detection in Cultural Sites through Adversarial Learning and Self-Training
Object detection algorithms allow to enable many interesting applications which can be implemented in different devices, such as smartphones and wearable devices.
Visual Object Tracking in First Person Vision
Despite a few previous attempts to exploit trackers in the FPV domain, a methodical analysis of the performance of state-of-the-art trackers is still missing.
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain
To encourage research in this field, we present MECCANO, a multimodal dataset of egocentric videos to study humans behavior understanding in industrial-like settings.
Panoptic Segmentation using Synthetic and Real Data
Motivated by this observation, we propose a pipeline which allows to generate synthetic images from 3D models of real environments and real objects.
Weakly Supervised Attended Object Detection Using Gaze Data as Annotations
Since labeling large amounts of data to train a standard object detector is expensive in terms of costs and time, we propose a weakly supervised version of the task which leans only on gaze data and a frame-level label indicating the class of the attended object.
Egocentric Human-Object Interaction Detection Exploiting Synthetic Data
We consider the problem of detecting Egocentric HumanObject Interactions (EHOIs) in industrial contexts.
Untrimmed Action Anticipation
Experiments show that the performance of current models designed for trimmed action anticipation is very limited and more research on this task is required.
Image-based Navigation in Real-World Environments via Multiple Mid-level Representations: Fusion Models, Benchmark and Efficient Evaluation
All the proposed navigation models have been trained with the Habitat simulator on a synthetic office environment and have been tested on the same real-world environment using a real robotic platform.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Towards Streaming Egocentric Action Anticipation
In contrast, in this paper, we propose a "streaming" egocentric action anticipation evaluation protocol which explicitly considers model runtime for performance assessment, assuming that predictions will be available only after the current video segment is processed, which depends on the processing time of a method.
Is First Person Vision Challenging for Object Tracking?
Our study extensively analyses the performance of recent visual trackers and baseline FPV trackers with respect to different aspects and considering a new performance measure.
Predicting the Future from First Person (Egocentric) Vision: A Survey
Egocentric videos can bring a lot of information about how humans perceive the world and interact with the environment, which can be beneficial for the analysis of human behaviour.
A Survey on Human-aware Robot Navigation
This paper is concerned with the navigation aspect of a socially-compliant robot and provides a survey of existing solutions for the relevant areas of research as well as an outlook on possible future directions.
Is First Person Vision Challenging for Object Tracking?
Despite a few previous attempts to exploit trackers in FPV applications, a methodical analysis of the performance of state-of-the-art visual trackers in this domain is still missing.
On Embodied Visual Navigation in Real Environments Through Habitat
Visual navigation models based on deep learning can learn effective policies when trained on large amounts of visual observations through reinforcement learning.
The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain
To fill this gap, we introduce MECCANO, the first dataset of egocentric videos to study human-object interactions in industrial-like settings.
Ranked #1 on
Action Recognition
on MECCANO
An Unsupervised Domain Adaptation Scheme for Single-Stage Artwork Recognition in Cultural Sites
To address this problem, we created a new dataset containing both synthetic and real images of 16 different artworks.
Ranked #1 on
Unsupervised Domain Adaptation
on UDA-CH
Rescaling Egocentric Vision
This paper introduces the pipeline to extend the largest dataset in egocentric vision, EPIC-KITCHENS.
Ranked #8 on
Action Anticipation
on EPIC-KITCHENS-100
SceneAdapt: Scene-based domain adaptation for semantic segmentation using adversarial learning
We formalize this problem as a domain adaptation task and introduce a novel dataset of urban scenes with the related semantic labels.
Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video
The experiments show that the proposed architecture is state-of-the-art in the domain of egocentric videos, achieving top performances in the 2019 EPIC-Kitchens egocentric action anticipation challenge.
Ranked #7 on
Action Anticipation
on EPIC-KITCHENS-100 (test)
The EPIC-KITCHENS Dataset: Collection, Challenges and Baselines
Our dataset features 55 hours of video consisting of 11. 5M frames, which we densely labelled for a total of 39. 6K action segments and 454. 2K object bounding boxes.
Knowledge Distillation for Action Anticipation via Label Smoothing
Since multiple actions may equally occur in the future, we treat action anticipation as a multi-label problem with missing labels extending the concept of label smoothing.
EGO-CH: Dataset and Fundamental Tasks for Visitors BehavioralUnderstanding using Egocentric Vision
Equipping visitors of a cultural site with a wearable device allows to easily collect information about their preferences which can be exploited to improve the fruition of cultural goods with augmented reality.
What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attention
Our method is ranked first in the public leaderboard of the EPIC-Kitchens egocentric action anticipation challenge 2019.
Ranked #3 on
Egocentric Activity Recognition
on EPIC-KITCHENS-55
Egocentric Visitors Localization in Cultural Sites
We consider the problem of localizing visitors in a cultural site from egocentric (first person) images.
Next-Active-Object prediction from Egocentric Videos
Although First Person Vision systems can sense the environment from the user's perspective, they are generally unable to predict his intentions and goals.
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset
First-person vision is gaining interest as it offers a unique viewpoint on people's interaction with objects, their attention, and even intention.
