Search Results for author:
Found 59 papers, 11 papers with code
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects
We interact with the world with our hands and see it through our own (egocentric) perspective.
Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimation
These two stereo constraints are used in a complementary manner to generate pseudo-labels, allowing reliable adaptation.
FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation
The dataset consists of multi-view videos of 32 participants performing mock biological experiments with a total duration of 14. 5 hours.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
Generative Hierarchical Temporal Transformer for Hand Action Recognition and Motion Prediction
We present a novel framework that concurrently tackles hand action recognition and 3D future hand motion prediction.
Exo2EgoDVC: Dense Video Captioning of Egocentric Procedural Activities Using Web Instructional Videos
We propose a novel benchmark for cross-view knowledge transfer of dense video captioning, adapting models from web instructional videos with exocentric views to an egocentric view.
Seeking Flat Minima with Mean Teacher on Semi- and Weakly-Supervised Domain Generalization for Object Detection
In contrast to the conventional domain generalization for object detection that requires labeled data from multiple domains, SS-DGOD and WS-DGOD require labeled data only from one domain and unlabeled or weakly-labeled data from multiple domains for training.
Image Cropping under Design Constraints
We explore two derived approaches, a proposal-based approach, and a heatmap-based approach, and we construct a dataset for evaluating the performance of the proposed approaches on image cropping under design constraints.
Proposal-based Temporal Action Localization with Point-level Supervision
Point-level supervised temporal action localization (PTAL) aims at recognizing and localizing actions in untrimmed videos where only a single point (frame) within every action instance is annotated in training data.
Surgical tool classification and localization: results and methods from the MICCAI 2022 SurgToolLoc challenge
Unfortunately, obtaining the annotations needed to train machine learning models to identify and localize surgical tools is a difficult task.
Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction
The Multiplane Image (MPI), containing a set of fronto-parallel RGBA layers, is an effective and efficient representation for view synthesis from sparse inputs.
Fine-grained Affordance Annotation for Egocentric Hand-Object Interaction Videos
Object affordance is an important concept in hand-object interaction, providing information on action possibilities based on human motor capacity and objects' physical property thus benefiting tasks such as action anticipation and robot imitation learning.
Weakly Supervised Temporal Sentence Grounding With Uncertainty-Guided Self-Training
The task of weakly supervised temporal sentence grounding aims at finding the corresponding temporal moments of a language description in the video, given video-language correspondence only at video-level.
DeCo: Decomposition and Reconstruction for Compositional Temporal Grounding via Coarse-To-Fine Contrastive Ranking
Compositional temporal grounding is the task of localizing dense action by using known words combined in novel ways in the form of novel query sentences for the actual grounding.
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Image cropping has progressed tremendously under the data-driven paradigm.
Surgical Skill Assessment via Video Semantic Aggregation
Automated video-based assessment of surgical skills is a promising task in assisting young surgical trainees, especially in poor-resource areas.
CompNVS: Novel View Synthesis with Scene Completion
We introduce a scalable framework for novel view synthesis from RGB-D images with largely incomplete scene coverage.
Compound Prototype Matching for Few-shot Action Recognition
Each global prototype is encouraged to summarize a specific aspect from the entire video, for example, the start/evolution of the action.
Precise Affordance Annotation for Egocentric Action Video Datasets
Object affordance is an important concept in human-object interaction, providing information on action possibilities based on human motor capacity and objects' physical property thus benefiting tasks such as action anticipation and robot imitation learning.
Object Instance Identification in Dynamic Environments
We study the problem of identifying object instances in a dynamic environment where people interact with the objects.
Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey
In this survey, we present a systematic review of 3D hand pose estimation from the perspective of efficient annotation and learning.
Domain Adaptive Hand Keypoint and Pixel Localization in the Wild
We aim to improve the performance of regressing hand keypoints and segmenting pixel-level hand masks under new imaging conditions (e. g., outdoors) when we only have labeled images taken under very different conditions (e. g., indoors).
Background Mixup Data Augmentation for Hand and Object-in-Contact Detection
Detecting the positions of human hands and objects-in-contact (hand-object detection) in each video frame is vital for understanding human activities from videos.
Interact Before Align: Leveraging Cross-Modal Knowledge for Domain Adaptive Action Recognition
Different from previous works, we find that the cross-domain alignment can be more effectively done by using cross-modal interaction first.
Stacked Temporal Attention: Improving First-person Action Recognition by Emphasizing Discriminative Clips
Previous works explored to address this problem by applying temporal attention but failed to consider the global context of the full video, which is critical for determining the relatively significant parts.
Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data
Then we design a novel Auxiliary Attention Block (AAB) to allow information from SAN to be utilized by the backbone encoder to focus on selective areas.
Neural Routing by Memory
However, they use the same procedure sequence for all inputs, regardless of the intermediate features. This paper proffers a simple yet effective idea of constructing parallel procedures and assigning similar intermediate features to the same specialized procedures in a divide-and-conquer fashion.
Hand-Object Contact Prediction via Motion-Based Pseudo-Labeling and Guided Progressive Label Correction
In this study, we introduce a video-based method for predicting contact between a hand and an object.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
Spatio-Temporal Perturbations for Video Attribution
The attribution method provides a direction for interpreting opaque neural networks in a visual way by identifying and visualizing the input regions/pixels that dominate the output of a network.
Foreground-Aware Stylization and Consensus Pseudo-Labeling for Domain Adaptation of First-Person Hand Segmentation
We validated our method on domain adaptation of hand segmentation from real and simulation images.
EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition 2021: Team M3EM Technical Report
In this report, we describe the technical details of our submission to the 2021 EPIC-KITCHENS-100 Unsupervised Domain Adaptation Challenge for Action Recognition.
GO-Finder: A Registration-Free Wearable System for Assisting Users in Finding Lost Objects via Hand-Held Object Discovery
People spend an enormous amount of time and effort looking for lost objects.
Towards Visually Explaining Video Understanding Networks with Perturbation
''Making black box models explainable'' is a vital problem that accompanies the development of deep learning networks.
A Computer-Aided Diagnosis System Using Artificial Intelligence for Hip Fractures -Multi-Institutional Joint Development Research-
Gradient-weighted class activation mapping (Grad-CAM) was used to conceptualize the diagnostic basis of the CAD system.
Manipulation-skill Assessment from Videos with Spatial Attention Network
Recent advances in computer vision have made it possible to automatically assess from videos the manipulation skills of humans in performing a task, which breeds many important applications in domains such as health rehabilitation and manufacturing.
Mutual Context Network for Jointly Estimating Egocentric Gaze and Actions
In this work, we address two coupled tasks of gaze prediction and action recognition in egocentric videos by exploring their mutual context.
Understanding hand-object manipulation by modeling the contextual relationship between actions, grasp types and object attributes
In the proposed model, we explore various semantic relationships between actions, grasp types and object attributes, and show how the context can be used to boost the recognition of each component.
Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transition
We present a new computational model for gaze prediction in egocentric videos by exploring patterns in temporal shift of gaze fixations (attention transition) that are dependent on egocentric manipulation tasks.
Future Person Localization in First-Person Videos
We present a new task that predicts future locations of people observed in first-person videos.
From RGB to Spectrum for Natural Scenes via Manifold-Based Mapping
Spectral analysis of natural scenes can provide much more detailed information about the scene than an ordinary RGB camera.
Fast Multi-frame Stereo Scene Flow with Motion Segmentation
This unified framework benefits all four tasks - stereo, optical flow, visual odometry and motion segmentation leading to overall higher accuracy and efficiency.
Hierarchical Gaussian Descriptors with Application to Person Re-Identification
To solve this problem, we describe a local region in an image via hierarchical Gaussian distribution in which both means and covariances are included in their parameters.
Privacy-Preserving Visual Learning Using Doubly Permuted Homomorphic Encryption
We propose a privacy-preserving framework for learning visual classifiers by leveraging distributed private image data.
Ego-Surfing: Person Localization in First-Person Videos Using Ego-Motion Signatures
We envision a future time when wearable cameras are worn by the masses and recording first-person point-of-view videos of everyday life.
Exploiting Spectral-Spatial Correlation for Coded Hyperspectral Image Restoration
In this paper, we propose an effective method for coded hyperspectral image restoration, which exploits extensive structure sparsity in the hyperspectral image.
Joint Recovery of Dense Correspondence and Cosegmentation in Two Images
We propose a new technique to jointly recover cosegmentation and dense per-pixel correspondence in two images.
Recognizing Micro-Actions and Reactions From Paired Egocentric Videos
We aim to understand the dynamics of social interactions between two people by recognizing their actions and reactions using a head-mounted camera.
Hierarchical Gaussian Descriptor for Person Re-Identification
In both steps, unlike the hierarchical covariance descriptor, the proposed descriptor can model both the mean and the covariance information of pixel features properly.
Continuous 3D Label Stereo Matching using Local Expansion Moves
The local expansion moves extend traditional expansion moves by two ways: localization and spatial propagation.
Separating Fluorescent and Reflective Components by Using a Single Hyperspectral Image
This paper introduces a novel method to separate fluorescent and reflective components in the spectral domain.
Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Denoising
Hyperspectral imaging is beneficial in a diverse range of applications from diagnostic medicine, to agriculture, to surveillance to name a few.
Uncalibrated Photometric Stereo Based on Elevation Angle Recovery From BRDF Symmetry of Isotropic Materials
This sort of symmetry can be observed in a 1D BRDF slice from a subset of surface normals with the same azimuth angle, and we use it to devise an efficient modeling and solution method to constrain and recover the elevation angles of surface normals accurately.
Ego-Surfing First-Person Videos
We incorporate this feature into our proposed approach that computes the motion correlation over supervoxel hierarchies to localize target instances in observer videos.
Illumination and Reflectance Spectra Separation of a Hyperspectral Image Meets Low-Rank Matrix Factorization
This paper addresses the illumination and reflectance spectra separation (IRSS) problem of a hyperspectral image captured under general spectral illumination.
Learning-by-Synthesis for Appearance-based 3D Gaze Estimation
Unlike existing appearance-based methods that assume person-specific training data, we use a large amount of cross-subject training data to train a 3D gaze estimator.
Reflectance and Fluorescent Spectra Recovery based on Fluorescent Chromaticity Invariance under Varying Illumination
We then show that given the spectral reflectance and fluorescent chromaticity, the fluorescence absorption and emission spectra can also be estimated.
Shape-Preserving Half-Projective Warps for Image Stitching
It provides good alignment accuracy as projective warps while preserving the perspective of individual image as similarity warps.
Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances
We propose an uncalibrated photometric stereo method that works with general and unknown isotropic reflectances.
