Search Results for author:
Found 19 papers, 7 papers with code
TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric Vision
Our method TouchInsight comprises a neural network to predict the moment of a touch event, the finger making contact, and the touch location.
X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalization
Lately, there has been growing interest in adapting vision-language models (VLMs) to image and third-person video classification due to their success in zero-shot recognition.
DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions
For the textual guidance, we contribute comprehensive text descriptions to the GRAB dataset and show that they enable our method to have more fine-grained control over hand-object interactions.
On the Utility of 3D Hand Poses for Action Recognition
3D hand pose is an underexplored modality for action recognition.
Ranked #1 on
Action Recognition
on H2O (2 Hands and Objects)
BID: Boundary-Interior Decoding for Unsupervised Temporal Action Localization Pre-Trainin
Skeleton-based motion representations are robust for action localization and understanding for their invariance to perspective, lighting, and occlusion, compared with images.
Opening the Vocabulary of Egocentric Actions
Given a set of verbs and objects observed during training, the goal is to generalize the verbs to an open vocabulary of actions with seen and novel objects.
Ranked #1 on
Open Vocabulary Action Recognition
on Assembly101
(using extra training data)
Every Mistake Counts in Assembly
Our framework constructs a knowledge base with spatial and temporal beliefs based on observed mistakes.
Data-Free Class-Incremental Hand Gesture Recognition
Our sampling scheme outperforms SOTA methods significantly on two 3D skeleton gesture datasets, the publicly available SHREC 2017, and EgoGesture3D -- which we extract from a publicly available RGBD dataset.
LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space
To mitigate this asymmetry, we introduce a prior model that is conditioned on the runtime inputs and tie this prior space to the 3D face model via a normalizing flow in the latent space.
Pixel Codec Avatars
Telecommunication with photorealistic avatars in virtual or augmented reality is a promising path for achieving authentic face-to-face communication in 3D over remote physical distances.
F-CAD: A Framework to Explore Hardware Accelerators for Codec Avatar Decoding
Creating virtual avatars with realistic rendering is one of the most essential and challenging tasks to provide highly immersive virtual reality (VR) experiences.
Expressive Telepresence via Modular Codec Avatars
It is important to note that traditional person-specific CAs are learned from few training samples, and typically lack robustness as well as limited expressiveness when transferring facial expressions.
Audio- and Gaze-driven Facial Animation of Codec Avatars
Codec Avatars are a recent class of learned, photorealistic face models that accurately represent the geometry and texture of a person in 3D (i. e., for virtual reality), and are almost indistinguishable from video.
To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversations
In this paper, we introduce a neural architecture named Dyadic Residual-Attention Model (DRAM), which integrates intrapersonal (monadic) and interpersonal (dyadic) dynamics using selective attention to generate sequences of body pose conditioned on audio and body pose of the interlocutor and audio of the human operating the avatar.
Recycle-GAN: Unsupervised Video Retargeting
We introduce a data-driven approach for unsupervised video retargeting that translates content from one domain to another while preserving the style native to a domain, i. e., if contents of John Oliver's speech were to be transferred to Stephen Colbert, then the generated content/speech should be in Stephen Colbert's style.
Salient Object Subitizing
We study the problem of Salient Object Subitizing, i. e. predicting the existence and the number of salient objects in an image using holistic cues.
Learning Activity Progression in LSTMs for Activity Detection and Early Detection
In this work we improve training of temporal deep models to better learn activity progression for activity detection and early detection.
Do Less and Achieve More: Training CNNs for Action Recognition Utilizing Action Images from the Web
In contrast, collecting action images from the Web is much easier and training on images requires much less computation.
Ranked #14 on
Action Recognition
on ActivityNet
(using extra training data)
Space-Time Tree Ensemble for Action Recognition
Using the action vocabulary we then utilize tree mining with subsequent tree clustering and ranking to select a compact set of highly discriminative tree patterns.
