Search Results for author:
Found 25 papers, 7 papers with code
MediaPipe: A Framework for Building Perception Pipelines
A developer can use MediaPipe to build prototypes by combining existing perception components, to advance them to polished cross-platform applications and measure system performance and resource consumption on target platforms.
Distributed, Parallel, and Cluster Computing
BlazePose: On-device Real-time Body Pose tracking
We present BlazePose, a lightweight convolutional neural network architecture for human pose estimation that is tailored for real-time inference on mobile devices.
Ranked #1 on
3D Pose Estimation
on Google-Yoga
Attention Mesh: High-fidelity Face Mesh Prediction in Real-time
We present Attention Mesh, a lightweight architecture for 3D face mesh prediction that uses attention to semantically meaningful regions.
BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs
We present BlazeFace, a lightweight and well-performing face detector tailored for mobile GPU inference.
Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations
3D object detection has recently become popular due to many applications in robotics, augmented reality, autonomy, and image retrieval.
Ranked #2 on
Monocular 3D Object Detection
on Google Objectron
Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs
The relatively dense mesh model of 468 vertices is well-suited for face-based AR effects.
MediaPipe Hands: On-device Real-time Hand Tracking
We present a real-time on-device hand tracking pipeline that predicts hand skeleton from single RGB camera for AR/VR applications.
Finding Temporally Consistent Occlusion Boundaries in Videos using Geometric Context
Our proposed framework provides an efficient approach for finding temporally consistent occlusion boundaries in video by utilizing causality, redundancy in videos, and semantic layout of the scene.
Geometric Context from Videos
We present a novel algorithm for estimating the broad 3D geometric structure of outdoor video scenes.
On-Device Neural Net Inference with Mobile GPUs
On-device inference of machine learning models for mobile phones is desirable due to its lower latency and increased privacy.
Instant Motion Tracking and Its Applications to Augmented Reality
Augmented Reality (AR) brings immersive experiences to users.
MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
The former is used when there is only pose supervision, and the latter is for the case when shape supervision is available, even a weak one.
Ranked #3 on
Monocular 3D Object Detection
on Google Objectron
Instant 3D Object Tracking with Applications in Augmented Reality
Our tracker is capable of performing relative-scale 9-DoF tracking in real-time on mobile devices.
On the Estimation of the Number of Unreachable Peers in the Bitcoin P2P Network by Observation of Peer Announcements
Thus, the number of unreachable peers can only be estimated based on some indicators.
Cryptography and Security Networking and Internet Architecture
On-device Real-time Hand Gesture Recognition
We present an on-device real-time hand gesture recognition (HGR) system, which detects a set of predefined static gestures from a single RGB camera.
BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation
We present BlazePose GHUM Holistic, a lightweight neural network pipeline for 3D human body landmarks and pose estimation, specifically tailored to real-time on-device inference.
Efficient Heterogeneous Video Segmentation at the Edge
We introduce an efficient video segmentation system for resource-limited edge devices leveraging heterogeneous compute.
Guided Speech Enhancement Network
Multi-microphone speech enhancement problem is often decomposed into two decoupled steps: a beamformer that provides spatial filtering and a single-channel speech enhancement model that cleans up the beamformer output.
Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations
The rapid development and application of foundation models have revolutionized the field of artificial intelligence.
Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond
An authentic face restoration system is becoming increasingly demanding in many computer vision applications, e. g., image enhancement, video communication, and taking portrait.
Blendshapes GHUM: Real-time Monocular Facial Blendshape Prediction
We present Blendshapes GHUM, an on-device ML pipeline that predicts 52 facial blendshape coefficients at 30+ FPS on modern mobile phones, from a single monocular RGB image and enables facial motion capture applications like virtual avatars.
On-device Real-time Custom Hand Gesture Recognition
Our framework provides a pre-trained single-hand embedding model that can be fine-tuned for custom gesture recognition.
StreamVC: Real-Time Low-Latency Voice Conversion
We present StreamVC, a streaming voice conversion solution that preserves the content and prosody of any source speech while matching the voice timbre from any target speech.
Binaural Angular Separation Network
We propose a neural network model that can separate target speech sources from interfering sources at different angular regions using two microphones.
PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models
However, in the vision domain, existing RL-based reward finetuning methods are limited by their instability in large-scale training, rendering them incapable of generalizing to complex, unseen prompts.
