Search Results for author:
Found 28 papers, 11 papers with code
HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos
We introduce HOT3D, a publicly available dataset for egocentric hand and object tracking in 3D.
EgoLM: Multi-Modal Language Model of Egocentric Motions
Multi-modal sensor inputs are encoded and projected to the joint latent space of language models, and used to prompt motion generation or text generation for egomotion tracking or understanding, respectively.
HMD$^2$: Environment-aware Motion Generation from Single Egocentric Head-Mounted Device
This paper investigates the online generation of realistic full-body human motion using a single head-mounted device with an outward-facing color camera and the ability to perform visual SLAM.
EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Models
The advent of wearable computers enables a new source of context for AI that is embedded in egocentric sensor data.
Ranked #1 on
3D Reconstruction
on Aria Synthetic Environments
Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild
To the best of our knowledge, Nymeria dataset is the world's largest collection of human motion in the wild; first of its kind to provide synchronized and localized multi-device multimodal egocentric data; and the world's largest motion-language dataset.
Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Tracking
The dataset offers over 833 minutes (more than 3. 7M images) of multi-view RGB/monochrome image streams showing 19 subjects interacting with 33 diverse rigid objects, multi-modal signals such as eye gaze or scene point clouds, as well as comprehensive ground truth annotations including 3D poses of objects, hands, and cameras, and 3D models of hands and objects.
SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model
We introduce SceneScript, a method that directly produces full scene models as a sequence of structured language commands using an autoregressive, token-based approach.
Aria Everyday Activities Dataset
We present Aria Everyday Activities (AEA) Dataset, an egocentric multimodal open dataset recorded using Project Aria glasses.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
EgoBlur: Responsible Innovation in Aria
Project Aria pushes the frontiers of Egocentric AI with large-scale real-world data collection using purposely designed glasses with privacy first approach.
Project Aria: A New Tool for Egocentric Multi-Modal AI Research
Egocentric, multi-modal data as available on future augmented reality (AR) devices provides unique challenges and opportunities for machine perception.
Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perception
We introduce the Aria Digital Twin (ADT) - an egocentric dataset captured using Aria glasses with extensive object, environment, and human level ground truth.
EgoHumans: An Egocentric 3D Multi-Human Benchmark
We present EgoHumans, a new multi-view multi-human video benchmark to advance the state-of-the-art of egocentric human 3D pose estimation and tracking.
OrienterNet: Visual Localization in 2D Public Maps with Neural Matching
We bridge this gap by introducing OrienterNet, the first deep neural network that can localize an image with sub-meter accuracy using the same 2D semantic maps that humans use.
Ego-Humans: An Ego-Centric 3D Multi-Human Benchmark
We present EgoHumans, a new multi-view multi-human video benchmark to advance the state-of-the-art of egocentric human 3D pose estimation and tracking.
Nerfels: Renderable Neural Codes for Improved Camera Pose Estimation
This paper presents a framework that combines traditional keypoint-based camera pose optimization with an invertible neural rendering mechanism.
Self-supervised Neural Articulated Shape and Appearance Models
In addition, our representation enables a large variety of applications, such as few-shot reconstruction, the generation of novel articulations, and novel view-synthesis.
LISA: Learning Implicit Shape and Appearance of Hands
This paper proposes a do-it-all neural model of human hands, named LISA.
ERF: Explicit Radiance Field Reconstruction From Scratch
Robustly reconstructing such a volumetric scene model with millions of unknown variables from registered scene images only is a highly non-convex and complex optimization problem.
Ego4D: Around the World in 3,000 Hours of Egocentric Video
We introduce Ego4D, a massive-scale egocentric video dataset and benchmark suite.
ODAM: Object Detection, Association, and Mapping using Posed RGB Video
Localizing objects and estimating their extent in 3D is an important step towards high-level 3D scene understanding, which has many applications in Augmented Reality and Robotics.
Neural 3D Video Synthesis from Multi-view Video
We propose a novel approach for 3D video synthesis that is able to represent multi-view video recordings of a dynamic real-world scene in a compact, yet expressive representation that enables high-quality view synthesis and motion interpolation.
FroDO: From Detections to 3D Objects
We introduce FroDO, a method for accurate 3D reconstruction of object instances from RGB video that infers object location, pose and shape in a coarse-to-fine manner.
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Efficiently reconstructing complex and intricate surfaces at scale is a long-standing goal in machine perception.
The Replica Dataset: A Digital Replica of Indoor Spaces
We introduce Replica, a dataset of 18 highly photo-realistic 3D indoor scene reconstructions at room and building scale.
StereoDRNet: Dilated Residual Stereo Net
We propose a system that uses a convolution neural network (CNN) to estimate depth from a stereo pair followed by volumetric fusion of the predicted depth maps to produce a 3D reconstruction of a scene.
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
In this work, we introduce DeepSDF, a learned continuous Signed Distance Function (SDF) representation of a class of shapes that enables high quality shape representation, interpolation and completion from partial and noisy 3D input data.
Surface Light Field Fusion
We present an approach for interactively scanning highly reflective objects with a commodity RGBD sensor.
