Search Results for author:
Found 54 papers, 21 papers with code
Human-Object Interaction from Human-Level Instructions
Our experiments demonstrate the effectiveness of our high-level planner in generating plausible target layouts and our low-level motion generator in synthesizing realistic interactions for diverse objects.
Common Sense Reasoning
Human-Object Interaction Detection
+4
Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild
To the best of our knowledge, the Nymeria dataset is the world largest in-the-wild collection of human motion with natural and diverse activities; first of its kind to provide synchronized and localized multi-device multimodal egocentric data; and the world largest dataset with motion-language descriptions.
AddBiomechanics Dataset: Capturing the Physics of Human Motion at Scale
While reconstructing human poses in 3D from inexpensive sensors has advanced significantly in recent years, quantifying the dynamics of human motion, including the muscle-generated joint torques and external forces, remains a challenge.
EgoNav: Egocentric Scene-aware Human Trajectory Prediction
Wearable collaborative robots stand to assist human wearers who need fall prevention assistance or wear exoskeletons.
BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation
We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics.
DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation
Imitation learning from human hand motion data presents a promising avenue for imbuing robots with human-like dexterity in real-world manipulation tasks.
Iterative Motion Editing with Natural Language
Text-to-motion diffusion models can generate realistic animations from text prompts, but do not support fine-grained motion editing controls.
Controllable Human-Object Interaction Synthesis
We demonstrate that our learned interaction module can synthesize realistic human-object interactions, adhering to provided textual descriptions and sparse waypoint conditions.
Learning to Design and Use Tools for Robotic Manipulation
A designer policy is conditioned on task information and outputs a tool design that helps solve the task.
State of the Art on Diffusion Models for Visual Computing
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.
Object Motion Guided Human Motion Synthesis
We propose Object MOtion guided human MOtion synthesis (OMOMO), a conditional diffusion framework that can generate full-body manipulation behaviors from only the object motion.
DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics
We introduce DROP, a novel framework for modeling Dynamics Responses of humans using generative mOtion prior and Projective dynamics.
Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation
However, the challenges arise due to the high-dimensional action space of dexterous hand and complex compositional dynamics of the long-horizon tasks.
DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion
Unlike existing methods, our model grants users the flexibility to determine the number and arrangement of sensors tailored to the specific activity of interest, without the need for retraining.
CIRCLE: Capture In Rich Contextual Environments
Leveraging our dataset, the model learns to use ego-centric scene information to achieve nontrivial reaching tasks in the context of complex 3D scenes.
Scene Synthesis from Human Motion
Large-scale capture of human motion with diverse, complex scenes, while immensely useful, is often considered prohibitively costly.
Ranked #3 on
3D Semantic Scene Completion
on PRO-teXt
2D Semantic Segmentation task 1 (8 classes)
3D Semantic Scene Completion
+1
NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action
Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection.
Physically Plausible Animation of Human Upper Body from a Single Image
We present a new method for generating controllable, dynamically responsive, and photorealistic human animations.
Ego-Body Pose Estimation via Ego-Head Pose Estimation
In addition, collecting large-scale, high-quality datasets with paired egocentric videos and 3D human motions requires accurate motion capture devices, which often limit the variety of scenes in the videos to lab-like environments.
EDGE: Editable Dance Generation From Music
Dance is an important human art form, but creating new dances can be difficult and time-consuming.
Ranked #1 on
Motion Synthesis
on AIST++
(Beat alignment score metric)
GIMO: Gaze-Informed Human Motion Prediction in Context
We perform an extensive study of the benefits of leveraging the eye gaze for ego-centric human motion prediction with various state-of-the-art architectures.
Transformer Inertial Poser: Real-time Human Motion Reconstruction from Sparse IMUs with Simultaneous Terrain Generation
Real-time human motion reconstruction from a sparse set of (e. g. six) wearable IMUs provides a non-intrusive and economic approach to motion capture.
A Survey on Reinforcement Learning Methods in Character Animation
Reinforcement Learning is an area of Machine Learning focused on how agents can be trained to make sequential decisions, and achieve a particular goal within an arbitrary environment.
ADeLA: Automatic Dense Labeling With Attention for Viewpoint Shift in Semantic Segmentation
We describe a method to deal with performance drop in semantic segmentation caused by viewpoint changes within multi-camera systems, where temporally paired images are readily available, but the annotations may only be abundant for a few typical views.
Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration
Our method co-optimizes a human policy and a robot policy in an interactive learning process: the human policy learns to generate diverse and plausible collaborative behaviors from demonstrations while the robot policy learns to assist by estimating the unobserved latent strategy of its human collaborator.
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation, spanning a range of everyday household chores such as cleaning, maintenance, and food preparation.
iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Tasks
We evaluate the new capabilities of iGibson 2. 0 to enable robot learning of novel tasks, in the hope of demonstrating the potential of this new simulator to support new research in embodied AI.
ADeLA: Automatic Dense Labeling with Attention for Viewpoint Adaptation in Semantic Segmentation
Furthermore, to resolve ambiguities in converting the semantic images to semantic labels, we treat the view transformation network as a functional representation of an unknown mapping implied by the color images and propose functional label hallucination to generate pseudo-labels in the target domain.
DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis
We describe a method for unpaired realistic depth synthesis that learns diverse variations from the real-world depth scans and ensures geometric consistency between the synthetic and synthesized depth.
Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contact
We present a fast and feature-complete differentiable physics engine, Nimble (nimblephysics. org), that supports Lagrangian dynamics and hard contact constraints for articulated rigid body simulation.
Error-Aware Policy Learning: Zero-Shot Generalization in Partially Observable Dynamic Environments
An EAP takes as input the predicted future state error in the target environment, which is provided by an error-prediction function, simultaneously trained with the EAP.
Task-Specific Design Optimization and Fabrication for Inflated-Beam Soft Robots with Growable Discrete Joints
Soft robot serial chain manipulators with the capability for growth, stiffness control, and discrete joints have the potential to approach the dexterity of traditional robot arms, while improving safety, lowering cost, and providing an increased workspace, with potential application in home environments.
Robotics
Learning to Manipulate Amorphous Materials
We produce a final animation by using inverse kinematics to guide a character's arm and hand to match the motion of the manipulation tool such as a knife or a frying pan.
Protective Policy Transfer
When used with a set of thresholds, the safety estimator becomes a classifier for switching between the protective policy and the task policy.
Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshop
This report presents the debates, posters, and discussions of the Sim2Real workshop held in conjunction with the 2020 edition of the "Robotics: Science and System" conference.
COCOI: Contact-aware Online Context Inference for Generalizable Non-planar Pushing
General contact-rich manipulation problems are long-standing challenges in robotics due to the difficulty of understanding complicated contact physics.
Policy Transfer via Kinematic Domain Randomization and Adaptation
Transferring reinforcement learning policies trained in physics simulation to the real hardware remains a challenge, known as the "sim-to-real" gap.
Learning Task-Agnostic Action Spaces for Movement Optimization
We propose a novel method for exploring the dynamics of physically based animated characters, and learning a task-agnostic action space that makes movement optimization easier.
Bodies at Rest: 3D Human Pose and Shape Estimation from a Pressure Image using Synthetic Data
We describe a physics-based method that simulates human bodies at rest in a bed with a pressure sensing mat, and present PressurePose, a synthetic dataset with 206K pressure images with 3D human poses and shapes.
3D human pose and shape estimation
3D Human Shape Estimation
+1
Assistive Gym: A Physics Simulation Framework for Assistive Robotics
Assistive Gym models a person's physical capabilities and preferences for assistance, which are used to provide a reward function.
Visualizing Movement Control Optimization Landscapes
A large body of animation research focuses on optimization of movement control, either as action sequences or policy parameters.
Learning to Collaborate from Simulation for Robot-Assisted Dressing
We investigated the application of haptic feedback control and deep reinforcement learning (DRL) to robot-assisted dressing.
Estimating Mass Distribution of Articulated Objects using Non-prehensile Manipulation
Using our method, we train a robotic arm to estimate the mass distribution of an object with moving parts (e. g. an articulated rigid body system) by pushing it on a surface with unknown friction properties.
Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuation
In addition, the metabolic energy function on muscle activations is transformed to a nonlinear function of joint torques, joint configuration and joint velocity.
Sim-to-Real Transfer for Biped Locomotion
We present a new approach for transfer of dynamic robot control policies such as biped locomotion from simulation to real hardware.
Policy Transfer with Strategy Optimization
Transfer learning using domain randomization is a promising approach, but it usually assumes that the target environment is close to the distribution of the training environments, thus relying heavily on accurate system identification.
Data-Augmented Contact Model for Rigid Body Simulation
Accurately modeling contact behaviors for real-world, near-rigid materials remains a grand challenge for existing rigid-body physics simulators.
Learning Symmetric and Low-energy Locomotion
Indeed, a standard benchmark for DRL is to automatically create a running controller for a biped character from a simple reward function.
Deep Haptic Model Predictive Control for Robot-Assisted Dressing
The physical implications of dressing are complicated by non-rigid garments, which can result in a robot indirectly applying high forces to a person's body.
Multi-task Learning with Gradient Guided Policy Specialization
Then, during the specialization training stage we selectively split the weights of the policy based on a per-weight metric that measures the disagreement among the multiple tasks.
Learning a Unified Control Policy for Safe Falling
With this mixture of actor-critic architecture, the discrete contact sequence planning is solved through the selection of the best critics while the continuous control problem is solved by the optimization of actors.
Preparing for the Unknown: Learning a Universal Policy with Online System Identification
Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment.
A Linear-Time Variational Integrator for Multibody Systems
Our key contribution is to derive a recursive algorithm that evaluates DEL equations in $O(n)$, which scales up well for complex multibody systems such as humanoid robots.
Robotics
Traversing Environments Using Possibility Graphs for Humanoid Robots
Locomotion for legged robots poses considerable challenges when confronted by obstacles and adverse environments.
