Search Results for author: Michael J. Black

Found 121 papers, 65 papers with code

MeshDiffusion: Score-based Generative 3D Mesh Modeling

no code implementations14 Mar 2023 Zhen Liu, Yao Feng, Michael J. Black, Derek Nowrouzezahrai, Liam Paull, Weiyang Liu

We consider the task of generating realistic 3D shapes, which is useful for a variety of applications such as automatic scene generation and physical simulation.

Scene Generation

Detecting Human-Object Contact in Images

no code implementations6 Mar 2023 Yixin Chen, Sai Kumar Dwivedi, Michael J. Black, Dimitrios Tzionas

To build HOT, we use two data sources: (1) We use the PROX dataset of 3D human meshes moving in 3D scenes, and automatically annotate 2D image areas for contact via 3D mesh proximity and projection.

PointAvatar: Deformable Point-based Head Avatars from Videos

1 code implementation16 Dec 2022 Yufeng Zheng, Wang Yifan, Gordon Wetzstein, Michael J. Black, Otmar Hilliges

The ability to create realistic, animatable and relightable head avatars from casual video sequences would open up wide ranging applications in communication and entertainment.

HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics

no code implementations14 Dec 2022 Artur Grigorev, Bernhard Thomaszewski, Michael J. Black, Otmar Hilliges

We propose a method that leverages graph neural networks, multi-level message passing, and unsupervised training to enable real-time prediction of realistic clothing dynamics.

ECON: Explicit Clothed humans Optimized via Normal integration

1 code implementation14 Dec 2022 Yuliang Xiu, Jinlong Yang, Xu Cao, Dimitrios Tzionas, Michael J. Black

To increase robustness for these cases, existing work uses an explicit parametric body model to constrain surface reconstruction, but this limits the recovery of free-form surfaces such as loose clothing that deviates from the body.

Surface Reconstruction

MIME: Human-Aware 3D Scene Generation

no code implementations8 Dec 2022 Hongwei Yi, Chun-Hao P. Huang, Shashank Tripathi, Lea Hering, Justus Thies, Michael J. Black

We propose MIME (Mining Interaction and Movement to infer 3D Environments), which is a generative model of indoor scenes that produces furniture layouts that are consistent with the human movement.

Scene Generation

Generating Holistic 3D Human Motion from Speech

no code implementations8 Dec 2022 Hongwei Yi, Hualin Liang, Yifei Liu, Qiong Cao, Yandong Wen, Timo Bolkart, DaCheng Tao, Michael J. Black

This work addresses the problem of generating 3D holistic body motions from human speech.

Fast-SNARF: A Fast Deformer for Articulated Neural Fields

1 code implementation28 Nov 2022 Xu Chen, Tianjian Jiang, Jie Song, Max Rietmann, Andreas Geiger, Michael J. Black, Otmar Hilliges

A key challenge in making such methods applicable to articulated objects, such as the human body, is to model the deformation of 3D locations between the rest pose (a canonical space) and the deformed space.

3D Reconstruction Novel View Synthesis

SUPR: A Sparse Unified Part-Based Human Representation

1 code implementation25 Oct 2022 Ahmed A. A. Osman, Timo Bolkart, Dimitrios Tzionas, Michael J. Black

Using novel 4D scans of feet, we train a model with an extended kinematic tree that captures the range of motion of the toes.

Capturing and Animation of Body and Clothing from Monocular Video

1 code implementation4 Oct 2022 Yao Feng, Jinlong Yang, Marc Pollefeys, Michael J. Black, Timo Bolkart

Building on this insight, we propose SCARF (Segmented Clothed Avatar Radiance Field), a hybrid model combining a mesh-based body with a neural radiance field.

Virtual Try-on

SmartMocap: Joint Estimation of Human and Camera Motion using Uncalibrated RGB Cameras

1 code implementation28 Sep 2022 Nitin Saini, Chun-Hao P. Huang, Michael J. Black, Aamir Ahmad

Second, we learn a probability distribution of short human motion sequences ($\sim$1sec) relative to the ground plane and leverage it to disambiguate between the camera and human motion.

InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction

no code implementations26 Sep 2022 Yinghao Huang, Omid Tehari, Michael J. Black, Dimitrios Tzionas

With this method we capture the InterCap dataset, which contains 10 subjects (5 males and 5 females) interacting with 10 objects of various sizes and affordances, including contact with the hands or feet.

Pose Estimation

Neural Point-based Shape Modeling of Humans in Challenging Clothing

no code implementations14 Sep 2022 Qianli Ma, Jinlong Yang, Michael J. Black, Siyu Tang

Specifically, we extend point-based methods with a coarse stage, that replaces canonicalization with a learned pose-independent "coarse shape" that can capture the rough surface geometry of clothing like skirts.

TEACH: Temporal Action Composition for 3D Humans

1 code implementation9 Sep 2022 Nikos Athanasiou, Mathis Petrovich, Michael J. Black, Gül Varol

In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition.

Motion Synthesis

LED: Latent Variable-based Estimation of Density

no code implementations23 Jun 2022 Omri Ben-Dov, Pravir Singh Gupta, Victoria Fernandez Abrevaya, Michael J. Black, Partha Ghosh

Modern generative models are roughly divided into two main categories: (1) models that can produce high-quality random samples, but cannot estimate the exact density of new data points and (2) those that provide exact density estimation, at the expense of sample quality and compactness of the latent space.

Density Estimation

Accurate 3D Body Shape Regression using Metric and Semantic Attributes

1 code implementation CVPR 2022 Vasileios Choutas, Lea Muller, Chun-Hao P. Huang, Siyu Tang, Dimitrios Tzionas, Michael J. Black

Since paired data with images and 3D body shape are rare, we exploit two sources of information: (1) we collect internet images of diverse "fashion" models together with a small set of anthropometric measurements; (2) we collect linguistic shape attributes for a wide range of 3D body meshes and the model images.

3D Human Reconstruction 3D Human Shape Estimation

Towards Racially Unbiased Skin Tone Estimation via Scene Disambiguation

no code implementations8 May 2022 Haiwen Feng, Timo Bolkart, Joachim Tesch, Michael J. Black, Victoria Abrevaya

Our experimental results show significant improvement compared to state-of-the-art methods on albedo estimation, both in terms of accuracy and fairness.


Articulated Objects in Free-form Hand Interaction

no code implementations28 Apr 2022 Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, Otmar Hilliges

Consequently, we introduce ARCTIC - the first dataset of free-form interactions of hands and articulated objects.

3D Reconstruction

TEMOS: Generating diverse human motions from textual descriptions

1 code implementation25 Apr 2022 Mathis Petrovich, Michael J. Black, Gül Varol

In contrast to most previous work which focuses on generating a single, deterministic, motion from a textual description, we design a variational approach that can produce multiple diverse human motions.

EMOCA: Emotion Driven Monocular Face Capture and Animation

1 code implementation CVPR 2022 Radek Danecek, Michael J. Black, Timo Bolkart

While EMOCA achieves 3D reconstruction errors that are on par with the current best methods, it significantly outperforms them in terms of the quality of the reconstructed expression and the perceived emotional content.

3D Face Reconstruction 3D Reconstruction +2

OSSO: Obtaining Skeletal Shape from Outside

1 code implementation CVPR 2022 Marilyn Keller, Silvia Zuffi, Michael J. Black, Sergi Pujades

We address the problem of inferring the anatomic skeleton of a person, in an arbitrary pose, from the 3D surface of the body; i. e. we predict the inside (bones) from the outside (skin).

BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information

no code implementations CVPR 2022 Nadine Rueegg, Silvia Zuffi, Konrad Schindler, Michael J. Black

But, even with a better shape model, the problem of regressing dog shape from an image is still challenging because we lack paired images with 3D ground truth.

LocATe: End-to-end Localization of Actions in 3D with Transformers

no code implementations21 Mar 2022 Jiankai Sun, Bolei Zhou, Michael J. Black, Arjun Chandrasekaran

An important component of this problem is 3D Temporal Action Localization (3D-TAL), which involves recognizing what actions a person is performing, and when.

Action Recognition object-detection +2

Human-Aware Object Placement for Visual Environment Reconstruction

1 code implementation CVPR 2022 Hongwei Yi, Chun-Hao P. Huang, Dimitrios Tzionas, Muhammed Kocabas, Mohamed Hassan, Siyu Tang, Justus Thies, Michael J. Black

In fact, we demonstrate that these human-scene interactions (HSIs) can be leveraged to improve the 3D reconstruction of a scene from a monocular RGB video.

3D Reconstruction

AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation

1 code implementation20 Jan 2022 Nitin Saini, Elia Bonetto, Eric Price, Aamir Ahmad, Michael J. Black

In this letter, we present a novel markerless 3D human motion capture (MoCap) system for unstructured, outdoor environments that uses a team of autonomous unmanned aerial vehicles (UAVs) with on-board RGB cameras and computation.

3D human pose and shape estimation

gDNA: Towards Generative Detailed Neural Avatars

no code implementations CVPR 2022 Xu Chen, Tianjian Jiang, Jie Song, Jinlong Yang, Michael J. Black, Andreas Geiger, Otmar Hilliges

Furthermore, we show that our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.

Embodied Hands: Modeling and Capturing Hands and Bodies Together

no code implementations7 Jan 2022 Javier Romero, Dimitrios Tzionas, Michael J. Black

We attach MANO to a standard parameterized 3D body shape model (SMPL), resulting in a fully articulated body and hand model (SMPL+H).

GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping

1 code implementation CVPR 2022 Omid Taheri, Vasileios Choutas, Michael J. Black, Dimitrios Tzionas

This is challenging, as it requires the avatar to walk towards the object with foot-ground contact, orient the head towards it, reach out, and grasp it with a realistic hand pose and hand-object contact.

I M Avatar: Implicit Morphable Head Avatars from Videos

1 code implementation CVPR 2022 Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, Otmar Hilliges

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details.

InvGAN: Invertible GANs

no code implementations8 Dec 2021 Partha Ghosh, Dominik Zietlow, Michael J. Black, Larry S. Davis, Xiaochen Hu

Our \textbf{InvGAN}, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model.

Data Augmentation Image Inpainting +1

SOMA: Solving Optical Marker-Based MoCap Automatically

2 code implementations ICCV 2021 Nima Ghorbani, Michael J. Black

Commercial auto-labeling tools require a specific calibration procedure at capture time, which is not possible for archival data.

Learning to Regress Bodies from Images using Differentiable Semantic Rendering

1 code implementation ICCV 2021 Sai Kumar Dwivedi, Nikos Athanasiou, Muhammed Kocabas, Michael J. Black

For Minimally-Clothed regions, we define the DSR-MC loss, which encourages a tight match between a rendered SMPL body and the minimally-clothed regions of the image.

Ranked #14 on 3D Human Pose Estimation on MPI-INF-3DHP (PA-MPJPE metric)

3D human pose and shape estimation

The Power of Points for Modeling Humans in Clothing

no code implementations ICCV 2021 Qianli Ma, Jinlong Yang, Siyu Tang, Michael J. Black

The geometry feature can be optimized to fit a previously unseen scan of a person in clothing, enabling the scan to be reposed realistically.

PARE: Part Attention Regressor for 3D Human Body Estimation

1 code implementation ICCV 2021 Muhammed Kocabas, Chun-Hao P. Huang, Otmar Hilliges, Michael J. Black

Despite significant progress, we show that state of the art 3D human pose and shape estimation methods remain sensitive to partial occlusion and can produce dramatically wrong predictions although much of the body is observable.

3D human pose and shape estimation 3D Multi-Person Pose Estimation

LEAP: Learning Articulated Occupancy of People

no code implementations CVPR 2021 Marko Mihajlovic, Yan Zhang, Michael J. Black, Siyu Tang

Substantial progress has been made on modeling rigid 3D objects using deep implicit representations.

Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

1 code implementation ICCV 2021 Mathis Petrovich, Michael J. Black, Gül Varol

By sampling from this latent space and querying a certain duration through a series of positional encodings, we synthesize variable-length motion sequences conditioned on a categorical action.

Action Recognition Denoising +2

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

1 code implementation ICCV 2021 Xu Chen, Yufeng Zheng, Michael J. Black, Otmar Hilliges, Andreas Geiger

However, this is problematic since the backward warp field is pose dependent and thus requires large amounts of data to learn.

On Self-Contact and Human Pose

1 code implementation CVPR 2021 Lea Müller, Ahmed A. A. Osman, Siyu Tang, Chun-Hao P. Huang, Michael J. Black

Third, we develop a novel HPS optimization method, SMPLify-XMC, that includes contact constraints and uses the known 3DCP body pose during fitting to create near ground-truth poses for MTP images.

Ranked #31 on 3D Human Pose Estimation on 3DPW (MPJPE metric)

3D Human Pose Estimation

SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks

1 code implementation CVPR 2021 Shunsuke Saito, Jinlong Yang, Qianli Ma, Michael J. Black

We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar.

Weakly-supervised Learning

Populating 3D Scenes by Learning Human-Scene Interaction

1 code implementation CVPR 2021 Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, Michael J. Black

Second, we show that POSA's learned representation of body-scene interaction supports monocular human pose estimation that is consistent with a 3D scene, improving on the state of the art.

Pose Estimation

We are More than Our Joints: Predicting how 3D Bodies Move

no code implementations CVPR 2021 Yan Zhang, Michael J. Black, Siyu Tang

We note that motion prediction methods accumulate errors over time, resulting in joints or markers that diverge from true human bodies.

Human motion prediction motion prediction +1

Monocular, One-stage, Regression of Multiple 3D People

1 code implementation ICCV 2021 Yu Sun, Qian Bao, Wu Liu, Yili Fu, Michael J. Black, Tao Mei

Through a body-center-guided sampling process, the body mesh parameters of all people in the image are easily extracted from the Mesh Parameter map.

 Ranked #1 on 3D Multi-Person Mesh Recovery on Relative Human (using extra training data)

3D Depth Estimation 3D Multi-Person Mesh Recovery +2

GRAB: A Dataset of Whole-Body Human Grasping of Objects

2 code implementations ECCV 2020 Omid Taheri, Nima Ghorbani, Michael J. Black, Dimitrios Tzionas

Training computers to understand, model, and synthesize human grasping requires a rich dataset containing complex 3D object shapes, detailed contact information, hand pose and shape, and the 3D body motion over time.

Grasp Contact Prediction Grasp Generation +1

STAR: Sparse Trained Articulated Human Body Regressor

1 code implementation ECCV 2020 Ahmed A. A. Osman, Timo Bolkart, Michael J. Black

The SMPL body model is widely used for the estimation, synthesis, and analysis of 3D human pose and shape.

SMPLpix: Neural Avatars from 3D Human Models

1 code implementation16 Aug 2020 Sergey Prokudin, Michael J. Black, Javier Romero

Recent advances in deep generative models have led to an unprecedented level of realism for synthetically generated images of humans.

PLACE: Proximity Learning of Articulation and Contact in 3D Environments

1 code implementation12 Aug 2020 Siwei Zhang, Yan Zhang, Qianli Ma, Michael J. Black, Siyu Tang

To synthesize realistic human-scene interactions, it is essential to effectively represent the physical contact and proximity between the body and the world.

Perpetual Motion: Generating Unbounded Human Motion

no code implementations27 Jul 2020 Yan Zhang, Michael J. Black, Siyu Tang

To address this problem, we propose a model to generate non-deterministic, \textit{ever-changing}, perpetual human motion, in which the global trajectory and the body pose are cross-conditioned.

Motion Estimation Time Series Analysis

AirCapRL: Autonomous Aerial Human Motion Capture using Deep Reinforcement Learning

no code implementations13 Jul 2020 Rahul Tallamraju, Nitin Saini, Elia Bonetto, Michael Pabst, Yu Tang Liu, Michael J. Black, Aamir Ahmad

We focus on vision-based MoCap, where the objective is to estimate the trajectory of body pose and shape of a single moving person using multiple micro aerial vehicles.

Decision Making reinforcement-learning +1

Generating 3D People in Scenes without People

3 code implementations CVPR 2020 Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J. Black, Siyu Tang

However, this is a challenging task for a computer as solving it requires that (1) the generated human bodies to be semantically plausible within the 3D environment (e. g. people sitting on the sofa or cooking near the stove), and (2) the generated human-scene interaction to be physically feasible such that the human body and scene do not interpenetrate while, at the same time, body-scene contact supports physical interactions.

Pose Estimation

Learning Multi-Human Optical Flow

2 code implementations24 Oct 2019 Anurag Ranjan, David T. Hoffmann, Dimitrios Tzionas, Siyu Tang, Javier Romero, Michael J. Black

Therefore, we develop a dataset of multi-human optical flow and train optical flow networks on this dataset.

Optical Flow Estimation

Attacking Optical Flow

1 code implementation ICCV 2019 Anurag Ranjan, Joel Janai, Andreas Geiger, Michael J. Black

In this paper, we extend adversarial patch attacks to optical flow networks and show that such attacks can compromise their performance.

Optical Flow Estimation Self-Driving Cars

Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop

1 code implementation ICCV 2019 Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas Daniilidis

Our approach is self-improving by nature, since better network estimates can lead the optimization to better solutions, while more accurate optimization fits provide better supervision for the network.

3D Human Shape Estimation 3D Multi-Person Pose Estimation

Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"

1 code implementation ICCV 2019 Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, Michael J. Black

In contrast to research on human pose, shape and texture estimation, training data for endangered species is limited, the animals are in complex natural scenes with occlusion, they are naturally camouflaged, travel in herds, and look similar to each other.

Pose Estimation Texture Synthesis

Resolving 3D Human Pose Ambiguities with 3D Scene Constraints

1 code implementation ICCV 2019 Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, Michael J. Black

To motivate this, we show that current 3D human pose estimation methods produce results that are not consistent with the 3D scene.

3D Human Pose Estimation

Capture, Learning, and Synthesis of 3D Speaking Styles

1 code implementation CVPR 2019 Daniel Cudeiro, Timo Bolkart, Cassidy Laidlaw, Anurag Ranjan, Michael J. Black

To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers.

3D Face Animation Talking Face Generation

AMASS: Archive of Motion Capture as Surface Shapes

4 code implementations ICCV 2019 Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, Michael J. Black

We achieve this using a new method, MoSh++, that converts mocap data into realistic 3D human meshes represented by a rigged body model; here we use SMPL [doi:10. 1145/2816795. 2818013], which is widely used and provides a standard skeletal representation as well as a fully rigged surface mesh.

Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences

no code implementations17 Oct 2018 Nikolas Hesse, Sergi Pujades, Michael J. Black, Michael Arens, Ulrich G. Hofmann, A. Sebastian Schroeder

To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants.

Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera

no code implementations ECCV 2018 Timo von Marcard, Roberto Henschel, Michael J. Black, Bodo Rosenhahn, Gerard Pons-Moll

In this work, we propose a method that combines a single hand-held camera and a set of Inertial Measurement Units (IMUs) attached at the body limbs to estimate accurate 3D poses in the wild.

3D Pose Estimation

Generating 3D faces using Convolutional Mesh Autoencoders

2 code implementations ECCV 2018 Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, Michael J. Black

To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface.

3D Face Modelling Face Model

Learning Human Optical Flow

1 code implementation14 Jun 2018 Anurag Ranjan, Javier Romero, Michael J. Black

Given this, we devise an optical flow algorithm specifically for human motion and show that it is superior to generic flow methods.

Optical Flow Estimation

Lions and Tigers and Bears: Capturing Non-Rigid, 3D, Articulated Shape From Images

no code implementations CVPR 2018 Silvia Zuffi, Angjoo Kanazawa, Michael J. Black

Animals are widespread in nature and the analysis of their shape and motion is important in many fields and industries.

Resisting Adversarial Attacks using Gaussian Mixture Variational Autoencoders

no code implementations31 May 2018 Partha Ghosh, Arpan Losalka, Michael J. Black

Our model has the form of a variational autoencoder, with a Gaussian mixture prior on the latent vector.

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation

1 code implementation CVPR 2019 Anurag Ranjan, Varun Jampani, Lukas Balles, Kihwan Kim, Deqing Sun, Jonas Wulff, Michael J. Black

We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical flow, and segmentation of a video into the static scene and moving regions.

Depth Prediction Monocular Depth Estimation +3

On the Integration of Optical Flow and Action Recognition

no code implementations22 Dec 2017 Laura Sevilla-Lara, Yiyi Liao, Fatma Guney, Varun Jampani, Andreas Geiger, Michael J. Black

Here we take a deeper look at the combination of flow and action recognition, and investigate why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better.

Action Recognition Optical Flow Estimation +1

End-to-end Recovery of Human Shape and Pose

7 code implementations CVPR 2018 Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik

The main objective is to minimize the reprojection loss of keypoints, which allow our model to be trained using images in-the-wild that only have ground truth 2D annotations.

3D Hand Pose Estimation 3D Human Shape Estimation +4

Towards Accurate Markerless Human Shape and Pose Estimation over Time

no code implementations24 Jul 2017 Yinghao Huang, Federica Bogo, Christoph Lassner, Angjoo Kanazawa, Peter V. Gehler, Ijaz Akhter, Michael J. Black

Existing marker-less motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, which narrows its application scenarios.

Pose Estimation

Dynamic FAUST: Registering Human Bodies in Motion

no code implementations CVPR 2017 Federica Bogo, Javier Romero, Gerard Pons-Moll, Michael J. Black

We propose a new mesh registration method that uses both 3D geometry and texture information to register all scans in a sequence to a common reference topology.

Semantic Multi-View Stereo: Jointly Estimating Objects and Voxels

no code implementations CVPR 2017 Ali Osman Ulusoy, Michael J. Black, Andreas Geiger

Due to its probabilistic nature, the approach is able to cope with the approximate geometry of the 3D models as well as input shapes that are not present in the scene.

3D Reconstruction

On human motion prediction using recurrent neural networks

8 code implementations CVPR 2017 Julieta Martinez, Michael J. Black, Javier Romero

Human motion modelling is a classical problem at the intersection of graphics and computer vision, with applications spanning human-computer interaction, motion synthesis, and motion prediction for virtual and augmented reality.

Human motion prediction Human Pose Forecasting +3

Optical Flow in Mostly Rigid Scenes

no code implementations CVPR 2017 Jonas Wulff, Laura Sevilla-Lara, Michael J. Black

Existing algorithms typically focus on either recovering motion and structure under the assumption of a purely static world or optical flow for general unconstrained scenes.

Motion Estimation Optical Flow Estimation

Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

no code implementations23 Mar 2017 Timo von Marcard, Bodo Rosenhahn, Michael J. Black, Gerard Pons-Moll

We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body.

3D Human Pose Estimation

Unite the People: Closing the Loop Between 3D and 2D Human Representations

2 code implementations CVPR 2017 Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, Peter V. Gehler

With a comprehensive set of experiments, we show how this data can be used to train discriminative models that produce results with an unprecedented level of detail: our models predict 31 segments and 91 landmark locations on the body.

 Ranked #1 on Monocular 3D Human Pose Estimation on Human3.6M (Use Video Sequence metric)

3D human pose and shape estimation Monocular 3D Human Pose Estimation

Learning from Synthetic Humans

2 code implementations CVPR 2017 Gül Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, Cordelia Schmid

In this work we present SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data.

2D Human Pose Estimation 3D Human Pose Estimation +1

3D Menagerie: Modeling the 3D shape and pose of animals

no code implementations CVPR 2017 Silvia Zuffi, Angjoo Kanazawa, David Jacobs, Michael J. Black

The best human body models are learned from thousands of 3D scans of people in specific poses, which is infeasible with live animals.

Intrinsic Depth: Improving Depth Transfer With Intrinsic Images

no code implementations ICCV 2015 Naejin Kong, Michael J. Black

In contrast to raw RGB values, albedo and shading provide a richer, more physical, foundation for depth transfer.

Depth Estimation Optical Flow Estimation

Detailed Full-Body Reconstructions of Moving People From Monocular RGB-D Sequences

no code implementations ICCV 2015 Federica Bogo, Michael J. Black, Matthew Loper, Javier Romero

The method then uses geometry and image texture over time to obtain accurate shape, pose, and appearance information despite unconstrained motion, partial views, varying resolution, occlusion, and soft tissue deformation.

Efficient Sparse-to-Dense Optical Flow Estimation Using a Learned Basis and Layers

no code implementations CVPR 2015 Jonas Wulff, Michael J. Black

Given a set of sparse matches, we regress to dense optical flow using a learned set of full-frame basis flow fields.

Optical Flow Estimation

Pose-Conditioned Joint Angle Limits for 3D Human Pose Reconstruction

no code implementations CVPR 2015 Ijaz Akhter, Michael J. Black

Second, we define a general parametrization of body pose and a new, multi-stage, method to estimate 3D pose from 2D joint locations using an over-complete dictionary of poses.

Ranked #98 on 3D Human Pose Estimation on Human3.6M (PA-MPJPE metric)

3D Human Pose Estimation 3D Pose Estimation

Model Transport: Towards Scalable Transfer Learning on Manifolds

no code implementations CVPR 2014 Oren Freifeld, Soren Hauberg, Michael J. Black

We demonstrate the approach by transferring PCA and logistic-regression models of real-world data involving 3D shapes and image descriptors.

regression Transfer Learning

FAUST: Dataset and Evaluation for 3D Mesh Registration

no code implementations CVPR 2014 Federica Bogo, Javier Romero, Matthew Loper, Michael J. Black

We address this with a novel mesh registration technique that combines 3D shape and appearance information to produce high-quality alignments.


Grassmann Averages for Scalable Robust PCA

no code implementations CVPR 2014 Soren Hauberg, Aasa Feragen, Michael J. Black

We exploit that averages can be made robust to formulate the Robust Grassmann Average (RGA) as a form of robust PCA.

Dimensionality Reduction Shadow Removal +1

From Deformations to Parts: Motion-based Segmentation of 3D Objects

2 code implementations NeurIPS 2012 Soumya Ghosh, Matthew Loper, Erik B. Sudderth, Michael J. Black

We develop a method for discovering the parts of an articulated object from aligned meshes capturing various three-dimensional (3D) poses.

A Geometric take on Metric Learning

no code implementations NeurIPS 2012 Søren Hauberg, Oren Freifeld, Michael J. Black

We then show that this structure gives us a principled way to perform dimensionality reduction and regression according to the learned metrics.

Dimensionality Reduction Metric Learning +1

Synergies in learning words and their referents

no code implementations NeurIPS 2010 Mark Johnson, Katherine Demuth, Bevan Jones, Michael J. Black

This paper presents Bayesian non-parametric models that simultaneously learn to segment words from phoneme strings and learn the referents of some of those words, and shows that there is a synergistic interaction in the acquisition of these two kinds of linguistic information.

Language Acquisition Topic Models

An Additive Latent Feature Model for Transparent Object Recognition

no code implementations NeurIPS 2009 Mario Fritz, Gary Bradski, Sergey Karayev, Trevor Darrell, Michael J. Black

The appearance of a transparent patch is determined in part by the refraction of a background pattern through a transparent medium: the energy from the background usually dominates the patch appearance.

Object Recognition Quantization +1

Cannot find the paper you are looking for? You can Submit a new open access paper.