Search Results for author: Michael J. Black

Found 86 papers, 42 papers with code

SOMA: Solving Optical Marker-Based MoCap Automatically

2 code implementations ICCV 2021 Nima Ghorbani, Michael J. Black

Commercial auto-labeling tools require a specific calibration procedure at capture time, which is not possible for archival data.

Motion Capture

Learning to Regress Bodies from Images using Differentiable Semantic Rendering

1 code implementation ICCV 2021 Sai Kumar Dwivedi, Nikos Athanasiou, Muhammed Kocabas, Michael J. Black

For Minimally-Clothed regions, we define the DSR-MC loss, which encourages a tight match between a rendered SMPL body and the minimally-clothed regions of the image.

Ranked #7 on 3D Human Pose Estimation on 3DPW (using extra training data)

3D Human Pose Estimation

SPEC: Seeing People in the Wild with an Estimated Camera

1 code implementation ICCV 2021 Muhammed Kocabas, Chun-Hao P. Huang, Joachim Tesch, Lea Müller, Otmar Hilliges, Michael J. Black

We then train a novel network that concatenates the camera calibration to the image features and uses these together to regress 3D body shape and pose.

The Power of Points for Modeling Humans in Clothing

no code implementations ICCV 2021 Qianli Ma, Jinlong Yang, Siyu Tang, Michael J. Black

The geometry feature can be optimized to fit a previously unseen scan of a person in clothing, enabling the scan to be reposed realistically.

Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation

no code implementations1 Jul 2021 Zicong Fan, Adrian Spurr, Muhammed Kocabas, Siyu Tang, Michael J. Black, Otmar Hilliges

In this paper we demonstrate that self-similarity, and the resulting ambiguities in assigning pixel observations to the respective hands and their parts, is a major cause of the final 3D pose error.

Hand Pose Estimation

AGORA: Avatars in Geography Optimized for Regression Analysis

1 code implementation CVPR 2021 Priyanka Patel, Chun-Hao P. Huang, Joachim Tesch, David T. Hoffmann, Shashank Tripathi, Michael J. Black

Additionally, we fine-tune methods on AGORA and show improved performance on both AGORA and 3DPW, confirming the realism of the dataset.

3D Human Pose Estimation

PARE: Part Attention Regressor for 3D Human Body Estimation

1 code implementation ICCV 2021 Muhammed Kocabas, Chun-Hao P. Huang, Otmar Hilliges, Michael J. Black

Despite significant progress, we show that state of the art 3D human pose and shape estimation methods remain sensitive to partial occlusion and can produce dramatically wrong predictions although much of the body is observable.

Ranked #2 on 3D Human Pose Estimation on 3DPW (using extra training data)

3D Human Pose Estimation

LEAP: Learning Articulated Occupancy of People

no code implementations CVPR 2021 Marko Mihajlovic, Yan Zhang, Michael J. Black, Siyu Tang

Substantial progress has been made on modeling rigid 3D objects using deep implicit representations.

Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

1 code implementation ICCV 2021 Mathis Petrovich, Michael J. Black, Gül Varol

By sampling from this latent space and querying a certain duration through a series of positional encodings, we synthesize variable-length motion sequences conditioned on a categorical action.

Action Recognition Denoising +2

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

1 code implementation ICCV 2021 Xu Chen, Yufeng Zheng, Michael J. Black, Otmar Hilliges, Andreas Geiger

However, this is problematic since the backward warp field is pose dependent and thus requires large amounts of data to learn.

On Self-Contact and Human Pose

no code implementations CVPR 2021 Lea Müller, Ahmed A. A. Osman, Siyu Tang, Chun-Hao P. Huang, Michael J. Black

Third, we develop a novel HPS optimization method, SMPLify-XMC, that includes contact constraints and uses the known 3DCP body pose during fitting to create near ground-truth poses for MTP images.

Pose Estimation

SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks

no code implementations CVPR 2021 Shunsuke Saito, Jinlong Yang, Qianli Ma, Michael J. Black

We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar.

Populating 3D Scenes by Learning Human-Scene Interaction

1 code implementation CVPR 2021 Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, Michael J. Black

Second, we show that POSA's learned representation of body-scene interaction supports monocular human pose estimation that is consistent with a 3D scene, improving on the state of the art.

Pose Estimation

We are More than Our Joints: Predicting how 3D Bodies Move

no code implementations CVPR 2021 Yan Zhang, Michael J. Black, Siyu Tang

We note that motion prediction methods accumulate errors over time, resulting in joints or markers that diverge from true human bodies.

Human motion prediction Motion Capture +2

Monocular, One-stage, Regression of Multiple 3D People

2 code implementations ICCV 2021 Yu Sun, Qian Bao, Wu Liu, Yili Fu, Michael J. Black, Tao Mei

Through a body-center-guided sampling process, the body mesh parameters of all people in the image are easily extracted from the Mesh Parameter map.

3D Multi-Person Mesh Recovery

GRAB: A Dataset of Whole-Body Human Grasping of Objects

2 code implementations ECCV 2020 Omid Taheri, Nima Ghorbani, Michael J. Black, Dimitrios Tzionas

Training computers to understand, model, and synthesize human grasping requires a rich dataset containing complex 3D object shapes, detailed contact information, hand pose and shape, and the 3D body motion over time.

Grasp Contact Prediction Grasp Generation +1

STAR: Sparse Trained Articulated Human Body Regressor

1 code implementation ECCV 2020 Ahmed A. A. Osman, Timo Bolkart, Michael J. Black

The SMPL body model is widely used for the estimation, synthesis, and analysis of 3D human pose and shape.

SMPLpix: Neural Avatars from 3D Human Models

no code implementations16 Aug 2020 Sergey Prokudin, Michael J. Black, Javier Romero

Recent advances in deep generative models have led to an unprecedented level of realism for synthetically generated images of humans.

PLACE: Proximity Learning of Articulation and Contact in 3D Environments

1 code implementation12 Aug 2020 Siwei Zhang, Yan Zhang, Qianli Ma, Michael J. Black, Siyu Tang

To synthesize realistic human-scene interactions, it is essential to effectively represent the physical contact and proximity between the body and the world.

Perpetual Motion: Generating Unbounded Human Motion

no code implementations27 Jul 2020 Yan Zhang, Michael J. Black, Siyu Tang

To address this problem, we propose a model to generate non-deterministic, \textit{ever-changing}, perpetual human motion, in which the global trajectory and the body pose are cross-conditioned.

Motion Estimation Time Series

AirCapRL: Autonomous Aerial Human Motion Capture using Deep Reinforcement Learning

no code implementations13 Jul 2020 Rahul Tallamraju, Nitin Saini, Elia Bonetto, Michael Pabst, Yu Tang Liu, Michael J. Black, Aamir Ahmad

We focus on vision-based MoCap, where the objective is to estimate the trajectory of body pose and shape of a single moving person using multiple micro aerial vehicles.

Decision Making Motion Capture

Generating 3D People in Scenes without People

3 code implementations CVPR 2020 Yan Zhang, Mohamed Hassan, Heiko Neumann, Michael J. Black, Siyu Tang

However, this is a challenging task for a computer as solving it requires that (1) the generated human bodies to be semantically plausible within the 3D environment (e. g. people sitting on the sofa or cooking near the stove), and (2) the generated human-scene interaction to be physically feasible such that the human body and scene do not interpenetrate while, at the same time, body-scene contact supports physical interactions.

Pose Estimation

Learning Multi-Human Optical Flow

2 code implementations24 Oct 2019 Anurag Ranjan, David T. Hoffmann, Dimitrios Tzionas, Siyu Tang, Javier Romero, Michael J. Black

Therefore, we develop a dataset of multi-human optical flow and train optical flow networks on this dataset.

Motion Capture Optical Flow Estimation

Attacking Optical Flow

1 code implementation ICCV 2019 Anurag Ranjan, Joel Janai, Andreas Geiger, Michael J. Black

In this paper, we extend adversarial patch attacks to optical flow networks and show that such attacks can compromise their performance.

Optical Flow Estimation Self-Driving Cars

Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop

1 code implementation ICCV 2019 Nikos Kolotouros, Georgios Pavlakos, Michael J. Black, Kostas Daniilidis

Our approach is self-improving by nature, since better network estimates can lead the optimization to better solutions, while more accurate optimization fits provide better supervision for the network.

3D Human Pose Estimation

Resolving 3D Human Pose Ambiguities with 3D Scene Constraints

1 code implementation ICCV 2019 Mohamed Hassan, Vasileios Choutas, Dimitrios Tzionas, Michael J. Black

To motivate this, we show that current 3D human pose estimation methods produce results that are not consistent with the 3D scene.

3D Human Pose Estimation Motion Capture

Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"

1 code implementation ICCV 2019 Silvia Zuffi, Angjoo Kanazawa, Tanya Berger-Wolf, Michael J. Black

In contrast to research on human pose, shape and texture estimation, training data for endangered species is limited, the animals are in complex natural scenes with occlusion, they are naturally camouflaged, travel in herds, and look similar to each other.

Pose Estimation Texture Synthesis

Capture, Learning, and Synthesis of 3D Speaking Styles

1 code implementation CVPR 2019 Daniel Cudeiro, Timo Bolkart, Cassidy Laidlaw, Anurag Ranjan, Michael J. Black

To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers.

3D Face Animation Talking Face Generation +1

AMASS: Archive of Motion Capture as Surface Shapes

2 code implementations ICCV 2019 Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, Michael J. Black

We achieve this using a new method, MoSh++, that converts mocap data into realistic 3D human meshes represented by a rigged body model; here we use SMPL [doi:10. 1145/2816795. 2818013], which is widely used and provides a standard skeletal representation as well as a fully rigged surface mesh.

Motion Capture

Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences

no code implementations17 Oct 2018 Nikolas Hesse, Sergi Pujades, Michael J. Black, Michael Arens, Ulrich G. Hofmann, A. Sebastian Schroeder

To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants.

Recovering Accurate 3D Human Pose in The Wild Using IMUs and a Moving Camera

no code implementations ECCV 2018 Timo von Marcard, Roberto Henschel, Michael J. Black, Bodo Rosenhahn, Gerard Pons-Moll

In this work, we propose a method that combines a single hand-held camera and a set of Inertial Measurement Units (IMUs) attached at the body limbs to estimate accurate 3D poses in the wild.

3D Pose Estimation

Generating 3D faces using Convolutional Mesh Autoencoders

2 code implementations ECCV 2018 Anurag Ranjan, Timo Bolkart, Soubhik Sanyal, Michael J. Black

To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface.


Learning Human Optical Flow

1 code implementation14 Jun 2018 Anurag Ranjan, Javier Romero, Michael J. Black

Given this, we devise an optical flow algorithm specifically for human motion and show that it is superior to generic flow methods.

Motion Capture Optical Flow Estimation

Lions and Tigers and Bears: Capturing Non-Rigid, 3D, Articulated Shape From Images

no code implementations CVPR 2018 Silvia Zuffi, Angjoo Kanazawa, Michael J. Black

Animals are widespread in nature and the analysis of their shape and motion is important in many fields and industries.

Resisting Adversarial Attacks using Gaussian Mixture Variational Autoencoders

no code implementations31 May 2018 Partha Ghosh, Arpan Losalka, Michael J. Black

Our model has the form of a variational autoencoder, with a Gaussian mixture prior on the latent vector.

Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation

1 code implementation CVPR 2019 Anurag Ranjan, Varun Jampani, Lukas Balles, Kihwan Kim, Deqing Sun, Jonas Wulff, Michael J. Black

We address the unsupervised learning of several interconnected problems in low-level vision: single view depth prediction, camera motion estimation, optical flow, and segmentation of a video into the static scene and moving regions.

Monocular Depth Estimation Motion Estimation +2

On the Integration of Optical Flow and Action Recognition

no code implementations22 Dec 2017 Laura Sevilla-Lara, Yiyi Liao, Fatma Guney, Varun Jampani, Andreas Geiger, Michael J. Black

Here we take a deeper look at the combination of flow and action recognition, and investigate why optical flow is helpful, what makes a flow method good for action recognition, and how we can make it better.

Action Recognition Optical Flow Estimation

End-to-end Recovery of Human Shape and Pose

5 code implementations CVPR 2018 Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik

The main objective is to minimize the reprojection loss of keypoints, which allow our model to be trained using images in-the-wild that only have ground truth 2D annotations.

Monocular 3D Human Pose Estimation Weakly-supervised 3D Human Pose Estimation

Towards Accurate Markerless Human Shape and Pose Estimation over Time

no code implementations24 Jul 2017 Yinghao Huang, Federica Bogo, Christoph Lassner, Angjoo Kanazawa, Peter V. Gehler, Ijaz Akhter, Michael J. Black

Existing marker-less motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, which narrows its application scenarios.

Motion Capture Pose Estimation

Semantic Multi-View Stereo: Jointly Estimating Objects and Voxels

no code implementations CVPR 2017 Ali Osman Ulusoy, Michael J. Black, Andreas Geiger

Due to its probabilistic nature, the approach is able to cope with the approximate geometry of the 3D models as well as input shapes that are not present in the scene.

3D Reconstruction

Dynamic FAUST: Registering Human Bodies in Motion

no code implementations CVPR 2017 Federica Bogo, Javier Romero, Gerard Pons-Moll, Michael J. Black

We propose a new mesh registration method that uses both 3D geometry and texture information to register all scans in a sequence to a common reference topology.

On human motion prediction using recurrent neural networks

6 code implementations CVPR 2017 Julieta Martinez, Michael J. Black, Javier Romero

Human motion modelling is a classical problem at the intersection of graphics and computer vision, with applications spanning human-computer interaction, motion synthesis, and motion prediction for virtual and augmented reality.

Human motion prediction Motion Estimation +2

Optical Flow in Mostly Rigid Scenes

no code implementations CVPR 2017 Jonas Wulff, Laura Sevilla-Lara, Michael J. Black

Existing algorithms typically focus on either recovering motion and structure under the assumption of a purely static world or optical flow for general unconstrained scenes.

Motion Estimation Optical Flow Estimation

Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

no code implementations23 Mar 2017 Timo von Marcard, Bodo Rosenhahn, Michael J. Black, Gerard Pons-Moll

We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body.

3D Human Pose Estimation Motion Capture

Unite the People: Closing the Loop Between 3D and 2D Human Representations

2 code implementations CVPR 2017 Christoph Lassner, Javier Romero, Martin Kiefel, Federica Bogo, Michael J. Black, Peter V. Gehler

With a comprehensive set of experiments, we show how this data can be used to train discriminative models that produce results with an unprecedented level of detail: our models predict 31 segments and 91 landmark locations on the body.

Monocular 3D Human Pose Estimation

Learning from Synthetic Humans

2 code implementations CVPR 2017 Gül Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, Cordelia Schmid

In this work we present SURREAL (Synthetic hUmans foR REAL tasks): a new large-scale dataset with synthetically-generated but realistic images of people rendered from 3D sequences of human motion capture data.

3D Human Pose Estimation Human Part Segmentation +1

3D Menagerie: Modeling the 3D shape and pose of animals

no code implementations CVPR 2017 Silvia Zuffi, Angjoo Kanazawa, David Jacobs, Michael J. Black

The best human body models are learned from thousands of 3D scans of people in specific poses, which is infeasible with live animals.

Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

2 code implementations27 Jul 2016 Federica Bogo, Angjoo Kanazawa, Christoph Lassner, Peter Gehler, Javier Romero, Michael J. Black

We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints.

Detailed Full-Body Reconstructions of Moving People From Monocular RGB-D Sequences

no code implementations ICCV 2015 Federica Bogo, Michael J. Black, Matthew Loper, Javier Romero

The method then uses geometry and image texture over time to obtain accurate shape, pose, and appearance information despite unconstrained motion, partial views, varying resolution, occlusion, and soft tissue deformation.

Pose-Conditioned Joint Angle Limits for 3D Human Pose Reconstruction

no code implementations CVPR 2015 Ijaz Akhter, Michael J. Black

Second, we define a general parametrization of body pose and a new, multi-stage, method to estimate 3D pose from 2D joint locations using an over-complete dictionary of poses.

3D Pose Estimation Motion Capture

Efficient Sparse-to-Dense Optical Flow Estimation Using a Learned Basis and Layers

no code implementations CVPR 2015 Jonas Wulff, Michael J. Black

Given a set of sparse matches, we regress to dense optical flow using a learned set of full-frame basis flow fields.

Optical Flow Estimation

Model Transport: Towards Scalable Transfer Learning on Manifolds

no code implementations CVPR 2014 Oren Freifeld, Soren Hauberg, Michael J. Black

We demonstrate the approach by transferring PCA and logistic-regression models of real-world data involving 3D shapes and image descriptors.

Transfer Learning

FAUST: Dataset and Evaluation for 3D Mesh Registration

no code implementations CVPR 2014 Federica Bogo, Javier Romero, Matthew Loper, Michael J. Black

We address this with a novel mesh registration technique that combines 3D shape and appearance information to produce high-quality alignments.

Grassmann Averages for Scalable Robust PCA

no code implementations CVPR 2014 Soren Hauberg, Aasa Feragen, Michael J. Black

We exploit that averages can be made robust to formulate the Robust Grassmann Average (RGA) as a form of robust PCA.

Dimensionality Reduction Shadow Removal +1

From Deformations to Parts: Motion-based Segmentation of 3D Objects

2 code implementations NeurIPS 2012 Soumya Ghosh, Matthew Loper, Erik B. Sudderth, Michael J. Black

We develop a method for discovering the parts of an articulated object from aligned meshes capturing various three-dimensional (3D) poses.

A Geometric take on Metric Learning

no code implementations NeurIPS 2012 Søren Hauberg, Oren Freifeld, Michael J. Black

We then show that this structure gives us a principled way to perform dimensionality reduction and regression according to the learned metrics.

Dimensionality Reduction Metric Learning

Synergies in learning words and their referents

no code implementations NeurIPS 2010 Mark Johnson, Katherine Demuth, Bevan Jones, Michael J. Black

This paper presents Bayesian non-parametric models that simultaneously learn to segment words from phoneme strings and learn the referents of some of those words, and shows that there is a synergistic interaction in the acquisition of these two kinds of linguistic information.

Language Acquisition Topic Models

Training Factor Graphs with Reinforcement Learning for Efficient MAP Inference

no code implementations NeurIPS 2009 Khashayar Rohanimanesh, Sameer Singh, Andrew McCallum, Michael J. Black

Large, relational factor graphs with structure defined by first-order logic or other languages give rise to notoriously difficult inference problems.

An Additive Latent Feature Model for Transparent Object Recognition

no code implementations NeurIPS 2009 Mario Fritz, Gary Bradski, Sergey Karayev, Trevor Darrell, Michael J. Black

The appearance of a transparent patch is determined in part by the refraction of a background pattern through a transparent medium: the energy from the background usually dominates the patch appearance.

Object Recognition Quantization

Cannot find the paper you are looking for? You can Submit a new open access paper.