Search Results for author: Otmar Hilliges

Found 95 papers, 43 papers with code

The DIDI dataset: Digital Ink Diagram data

2 code implementations20 Feb 2020 Philippe Gervais, Thomas Deselaers, Emre Aksan, Otmar Hilliges

We are releasing a dataset of diagram drawings with dynamic drawing information.

Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decomposition

1 code implementation CVPR 2023 Chen Guo, Tianjian Jiang, Xu Chen, Jie Song, Otmar Hilliges

Specifically, we define a temporally consistent human representation in canonical space and formulate a global optimization over the background model, the canonical human shape and texture, and per-frame human pose parameters.

3D Human Reconstruction Surface Reconstruction

I M Avatar: Implicit Morphable Head Avatars from Videos

1 code implementation CVPR 2022 Yufeng Zheng, Victoria Fernández Abrevaya, Marcel C. Bühler, Xu Chen, Michael J. Black, Otmar Hilliges

Traditional 3D morphable face models (3DMMs) provide fine-grained control over expression but cannot easily capture geometric and appearance details.

MORPH

PARE: Part Attention Regressor for 3D Human Body Estimation

1 code implementation ICCV 2021 Muhammed Kocabas, Chun-Hao P. Huang, Otmar Hilliges, Michael J. Black

Despite significant progress, we show that state of the art 3D human pose and shape estimation methods remain sensitive to partial occlusion and can produce dramatically wrong predictions although much of the body is observable.

3D human pose and shape estimation 3D Multi-Person Pose Estimation

X-Avatar: Expressive Human Avatars

1 code implementation CVPR 2023 Kaiyue Shen, Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Julien Valentin, Jie Song, Otmar Hilliges

Our method models bodies, hands, facial expressions and appearance in a holistic fashion and can be learned from either full 3D scans or RGB-D data.

3D Human Reconstruction

Few-Shot Adaptive Gaze Estimation

1 code implementation ICCV 2019 Seonwook Park, Shalini De Mello, Pavlo Molchanov, Umar Iqbal, Otmar Hilliges, Jan Kautz

Inter-personal anatomical differences limit the accuracy of person-independent gaze estimation networks.

 Ranked #1 on Gaze Estimation on MPII Gaze (using extra training data)

Gaze Estimation Meta-Learning

PointAvatar: Deformable Point-based Head Avatars from Videos

1 code implementation CVPR 2023 Yufeng Zheng, Wang Yifan, Gordon Wetzstein, Michael J. Black, Otmar Hilliges

The ability to create realistic, animatable and relightable head avatars from casual video sequences would open up wide ranging applications in communication and entertainment.

SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes

1 code implementation ICCV 2021 Xu Chen, Yufeng Zheng, Michael J. Black, Otmar Hilliges, Andreas Geiger

However, this is problematic since the backward warp field is pose dependent and thus requires large amounts of data to learn.

Fast-SNARF: A Fast Deformer for Articulated Neural Fields

1 code implementation28 Nov 2022 Xu Chen, Tianjian Jiang, Jie Song, Max Rietmann, Andreas Geiger, Michael J. Black, Otmar Hilliges

A key challenge in making such methods applicable to articulated objects, such as the human body, is to model the deformation of 3D locations between the rest pose (a canonical space) and the deformed space.

3D Reconstruction Computational Efficiency +1

ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation

1 code implementation CVPR 2023 Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J. Black, Otmar Hilliges

In part this is because there exist no datasets with ground-truth 3D annotations for the study of physically consistent and synchronised motion of hands and articulated objects.

3D Reconstruction Object

ETH-XGaze: A Large Scale Dataset for Gaze Estimation under Extreme Head Pose and Gaze Variation

1 code implementation ECCV 2020 Xucong Zhang, Seonwook Park, Thabo Beeler, Derek Bradley, Siyu Tang, Otmar Hilliges

We show that our dataset can significantly improve the robustness of gaze estimation methods across different head poses and gaze angles.

 Ranked #1 on Gaze Estimation on ETH-XGaze (using extra training data)

Gaze Estimation

HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics

1 code implementation CVPR 2023 Artur Grigorev, Bernhard Thomaszewski, Michael J. Black, Otmar Hilliges

We propose a method that leverages graph neural networks, multi-level message passing, and unsupervised training to enable real-time prediction of realistic clothing dynamics.

Photo-Realistic Monocular Gaze Redirection Using Generative Adversarial Networks

1 code implementation ICCV 2019 Zhe He, Adrian Spurr, Xucong Zhang, Otmar Hilliges

In this work, we present a novel method to alleviate this problem by leveraging generative adversarial training to synthesize an eye image conditioned on a target gaze direction.

Gaze Estimation gaze redirection

FLARE: Fast Learning of Animatable and Relightable Mesh Avatars

1 code implementation26 Oct 2023 Shrisha Bharadwaj, Yufeng Zheng, Otmar Hilliges, Michael J. Black, Victoria Fernandez-Abrevaya

Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems.

Cross-modal Deep Variational Hand Pose Estimation

1 code implementation CVPR 2018 Adrian Spurr, Jie Song, Seonwook Park, Otmar Hilliges

Furthermore, we show that our proposed method can be used without changes on depth images and performs comparably to specialized methods.

Hand Pose Estimation

Towards End-to-end Video-based Eye-Tracking

1 code implementation ECCV 2020 Seonwook Park, Emre Aksan, Xucong Zhang, Otmar Hilliges

Estimating eye-gaze from images alone is a challenging task, in large parts due to un-observable person-specific factors.

DeepWriting: Making Digital Ink Editable via Deep Generative Modeling

1 code implementation25 Jan 2018 Emre Aksan, Fabrizio Pece, Otmar Hilliges

Digital ink promises to combine the flexibility and aesthetics of handwriting and the ability to process, search and edit digital text.

Handwriting generation Handwritten Word Generation +1

EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild

1 code implementation ICCV 2023 Manuel Kaufmann, Jie Song, Chen Guo, Kaiyue Shen, Tianjian Jiang, Chengcheng Tang, Juan Zarate, Otmar Hilliges

EMDB is a novel dataset that contains high-quality 3D SMPL pose and shape parameters with global body and camera trajectories for in-the-wild videos.

Pose Estimation

Human Performance Capture from Monocular Video in the Wild

1 code implementation29 Nov 2021 Chen Guo, Xu Chen, Jie Song, Otmar Hilliges

In this work, we propose a method capable of capturing the dynamic 3D human shape from a monocular video featuring challenging body poses, without any additional input.

3D Human Shape Estimation Autonomous Driving

Self-Learning Transformations for Improving Gaze and Head Redirection

2 code implementations NeurIPS 2020 Yufeng Zheng, Seonwook Park, Xucong Zhang, Shalini De Mello, Otmar Hilliges

Furthermore, we show that in the presence of limited amounts of real-world training data, our method allows for improvements in the downstream task of semi-supervised cross-dataset gaze estimation.

Disentanglement Gaze Estimation +1

A Spatio-temporal Transformer for 3D Human Motion Prediction

1 code implementation18 Apr 2020 Emre Aksan, Manuel Kaufmann, Peng Cao, Otmar Hilliges

We propose a novel Transformer-based architecture for the task of generative modelling of 3D human motion.

Human motion prediction motion prediction

PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning

1 code implementation ICCV 2021 Adrian Spurr, Aneesh Dahiya, Xi Wang, Xucong Zhang, Otmar Hilliges

Encouraged by the success of contrastive learning on image classification tasks, we propose a new self-supervised method for the structured regression task of 3D hand pose estimation.

3D Hand Pose Estimation Contrastive Learning +3

D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactions

1 code implementation CVPR 2022 Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, Otmar Hilliges

We introduce the dynamic grasp synthesis task: given an object with a known 6D pose and a grasp reference, our goal is to generate motions that move the object to a target 6D pose.

Motion Synthesis Object

HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video

1 code implementation30 Nov 2023 Zicong Fan, Maria Parelli, Maria Eleni Kadoglou, Muhammed Kocabas, Xu Chen, Michael J. Black, Otmar Hilliges

Since humans interact with diverse objects every day, the holistic 3D capture of these interactions is important to understand and model human behaviour.

3D Reconstruction Object +1

STCN: Stochastic Temporal Convolutional Networks

1 code implementation ICLR 2019 Emre Aksan, Otmar Hilliges

Convolutional architectures have recently been shown to be competitive on many sequence modelling tasks when compared to the de-facto standard of recurrent neural networks (RNNs), while providing computational and modeling advantages due to inherent parallelism.

VariTex: Variational Neural Face Textures

1 code implementation ICCV 2021 Marcel C. Bühler, Abhimitra Meka, Gengyan Li, Thabo Beeler, Otmar Hilliges

In this paper, we propose VariTex - to the best of our knowledge the first method that learns a variational latent feature space of neural face textures, which allows sampling of novel identities.

Face Model

SAGA: Stochastic Whole-Body Grasping with Contact

1 code implementation19 Dec 2021 Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang, Otmar Hilliges, Fisher Yu, Siyu Tang

Given an initial pose and the generated whole-body grasping pose as the start and end of the motion respectively, we design a novel contact-aware generative motion infilling module to generate a diverse set of grasp-oriented motions.

Object

Monocular Neural Image Based Rendering with Continuous View Control

2 code implementations ICCV 2019 Xu Chen, Jie Song, Otmar Hilliges

The approach is self-supervised and only requires 2D images and associated view transforms for training.

Novel View Synthesis

Structured Prediction Helps 3D Human Motion Modelling

1 code implementation ICCV 2019 Emre Aksan, Manuel Kaufmann, Otmar Hilliges

This is implemented via a hierarchy of small-sized neural networks connected analogously to the kinematic chains in the human body as well as a joint-wise decomposition in the loss function.

Human motion prediction Motion Forecasting +2

CoSE: Compositional Stroke Embeddings

1 code implementation NeurIPS 2020 Emre Aksan, Thomas Deselaers, Andrea Tagliasacchi, Otmar Hilliges

We demonstrate qualitatively and quantitatively that our proposed approach is able to model the appearance of individual strokes, as well as the compositional structure of larger diagram drawings.

Content-Consistent Generation of Realistic Eyes with Style

1 code implementation8 Nov 2019 Marcel Bühler, Seonwook Park, Shalini De Mello, Xucong Zhang, Otmar Hilliges

Accurately labeled real-world training data can be scarce, and hence recent works adapt, modify or generate images to boost target datasets.

Semantic Segmentation

Guiding InfoGAN with Semi-Supervision

2 code implementations14 Jul 2017 Adrian Spurr, Emre Aksan, Otmar Hilliges

In this paper we propose a new semi-supervised GAN architecture (ss-InfoGAN) for image synthesis that leverages information from few labels (as little as 0. 22%, max.

Image Generation

Convolutional Autoencoders for Human Motion Infilling

1 code implementation22 Oct 2020 Manuel Kaufmann, Emre Aksan, Jie Song, Fabrizio Pece, Remo Ziegler, Otmar Hilliges

At the heart of our approach lies the idea to cast motion infilling as an inpainting problem and to train a convolutional de-noising autoencoder on image-like representations of motion sequences.

TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning

1 code implementation1 Sep 2022 Andrea Ziani, Zicong Fan, Muhammed Kocabas, Sammy Christen, Otmar Hilliges

We introduce TempCLR, a new time-coherent contrastive learning approach for the structured regression task of 3D hand reconstruction.

Contrastive Learning Hand Pose Estimation

Deep Pictorial Gaze Estimation

1 code implementation ECCV 2018 Seonwook Park, Adrian Spurr, Otmar Hilliges

In this paper, we introduce a novel deep neural network architecture specifically designed for the task of gaze estimation from single eye input.

Gaze Estimation

Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings

2 code implementations12 May 2018 Seonwook Park, Xucong Zhang, Andreas Bulling, Otmar Hilliges

Conventional feature-based and model-based gaze estimation methods have proven to perform well in settings with controlled illumination and specialized cameras.

Gaze Estimation

Unpaired Pose Guided Human Image Generation

1 code implementation8 Jan 2019 Xu Chen, Jie Song, Otmar Hilliges

This paper studies the task of full generative modelling of realistic images of humans, guided only by coarse sketch of the pose, while providing control over the specific instance or type of outfit worn by the user.

Image-to-Image Translation Translation

Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023

1 code implementation28 Jun 2023 Daoji Huang, Otmar Hilliges, Luc van Gool, Xi Wang

We present Palm, a solution to the Long-Term Action Anticipation (LTA) task utilizing vision-language and large language models.

Action Anticipation Image Captioning +3

Learning Human Motion Models for Long-term Predictions

no code implementations10 Apr 2017 Partha Ghosh, Jie Song, Emre Aksan, Otmar Hilliges

Furthermore, we propose new evaluation protocols to assess the quality of synthetic motion sequences even for which no ground truth data exists.

Plan3D: Viewpoint and Trajectory Optimization for Aerial Multi-View Stereo Reconstruction

no code implementations25 May 2017 Benjamin Hepp, Matthias Nießner, Otmar Hilliges

We introduce a new method that efficiently computes a set of viewpoints and trajectories for high-quality 3D reconstructions in outdoor environments.

End-to-end Learning for Graph Decomposition

no code implementations ICCV 2019 Jie Song, Bjoern Andres, Michael Black, Otmar Hilliges, Siyu Tang

The new optimization problem can be viewed as a Conditional Random Field (CRF) in which the random variables are associated with the binary edge labels of the initial graph and the hard constraints are introduced in the CRF as high-order potentials.

Clustering Multi-Person Pose Estimation

Demonstration-Guided Deep Reinforcement Learning of Control Policies for Dexterous Human-Robot Interaction

no code implementations27 Jun 2019 Sammy Christen, Stefan Stevsic, Otmar Hilliges

In this paper, we propose a method for training control policies for human-robot interactions such as handshakes or hand claps via Deep Reinforcement Learning.

reinforcement-learning Reinforcement Learning (RL)

Sample Efficient Learning of Path Following and Obstacle Avoidance Behavior for Quadrotors

no code implementations28 Jun 2019 Stefan Stevsic, Tobias Naegeli, Javier Alonso-Mora, Otmar Hilliges

This enables an easy to implement learning algorithm that is robust to errors of the model used in the model predictive controller.

Collision Avoidance Imitation Learning

Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning

no code implementations14 Feb 2020 Sammy Christen, Lukas Jendele, Emre Aksan, Otmar Hilliges

We present HiDe, a novel hierarchical reinforcement learning architecture that successfully solves long horizon control tasks and generalizes to unseen test scenarios.

Continuous Control Decision Making +2

Category Level Object Pose Estimation via Neural Analysis-by-Synthesis

no code implementations ECCV 2020 Xu Chen, Zijian Dong, Jie Song, Andreas Geiger, Otmar Hilliges

Many object pose estimation algorithms rely on the analysis-by-synthesis framework which requires explicit representations of individual object instances.

Image Generation Object +1

Spatial Attention Improves Iterative 6D Object Pose Estimation

no code implementations5 Jan 2021 Stefan Stevsic, Otmar Hilliges

Our main insight is that after the initial pose estimate, it is important to pay attention to distinct spatial features of the object in order to improve the estimation accuracy during alignment.

6D Pose Estimation 6D Pose Estimation using RGB +1

The Six Hug Commandments: Design and Evaluation of a Human-Sized Hugging Robot with Visual and Haptic Perception

no code implementations19 Jan 2021 Alexis E. Block, Sammy Christen, Roger Gassert, Otmar Hilliges, Katherine J. Kuchenbecker

We followed all six tenets to create a new robotic platform, HuggieBot 2. 0, that has a soft, warm, inflated body (HuggieChest) and uses visual and haptic sensing to deliver closed-loop hugging.

Robotics

Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimation

no code implementations10 Jun 2021 Adrian Spurr, Pavlo Molchanov, Umar Iqbal, Jan Kautz, Otmar Hilliges

Hand pose estimation is difficult due to different environmental conditions, object- and self-occlusion as well as diversity in hand shape and appearance.

Hand Pose Estimation valid

A Skeleton-Driven Neural Occupancy Representation for Articulated Hands

no code implementations23 Sep 2021 Korrawe Karunratanakul, Adrian Spurr, Zicong Fan, Otmar Hilliges, Siyu Tang

We present Hand ArticuLated Occupancy (HALO), a novel representation of articulated hands that bridges the advantages of 3D keypoints and neural implicit surfaces and can be used in end-to-end trainable architectures.

Render In-between: Motion Guided Video Synthesis for Action Interpolation

no code implementations1 Nov 2021 Hsuan-I Ho, Xu Chen, Jie Song, Otmar Hilliges

We propose to address these issues in a motion-guided frame-upsampling framework that is capable of producing realistic human motion and appearance.

Neural Rendering

Learning Functionally Decomposed Hierarchies for Continuous Navigation Tasks

no code implementations25 Sep 2019 Lukas Jendele, Sammy Christen, Emre Aksan, Otmar Hilliges

Hierarchical Reinforcement Learning (HRL) has held the promise to enhance the capabilities of RL agents via operation on different levels of temporal abstraction.

Continuous Control Decision Making +3

gDNA: Towards Generative Detailed Neural Avatars

no code implementations CVPR 2022 Xu Chen, Tianjian Jiang, Jie Song, Jinlong Yang, Michael J. Black, Andreas Geiger, Otmar Hilliges

Furthermore, we show that our method can be used on the task of fitting human models to raw scans, outperforming the previous state-of-the-art.

LiP-Flow: Learning Inference-time Priors for Codec Avatars via Normalizing Flows in Latent Space

no code implementations15 Mar 2022 Emre Aksan, Shugao Ma, Akin Caliskan, Stanislav Pidhorskyi, Alexander Richard, Shih-En Wei, Jason Saragih, Otmar Hilliges

To mitigate this asymmetry, we introduce a prior model that is conditioned on the runtime inputs and tie this prior space to the 3D face model via a normalizing flow in the latent space.

Face Model

SFP: State-free Priors for Exploration in Off-Policy Reinforcement Learning

no code implementations26 May 2022 Marco Bagatella, Sammy Christen, Otmar Hilliges

Several methods, such as behavioral priors, are able to leverage offline data in order to efficiently accelerate reinforcement learning on complex tasks.

Continuous Control Efficient Exploration +2

EyeNeRF: A Hybrid Representation for Photorealistic Synthesis, Animation and Relighting of Human Eyes

no code implementations16 Jun 2022 Gengyan Li, Abhimitra Meka, Franziska Müller, Marcel C. Bühler, Otmar Hilliges, Thabo Beeler

The challenge of synthesizing eyes is multifold as it requires 1) appropriate representations for the various components of the eye and the periocular region for coherent viewpoint synthesis, capable of representing diffuse, refractive and highly reflective surfaces, 2) disentangling skin and eye appearance from environmental illumination such that it may be rendered under novel lighting conditions, and 3) capturing eyeball motion and the deformation of the surrounding skin to enable re-gazing.

Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors

no code implementations6 Sep 2022 Xi Wang, Gen Li, Yen-Ling Kuo, Muhammed Kocabas, Emre Aksan, Otmar Hilliges

We further qualitatively evaluate the effectiveness of our method on real images and demonstrate its generalizability towards interaction types and object categories.

Human-Object Interaction Detection Object

Utilizing Synthetic Data in Supervised Learning for Robust 5-DoF Magnetic Marker Localization

no code implementations14 Nov 2022 Mengfan Wu, Thomas Langerak, Otmar Hilliges, Juan Zarate

However, traditionally, the tracking of magnetic markers is computationally expensive due to the requirement for iterative optimization procedures.

Position

HARP: Personalized Hand Reconstruction from a Monocular RGB Video

no code implementations CVPR 2023 Korrawe Karunratanakul, Sergey Prokudin, Otmar Hilliges, Siyu Tang

We present HARP (HAnd Reconstruction and Personalization), a personalized hand avatar creation approach that takes a short monocular RGB video of a human hand as input and reconstructs a faithful hand avatar exhibiting a high-fidelity appearance and geometry.

3D Hand Pose Estimation

InstantAvatar: Learning Avatars from Monocular Video in 60 Seconds

no code implementations CVPR 2023 Tianjian Jiang, Xu Chen, Jie Song, Otmar Hilliges

To achieve this efficiency we propose a carefully designed and engineered system, that leverages emerging acceleration structures for neural fields, in combination with an efficient empty space-skipping strategy for dynamic scenes.

Efficient Learning of High Level Plans from Play

no code implementations16 Mar 2023 Núria Armengol Urpí, Marco Bagatella, Otmar Hilliges, Georg Martius, Stelian Coros

Real-world robotic manipulation tasks remain an elusive challenge, since they involve both fine-grained environment interaction, as well as the ability to plan for long-horizon goals.

Motion Planning Reinforcement Learning (RL) +1

Hi4D: 4D Instance Segmentation of Close Human Interaction

no code implementations CVPR 2023 Yifei Yin, Chen Guo, Manuel Kaufmann, Juan Jose Zarate, Jie Song, Otmar Hilliges

We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact.

Instance Segmentation Semantic Segmentation

Human from Blur: Human Pose Tracking from Blurry Images

no code implementations ICCV 2023 Yiming Zhao, Denys Rozumnyi, Jie Song, Otmar Hilliges, Marc Pollefeys, Martin R. Oswald

The key idea is to tackle the inverse problem of image deblurring by modeling the forward problem with a 3D human model, a texture map, and a sequence of poses to describe human motion.

Deblurring Image Deblurring +2

Learning Human-to-Robot Handovers from Point Clouds

no code implementations CVPR 2023 Sammy Christen, Wei Yang, Claudia Pérez-D'Arpino, Otmar Hilliges, Dieter Fox, Yu-Wei Chao

We propose the first framework to learn control policies for vision-based human-to-robot handovers, a critical task for human-robot interaction.

Learning Locally Editable Virtual Humans

no code implementations CVPR 2023 Hsuan-I Ho, Lixin Xue, Jie Song, Otmar Hilliges

To this end, we construct a trainable feature codebook to store local geometry and texture features on the vertices of a deformable body model, thus exploiting its consistent topology under articulation.

AG3D: Learning to Generate 3D Avatars from 2D Image Collections

no code implementations ICCV 2023 Zijian Dong, Xu Chen, Jinlong Yang, Michael J. Black, Otmar Hilliges, Andreas Geiger

The key to progress is hence to learn generative models of 3D avatars from abundant unstructured 2D image collections.

EFE: End-to-end Frame-to-Gaze Estimation

no code implementations9 May 2023 Haldun Balim, Seonwook Park, Xi Wang, Xucong Zhang, Otmar Hilliges

In this paper, we propose a frame-to-gaze network that directly predicts both 3D gaze origin and 3D gaze direction from the raw frame out of the camera without any face or eye cropping.

Gaze Estimation

ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulation

no code implementations7 Sep 2023 HUI ZHANG, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, Otmar Hilliges

ArtiGrasp leverages reinforcement learning and physics simulations to train a policy that controls the global and local hand pose.

hand-object pose Object

Physically Plausible Full-Body Hand-Object Interaction Synthesis

no code implementations14 Sep 2023 Jona Braun, Sammy Christen, Muhammed Kocabas, Emre Aksan, Otmar Hilliges

Through a hierarchical framework, we first learn skill priors for both body and hand movements in a decoupled setting.

Human-Object Interaction Detection Object +1

PACE: Human and Camera Motion Estimation from in-the-wild Videos

no code implementations20 Oct 2023 Muhammed Kocabas, Ye Yuan, Pavlo Molchanov, Yunrong Guo, Michael J. Black, Otmar Hilliges, Jan Kautz, Umar Iqbal

This design combines the strengths of SLAM and motion priors, which leads to significant improvements in human and camera motion estimation.

Motion Estimation

CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling

no code implementations26 Oct 2023 Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, Romann M. Weber

While conditional diffusion models are known to have good coverage of the data distribution, they still face limitations in output diversity, particularly when sampled with a high classifier-free guidance scale for optimal image quality or when trained on small datasets.

Attribute Image Generation

SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers

no code implementations9 Nov 2023 Sammy Christen, Lan Feng, Wei Yang, Yu-Wei Chao, Otmar Hilliges, Jie Song

In this paper, we introduce a framework that can generate plausible human grasping motions suitable for training the robot.

SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion

no code implementations27 Nov 2023 Hsuan-I Ho, Jie Song, Otmar Hilliges

For the former, we employ a powerful generative diffusion model to hallucinate back appearances from the input images.

3D Human Reconstruction 3D Reconstruction +1

A Unified Approach for Text- and Image-guided 4D Scene Generation

no code implementations28 Nov 2023 Yufeng Zheng, Xueting Li, Koki Nagano, Sifei Liu, Karsten Kreis, Otmar Hilliges, Shalini De Mello

Large-scale diffusion generative models are greatly simplifying image, video and 3D asset creation from user-provided text prompts and images.

Scene Generation

LALM: Long-Term Action Anticipation with Language Models

no code implementations29 Nov 2023 Sanghwan Kim, Daoji Huang, Yongqin Xian, Otmar Hilliges, Luc van Gool, Xi Wang

Understanding human activity is a crucial yet intricate task in egocentric vision, a field that focuses on capturing visual perspectives from the camera wearer's viewpoint.

Action Anticipation Action Recognition +4

KinectFusion: Real-Time Dense Surface Mapping and Tracking

no code implementations ISMAR 2011 Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Andrew Fitzgibbon

We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware.

Cannot find the paper you are looking for? You can Submit a new open access paper.