Search Results for author:
Found 76 papers, 26 papers with code
Morphological Symmetries in Robotics
We present a comprehensive framework for studying and leveraging morphological symmetries in robotic systems.
VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent Space
Instead of predicting body model parameters or 3D vertex coordinates, our focus is on forecasting the proposed discrete latent representation, which can be decoded into a registered human mesh.
Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields
To quantify the uncertainty on the learned surface, we model a stochastic radiance field.
Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction
Recent advancements in learning techniques that employ coordinate-based neural representations have yielded remarkable results in multi-view 3D reconstruction tasks.
PoseFix: Correcting 3D Human Poses with Natural Language
Automatically producing instructions to modify one's posture could open the door to endless applications, such as personalized coaching and in-home physical therapy.
InstantAvatar: Efficient 3D Head Reconstruction via Surface Rendering
In order to speed up the reconstruction process, we propose a system that combines, for the first time, a voxel-grid neural field representation with a surface renderer.
On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis
We present a comprehensive study on discrete morphological symmetries of dynamical systems, which are commonly observed in biological and artificial locomoting systems, such as legged, swimming, and flying animals/robots/virtual characters.
Visual Semantic Relatedness Dataset for Image Captioning
Modern image captioning system relies heavily on extracting knowledge from images to capture the concept of a static story.
NeRFLight: Fast and Light Neural Radiance Fields Using a Shared Feature Grid
While original Neural Radiance Fields (NeRF) have shown impressive results in modeling the appearance of a scene with compact MLP architectures, they are not able to achieve real-time rendering.
PoseScript: Linking 3D Human Poses and Natural Language
Thirdly, we present a learned process for generating pose descriptions.
Belief Revision based Caption Re-ranker with Visual Semantic Information
In this work, we focus on improving the captions generated by image-caption generation systems.
SIRA: Relightable Avatars from a Single Image
Our key ingredients are two data-driven statistical models based on neural fields that resolve the ambiguities of single-view 3D surface reconstruction and appearance factorization.
Topic Detection in Continuous Sign Language Videos
Significant progress has been made recently on challenging tasks in automatic sign language understanding, such as sign language recognition, translation and production.
Back to MLP: A Simple Baseline for Human Motion Prediction
This paper tackles the problem of human motion prediction, consisting in forecasting future body poses from historically observed sequences.
Learned Vertex Descent: A New Direction for 3D Human Model Fitting
An exhaustive evaluation demonstrates that our approach is able to capture the underlying body of clothed people with very different body shapes, achieving a significant improvement compared to state-of-the-art.
Single-view 3D Body and Cloth Reconstruction under Complex Poses
We, therefore, propose a coarse-to-fine approach in which we first learn an implicit function that maps the input image to a 3D body shape with a low level of detail, but which correctly fits the underlying human pose, despite its complexity.
Ranked #1 on
3D Reconstruction
on 3DPeople
Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation
For this purpose, we build a residual-like permutation-invariant network that successfully refines potentially corrupted initial 3D poses estimated by an off-the-shelf detector.
3D Multi-Person Pose Estimation (absolute)
3D Multi-Person Pose Estimation (root-relative)
+1
LISA: Learning Implicit Shape and Appearance of Hands
This paper proposes a do-it-all neural model of human hands, named LISA.
HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE
Studies on the automatic processing of 3D human pose data have flourished in the recent past.
Conditional-Flow NeRF: Accurate 3D Modelling with Reliable Uncertainty Quantification
For this purpose, our method learns a distribution over all possible radiance fields modelling which is used to quantify the uncertainty associated with the modelled scene.
Enhancing Egocentric 3D Pose Estimation with Third Person Views
In this paper, we propose a novel approach to enhance the 3D body pose estimation of a person computed from videos captured from a single wearable camera.
PhysXNet: A Customizable Approach for LearningCloth Dynamics on Dressed People
We introduce PhysXNet, a learning-based approach to predict the dynamics of deformable clothes given 3D skeleton motion sequences of humans wearing these clothes.
Body Size and Depth Disambiguation in Multi-Person Reconstruction from Single Images
We address the problem of multi-person 3D body pose and shape estimation from a single image.
An Adaptable Approach to Learn Realistic Legged Locomotion without Examples
We present experimental results showing that even in a model-free setup and with a simple reactive control architecture, the learned policies can generate realistic and energy-efficient locomotion gaits for a bipedal and a quadrupedal robot.
Grasp-Oriented Fine-grained Cloth Segmentation without Real Supervision
We show that training our network solely with synthetic data and the proposed DA yields results competitive with models trained on real data.
FaceDet3D: Facial Expressions with 3D Geometric Detail Hallucination
Facial Expressions induce a variety of high-level details on the 3D face geometry.
Stochastic Neural Radiance Fields: Quantifying Uncertainty in Implicit 3D Representations
In this context, we propose Stochastic Neural Radiance Fields (S-NeRF), a generalization of standard NeRF that learns a probability distribution over all the possible radiance fields modeling the scene.
SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery
We present SIDER(Single-Image neural optimization for facial geometric DEtail Recovery), a novel photometric optimization method that recovers detailed facial geometry from a single image in an unsupervised manner.
H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction
In this paper, we tackle these limitations for the specific problem of few-shot full 3D head reconstruction, by endowing coordinate-based representations with a probabilistic shape prior that enables faster convergence and better generalization when using few input images (down to three).
Uncertainty-Aware Camera Pose Estimation from Points and Lines
The new PnP(L) methods outperform the state-of-the-art on real data in isolation, showing an increase in mean translation accuracy by 18% on a representative subset of KITTI, while the new uncertain refinement improves pose accuracy for most of the solvers, e. g. decreasing mean translation error for the EPnP by 16% compared to the standard refinement on the same dataset.
Multi-Person Extreme Motion Prediction
In this paper, we explore this problem when dealing with humans performing collaborative tasks, we seek to predict the future motion of two interacted persons given two sequences of their past skeletons.
SMPLicit: Topology-aware Generative Model for Clothed People
In this paper we introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry.
3D Human Pose, Shape and Texture from Low-Resolution Images and Videos
Two common approaches to deal with low-resolution images are applying super-resolution techniques to the input, which may result in unpleasant artifacts, or simply training one model for each resolution, which is impractical in many realistic applications.
Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps
While there exists many methods for manipulating rigid objects with parallel-jaw grippers, grasping with multi-finger robotic hands remains a quite unexplored research topic.
FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction
The facial details are represented as a vertex displacement map and used then by a Neural Renderer to photo-realistically render novel images of any single image in any desired expression and view.
D-NeRF: Neural Radiance Fields for Dynamic Scenes
In this paper we introduce D-NeRF, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a \emph{single} camera moving around the scene.
PI-Net: Pose Interacting Network for Multi-Person Monocular 3D Pose Estimation
Our pose interacting network, or PI-Net, inputs the initial pose estimates of a variable number of interactees into a recurrent architecture used to refine the pose of the person-of-interest.
3D Multi-Person Pose Estimation (root-relative)
3D Pose Estimation
3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning
3D human shape and pose estimation from monocular images has been an active area of research in computer vision, having a substantial impact on the development of new applications, from activity recognition to creating virtual avatars.
Ranked #71 on
3D Human Pose Estimation
on MPI-INF-3DHP
Neural Cellular Automata Manifold
In biological terms, our approach would play the role of the transcription factors, modulating the mapping of genes into specific proteins that drive cellular differentiation, which occurs right before the morphogenesis.
Textual Visual Semantic Dataset for Text Spotting
In this paper, we propose a visual context dataset for Text Spotting in the wild, where the publicly available dataset COCO-text [Veit et al. 2016] has been extended with information about the scene (such as objects and places appearing in the image) to enable researchers to include semantic relations between texts and scene in their Text Spotting systems, and to offer a common framework for such approaches.
C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds
Flow-based generative models have highly desirable properties like exact log-likelihood evaluation and exact latent-variable inference, however they are still in their infancy and have not received as much attention as alternative generative models.
Improving Map Re-localization with Deep 'Movable' Objects Segmentation on 3D LiDAR Point Clouds
Results show that we are able to accurately re-locate over a filtered map, consistently reducing trajectory errors between an average of 35. 1% with respect to a non-filtered map version and of 47. 9% with respect to a standalone map created on the current session.
Semantic Relatedness Based Re-ranker for Text Spotting
We present a scenario where semantic similarity is not enough, and we devise a neural approach to learn semantic relatedness.
3DPeople: Modeling the Geometry of Dressed Humans
Finally, we design a multi-resolution deep generative network that, given an input image of a dressed human, predicts his/her geometry image (and thus the clothed body shape) in an end-to-end manner.
Context-aware Human Motion Prediction
The problem of predicting human motion given a sequence of past observations is at the core of many applications in robotics and computer vision.
Fast video object segmentation with Spatio-Temporal GANs
To achieve this, we concentrate all the heavy computational load to the training phase with two critics that enforce spatial and temporal mask consistency over the last K frames.
Human Motion Prediction via Spatio-Temporal Inpainting
First, we represent the data using a spatio-temporal tensor of 3D skeleton coordinates which allows formulating the prediction problem as an inpainting one, for which GANs work particularly well.
Visual Re-ranking with Natural Language Understanding for Text Spotting
We propose a post-processing approach to improve scene text recognition accuracy by using occurrence probabilities of words (unigram language model), and the semantic correlation between scene and text.
Visual Semantic Re-ranker for Text Spotting
In this paper, we propose a post-processing approach to improve the accuracy of text spotting by using the semantic relation between the text and the scene.
Unsupervised Person Image Synthesis in Arbitrary Poses
Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen.
Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View
We propose a method for predicting the 3D shape of a deformable surface from a single view.
Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous Vehicles
In this paper we propose a novel approach to estimate dense optical flow from sparse lidar data acquired on an autonomous vehicle.
Deep Lidar CNN to Understand the Dynamics of Moving Vehicles
Perception technologies in Autonomous Driving are experiencing their golden age due to the advances in Deep Learning.
Deconvolutional Networks for Point-Cloud Vehicle Detection and Tracking in Driving Scenarios
The system is thoroughly evaluated on the KITTI tracking dataset, and we show the performance boost provided by our CNN-based vehicle detector over a standard geometric approach.
Joint Coarse-And-Fine Reasoning for Deep Optical Flow
We propose a novel representation for dense pixel-wise estimation tasks using CNNs that boosts accuracy and reduces training time, by explicitly exploiting joint coarse-and-fine reasoning.
GANimation: Anatomically-aware Facial Animation from a Single Image
Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis.
Image Collection Pop-Up: 3D Reconstruction and Clustering of Rigid and Non-Rigid Categories
This paper introduces an approach to simultaneously estimate 3D shape, camera pose, and object and type of deformation clustering, from partial 2D annotations in a multi-instance collection of images.
3D CNNs on Distance Matrices for Human Action Recognition
In this paper we are interested in recognizing human actions from sequences of 3D skeleton data.
Ranked #90 on
Skeleton Based Action Recognition
on NTU RGB+D
DUST: Dual Union of Spatio-Temporal Subspaces for Monocular Multiple Object 3D Reconstruction
We present an approach to reconstruct the 3D shape of multiple deforming objects from incomplete 2D trajectories acquired by a single camera.
Multi-Modal Fashion Product Retrieval
Finding a product in the fashion world can be a daunting task.
The BreakingNews Dataset
We present BreakingNews, a novel dataset with approximately 100K news articles including images, text and captions, and enriched with heterogeneous meta-data (e. g. GPS coordinates and popularity metrics).
3D Human Pose Estimation from a Single Image via Distance Matrix Regression
We follow a standard two-step pipeline by first detecting the 2D position of the $N$ body joints, and then using these observations to infer 3D pose.
Ranked #19 on
3D Human Pose Estimation
on HumanEva-I
BreakingNews: Article Annotation by Image and Text Processing
Deep Canonical Correlation Analysis is deployed for article illustration, and a new loss function based on Great Circle Distance is proposed for geolocation.
Discriminative Learning of Deep Convolutional Feature Point Descriptors
Deep learning has revolutionalized image-level tasks such as classification, but patch-level tasks, such as correspondence, still rely on hand-crafted features, e. g. SIFT.
Ranked #2 on
Satellite Image Classification
on SAT-4
Learning Shape, Motion and Elastic Models in Force Space
In this paper, we address the problem of simultaneously recovering the 3D shape and pose of a deformable and potentially elastic object from 2D motion.
Structured Prediction with Output Embeddings for Semantic Image Annotation
We address the task of annotating images with semantic tuples.
Simultaneous Pose and Non-Rigid Shape With Particle Dynamics
In this paper, we propose a sequential solution to simultaneously estimate camera pose and non-rigid 3D shape from a monocular video.
TED: A Tolerant Edit Distance for Segmentation Evaluation
This measure, which we call Tolerant Edit Distance (TED), is motivated by two observations: (1) Some errors, like small boundary shifts, are tolerable in practice.
Neuroaesthetics in Fashion: Modeling the Perception of Fashionability
Importantly, our model is able to give rich feedback back to the user, conveying which garments or even scenery she/he should change in order to improve fashionability.
Fracking Deep Convolutional Image Descriptors
In this paper we propose a novel framework for learning local image descriptors in a discriminative manner.
Very Fast Solution to the PnP Problem with Algebraic Outlier Rejection
Given a set of 3D-to-2D matches, we formulate pose estimation problem as a low-rank homogeneous sys- tem where the solution lies on its 1D null space.
Segmentation-aware Deformable Part Models
In this work we propose a technique to combine bottom-up segmentation, coming in the form of SLIC superpixels, with sliding window detectors, such as Deformable Part Models (DPMs).
A Joint Model for 2D and 3D Pose Estimation from a Single Image
We introduce a novel approach to automatically recover 3D human pose from a single image.
Ranked #25 on
3D Human Pose Estimation
on HumanEva-I
Dense Segmentation-Aware Descriptors
In this work we exploit segmentation to construct appearance descriptors that can robustly deal with occlusion and background changes.
