Search Results for author:
Found 99 papers, 28 papers with code
Unsupervised Learning of Optical Flow with Deep Feature Similarity
Deep unsupervised learning for optical flow has been proposed, where the loss measures image similarity with the warping function parameterized by estimated flow.
Distance-Normalized Unified Representation for Monocular 3D Object Detection
Monocular 3D object detection plays an important role in autonomous driving and still remains challenging.
Prompt Augmentation for Self-supervised Text-guided Image Manipulation
Text-guided image editing finds applications in various creative and practical fields.
TDDSR: Single-Step Diffusion with Two Discriminators for Super Resolution
Super-resolution methods are increasingly becoming popular for both real-world and face-specific tasks.
Multi-hypotheses Conditioned Point Cloud Diffusion for 3D Human Reconstruction from Occluded Images
Compared to previous implicit-function-based methods, the point cloud diffusion model can capture the global consistent features to generate the occluded regions, and the denoising process corrects the misaligned SMPL meshes.
Recent advances in interpretable machine learning using structure-based protein representations
Recent advancements in machine learning (ML) are transforming the field of structural biology.
Hand-object reconstruction via interaction-aware graph attention mechanism
Estimating the poses of both a hand and an object has become an important area of research due to the growing need for advanced vision computing.
Semi-Supervised 3D Object Detection with Channel Augmentation using Transformation Equivariance
We explore a novel teacher-student framework employing channel augmentation for 3D semi-supervised object detection.
Dense Hand-Object(HO) GraspNet with Full Grasping Taxonomy and Dynamics
Accurate hand and object 3D meshes are obtained by fitting the hand parametric model (MANO) and the hand implicit function (HALO) to multi-view RGBD frames, with the MoCap system only for objects.
InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion
Sampling from our model yields plausible and diverse two-hand shapes in close interaction with or without an object.
BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Image
Creating personalized hand avatars is important to offer a realistic experience to users on AR / VR platforms.
Energy-based Domain-Adaptive Segmentation with Depth Guidance
Our framework incorporates two novel components: energy-based feature fusion (EB2F) and energy-based reliable fusion Assessment (RFA) modules.
PromptRR: Diffusion Models as Prompt Generators for Single Image Reflection Removal
For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts.
Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting
To this end, we propose a transformer-based 360 image outpainting framework called Dream360, which can generate diverse, high-fidelity, and high-resolution panoramas from user-selected viewports, considering the spherical properties of 360 images.
Deep Video Restoration for Under-Display Camera
With the pipeline, we build the first large-scale UDC video restoration dataset called PexelsUDC, which includes two subsets named PexelsUDC-T and PexelsUDC-P corresponding to different displays for UDC.
LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement
In this paper, we address this limitation by proposing a degradation-aware learning scheme for LLIE using diffusion models, which effectively integrates degradation and image priors into the diffusion process, resulting in improved image enhancement.
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions
Second, we introduce a residual dense transformer block (RDTB) as the final GridFormer layer.
iEdit: Localised Text-guided Image Editing with Weak Supervision
We propose a novel learning method for text-guided image editing, namely \texttt{iEdit}, that generates images conditioned on a source image and a textual edit prompt.
MAPConNet: Self-supervised 3D Pose Transfer with Mesh and Point Contrastive Learning
Unsupervised methods have been proposed for graph convolutional models but they require ground truth correspondence between the source and target inputs.
PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching
However, these methods are often inefficient and limited by their reliance on pre-trained models that have not be designed specifically for pose estimation.
Unsupervised Contour Tracking of Live Cells by Mechanical and Cycle Consistency Losses
For quantitative evaluation, we labeled sparse tracking points along the contour of live cells from two live cell datasets taken with phase contrast and confocal fluorescence microscopes.
Im2Hands: Learning Attentive Implicit Representation of Interacting Two-Hand Shapes
To handle the shape complexity and interaction context between two hands, Im2Hands models the occupancy volume of two hands - conditioned on an RGB image and coarse 3D keypoints - by two novel attention-based modules responsible for (1) initial occupancy estimation and (2) context-aware occupancy refinement, respectively.
MoBYv2AL: Self-supervised Active Learning for Image Classification
We present MoBYv2AL, a novel self-supervised active learning framework for image classification.
3D Distillation: Improving Self-Supervised Monocular Depth Estimation on Reflective Surfaces
Self-supervised monocular depth estimation (SSMDE) aims at predicting the dense depth maps of monocular images, by learning to minimize a photometric loss using spatially neighboring image pairs during training.
Semi-Supervised Object Detection with Object-wise Contrastive Learning and Regression Uncertainty
Unlike existing suboptimal methods, we propose a two-step pseudo-label filtering for the classification and regression heads in a teacher-student framework.
Ranked #14 on
Semi-Supervised Object Detection
on COCO 1% labeled data
CRT-6D: Fast 6D Object Pose Estimation with Cascaded Refinement Transformers
Learning based 6D object pose estimation methods rely on computing large intermediate pose representations and/or iteratively refining an initial estimation with a slow render-compare pipeline.
Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian
We propose a framework that can deform an object in a 2D image as it exists in 3D space.
Modular Adaptive Policy Selection for Multi-Task Imitation Learning through Task Division
This is done by using proto-policies as modules to divide the tasks into simple sub-behaviours that can be shared.
A Unified Architecture of Semantic Segmentation and Hierarchical Generative Adversarial Networks for Expression Manipulation
Recently, hierarchical networks that consist of both a global network dealing with the whole image and multiple local networks focusing on local parts are showing success.
EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation
To enable KD across the unpaired modalities, we first propose a bidirectional modality reconstruction (BMR) module to bridge both modalities and simultaneously exploit them to distill knowledge via the crafted pairs, causing no extra computation in the inference.
Ranked #7 on
Event-based Object Segmentation
on MVSEC-SEG
SeCGAN: Parallel Conditional Generative Adversarial Networks for Face Editing via Semantic Consistency
Whilst both branches are required during training, the RGB branch is our primary network and the semantic branch is not needed for inference.
Visual Transformer for Task-aware Active Learning
In this paper, we present a novel pipeline for pool-based Active Learning.
SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning
This paper studies a theoretical framework for value factorisation with interpretability via Shapley value theory.
Label Geometry Aware Discriminator for Conditional Generative Networks
Multi-domain image-to-image translation with conditional Generative Adversarial Networks (GANs) can generate highly photo realistic images with desired target classes, yet these synthetic images have not always been helpful to improve downstream supervised tasks such as image classification.
Learning Feature Aggregation for Deep 3D Morphable Models
3D morphable models are widely used for the shape representation of an object class in computer vision and graphics applications.
Geometry-based Distance Decomposition for Monocular 3D Object Detection
The experimental results show that our method achieves the state-of-the-art performance on the monocular 3D Object Detection and Birds Eye View tasks of the KITTI dataset, and can generalize to images with different camera intrinsics.
Ranked #16 on
Monocular 3D Object Detection
on KITTI Cars Moderate
Adversarial Imitation Learning with Trajectorial Augmentation and Correction
Deep Imitation Learning requires a large number of expert demonstrations, which are not always easy to obtain, especially for complex tasks.
Active Learning for Bayesian 3D Hand Pose Estimation
We propose a Bayesian approximation to a deep learning architecture for 3D hand pose estimation.
3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning
We utilise this dataset to minimise the novel depth consistency loss via adversarial learning (note we do not have ground truth depth maps for generated face images) and the depth categorical loss of synthetic data on the discriminator.
Physics-Based Dexterous Manipulations with Estimated Hand Poses and Residual Reinforcement Learning
Dexterous manipulation of objects in virtual environments with our bare hands, by using only a depth sensor and a state-of-the-art 3D hand pose estimator (HPE), is challenging.
Sequential Graph Convolutional Network for Active Learning
We flip the label of newly queried nodes from unlabelled to labelled, re-train the learner to optimise the downstream task and the graph to minimise its modified objective.
Ranked #4 on
Active Learning
on CIFAR10 (10,000)
MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network
We perform augmentation by randomly sampling sensible labels from the label space of the few labelled examples available and assigning them as target labels to the abundant unlabelled examples from the same distribution as that of the labelled ones.
Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System
We test HDNO on MultiWoz 2. 0 and MultiWoz 2. 1, the datasets on multi-domain dialogues, in comparison with word-level E2E model trained with RL, LaRL and HDSA, showing improvements on the performance evaluated by automatic evaluation metrics and human evaluation.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction
To address these issues, we designed a public challenge (HANDS'19) to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set.
Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images
Most successful approaches to estimate the 6D pose of an object typically train a neural network by supervising the learning with annotated poses in real world images.
Additive Angular Margin for Few Shot Learning to Classify Clinical Endoscopy Images
In this work, we propose to use a few-shot learning approach that requires less training data and can be used to predict label classes of test samples from an unseen dataset.
EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning
While each phase is mainly for one of the three tasks, the networks in earlier phases are fine-tuned by respective loss functions in an end-to-end manner.
Inducing Optimal Attribute Representations for Conditional GANs
Existing conditional GANs commonly encode target domain label information as hard-coded categorical vectors in the form of 0s and 1s.
A Review on Object Pose Recovery: from 3D Bounding Box Detectors to Full 6D Pose Estimators
In this paper, we present the first comprehensive and most recent review of the methods on object pose recovery, from 3D bounding box detectors to full 6D pose estimators.
Accurate 6D Object Pose Estimation by Pose Conditioned Mesh Reconstruction
Current 6D object pose methods consist of deep CNN models fully optimized for a single object but with its architecture standardized among objects with different shapes.
Active 6D Multi-Object Pose Estimation in Cluttered Scenarios with Deep Reinforcement Learning
In this work, we explore how a strategic selection of camera movements can facilitate the task of 6D multi-object pose estimation in cluttered scenarios while respecting real-world constraints important in robotics and augmented reality applications, such as time and distance traveled.
Sampling Strategies for GAN Synthetic Data
Unlike previous studies of randomly augmenting the synthetic data with real data, we present our simple, effective and easy to implement synthetic data sampling methods to train deep CNNs more efficiently and accurately.
Real-time Background-aware 3D Textureless Object Pose Estimation
In this work, we present a modified fuzzy decision forest for real-time 3D object pose estimation based on typical template representation.
AugLabel: Exploiting Word Representations to Augment Labels for Face Attribute Classification
Augmenting data in image space (eg.
Shapley Q-value: A Local Reward Approach to Solve Global Reward Games
To deal with this problem, we i) introduce a cooperative-game theoretical framework called extended convex game (ECG) that is a superset of global reward game, and ii) propose a local reward approach called Shapley Q-value.
Multi-agent Reinforcement Learning
Policy Gradient Methods
+1
Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Rendering
Once the model is successfully fitted to input RGB images, its meshes i. e. shapes and articulations, are realistic, and we augment view-points on top of estimated dense hand poses.
Instance- and Category-level 6D Object Pose Estimation
6D object pose estimation is an important task that determines the 3D position and 3D rotation of an object in camera-centred coordinates.
RGB-based 3D Hand Pose Estimation via Privileged Learning with Depth Images
We explore different ways of using this privileged information: (1) using depth data to initially train a depth-based network, (2) using the features from the depth-based network of the paired depth images to constrain mid-level RGB network weights, and (3) using the foreground mask, obtained from the depth data, to suppress the responses from the background area.
HANDS18: Methods, Techniques and Applications for Hand Observation
The fourth instantiation of this workshop attracted significant interest from both academia and the industry.
A Summary of the 4th International Workshop on Recovering 6D Object Pose
The workshop featured four invited talks, oral and poster presentations of accepted workshop papers, and an introduction of the BOP benchmark for 6D object pose estimation.
Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitation
In this work, we capture the hand information by using a state-of-the-art hand pose estimator.
BOP: Benchmark for 6D Object Pose Estimation
We propose a benchmark for 6D pose estimation of a rigid object from a single RGB-D input image.
SHPR-Net: Deep Semantic Hand Pose Regression From Point Clouds
The semantic segmentation network assigns semantic labels for each point in the point set.
Ranked #7 on
Hand Pose Estimation
on MSRA Hands
Category-level 6D Object Pose Recovery in Depth Images
Intra-class variations, distribution shifts among source and target domains are the major challenges of category-level tasks.
Multi-Task Deep Networks for Depth-Based 6D Object Pose and Joint Registration in Crowd Scenarios
Our architecture jointly learns multiple sub-tasks: 2D detection, depth, and 3D pose estimation of individual objects; and joint registration of multiple objects.
Augmented Skeleton Space Transfer for Depth-based Hand Pose Estimation
By training the HPG and HPE in a single unified optimization framework enforcing that 1) the HPE agrees with the paired depth and skeleton entries; and 2) the HPG-HPE combination satisfies the cyclic consistency (both the input and the output of HPG-HPE are skeletons) observed via the newly generated unpaired skeletons, our algorithm constructs a HPE which is robust to variations that go beyond the coverage of the existing database.
Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model
We propose a novel end-to-end semi-supervised adversarial framework to generate photorealistic face images of new identities with wide ranges of expressions, poses, and illuminations conditioned by a 3D morphable model.
Ranked #16 on
Face Verification
on IJB-A
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
Official Torch7 implementation of "V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map", CVPR 2018
Ranked #5 on
Hand Pose Estimation
on HANDS 2017
Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network
The proposed method leverages the state-of-the-art hand pose estimators based on Convolutional Neural Networks to facilitate feature learning, while it models the multiple modes in a two-level hierarchy to reconcile single-valued and multi-valued mapping in its output.
Pose Guided RGBD Feature Learning for 3D Object Pose Estimation
In this paper we examine the effects of using object poses as guidance to learning robust features for 3D object pose estimation.
Learning Deep Convolutional Embeddings for Face Representation Using Joint Sample- and Set-based Supervision
In this work, we investigate several methods and strategies to learn deep embeddings for face recognition, using joint sample- and set-based optimization.
The 2017 Hands in the Million Challenge on 3D Hand Pose Estimation
We present the 2017 Hands in the Million Challenge, a public competition designed for the evaluation of the task of 3D hand pose estimation.
Recovering 6D Object Pose: A Review and Multi-modal Analysis
A large number of studies analyse object detection and pose estimation at visual level in 2D, discussing the effects of challenges such as occlusion, clutter, texture, etc., on the performances of the methods, which work in the context of RGB modality.
Deep Convolutional Decision Jungle for Image Classification
Each response map-or node-in both the convolutional and fully-connected layers selectively respond to class labels s. t.
Ranked #175 on
Image Classification
on CIFAR-100
(using extra training data)
BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
We also show significant improvements in egocentric hand pose estimation with a CNN trained on the new dataset.
First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
Our dataset and experiments can be of interest to communities of 3D hand pose estimation, 6D object pose, and robotics as well as action recognition.
Learning and Refining of Privileged Information-based RNNs for Action Recognition from Depth Sequences
Our PI-based classification loss maintains a consistency between latent PI and predicted distribution.
A Learning-based Variable Size Part Extraction Architecture for 6D Object Pose Recovery in Depth
The iterative refinement is accomplished based on finer (smaller) parts that are represented with more discriminative control point descriptors by using our Iterative Hough Forest.
Real-time Online Action Detection Forests using Spatio-temporal Contexts
Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed.
Kinematic-Layout-aware Random Forests for Depth-based Action Recognition
In this paper, we tackle the problem of 24 hours-monitoring patient actions in a ward such as "stretching an arm out of the bed", "falling out of the bed", where temporal movements are subtle or significant.
Transition Forests: Learning Discriminative Temporal Transitions for Action Recognition and Detection
A human action can be seen as transitions between one's body poses over time, where the transition depicts a temporal relation between two poses.
Siamese Regression Networks with Efficient mid-level Feature Extraction for 3D Object Pose Estimation
Furthermore, we argue that our pose-guided feature learning using our Siamese Regression Network generates more discriminative features that outperform the state of the art.
Latent Bi-constraint SVM for Video-based Object Recognition
LBSVM is based on Structured-Output SVM, but extends it to handle noisy video data and ensure consistency of the output decision throughout time.
Effective Backscatter Approximation for Photometry in Murky Water
Backscatter corresponds to a complex term with several unknown variables, and makes the problem of normal estimation hard.
Spatial Attention Deep Net with Partial PSO for Hierarchical Hybrid Hand Pose Estimation
In this paper, a hybrid hand pose estimation method is proposed by applying the kinematic hierarchy strategy to the input space (as well as the output space) of the discriminative method by a spatial attention mechanism and to the optimization of the generative method by hierarchical Particle Swarm Optimization (PSO).
Iterative Hough Forest with Histogram of Control Points for 6 DoF Object Registration from Depth Images
State-of-the-art techniques proposed for 6D object pose recovery depend on occlusion-free point clouds to accurately register objects in 3D space.
Latent-Class Hough Forests for 6 DoF Object Pose Estimation
In this paper we present Latent-Class Hough Forests, a method for object detection and 6 DoF pose estimation in heavily cluttered and occluded scenarios.
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
In this work, we present a complete framework for both single shot-based 6D object pose estimation and next-best-view prediction based on Hough Forests, the state of the art object pose estimator that performs classification and regression jointly.
Conditional Convolutional Neural Network for Modality-Aware Face Recognition
Faces in the wild are usually captured with various poses, illuminations and occlusions, and thus inherently multimodally distributed in many tasks.
Opening the Black Box: Hierarchical Sampling Optimization for Estimating Human Hand Pose
In this paper, we show that we can significantly improving upon black box optimization by exploiting high-level knowledge of the structure of the parameters and using a local surrogate energy function.
Enhanced Random Forest with Image/Patch-Level Learning for Image Understanding
Image understanding is an important research domain in the computer vision due to its wide real-world applications.
Multiple Object Tracking: A Literature Review
We inspect the recent advances in various aspects and propose some interesting directions for future research.
Bi-label Propagation for Generic Multiple Object Tracking
In this paper, we propose a label propagation framework to handle the multiple object tracking (MOT) problem for a generic object type (cf.
Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture
In contrast to prior forest-based methods, which take dense pixels as input, classify them independently and then estimate joint positions afterwards; our method can be considered as a structured coarse-to-fine search, starting from the centre of mass of a point cloud until locating all the skeletal joints.
Unified Face Analysis by Iterative Multi-Output Random Forests
In this paper, we present a unified method for joint face image analysis, i. e., simultaneously estimating head pose, facial expression and landmark positions in real-world face images.
Backscatter Compensated Photometric Stereo with 3 Sources
We compare our method with previous approaches through extensive experimental results, where a variety of objects are imaged in a big water tank whose turbidity is systematically increased, and show reconstruction quality which degrades little relative to clean water results even with a very significant scattering level.
Unconstrained Monocular 3D Human Pose Estimation by Action Detection and Cross-Modality Regression Forest
This work addresses the challenging problem of unconstrained 3D human pose estimation (HPE) from a novel perspective.
MCBoost: Multiple Classifier Boosting for Perceptual Co-clustering of Images and Visual Features
We present a new co-clustering problem of images and visual features.
