Search Results for author:
Found 33 papers, 4 papers with code
COSMU: Complete 3D human shape from monocular unconstrained images
The generated 2D normal maps are then processed by a multi-view attention-based neural implicit model that estimates an implicit representation of the 3D shape, ensuring the reproduction of details in both observed and occluded regions.
Benchmarking Monocular 3D Dog Pose Estimation Using In-The-Wild Motion Capture Data
To address this, we created 3DDogs-Wild, a naturalised version of the dataset where the optical markers are in-painted and the subjects are placed in diverse environments, enhancing its utility for training RGB image-based pose detectors.
NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrative
Existing video captioning benchmarks and models lack coherent representations of causal-temporal narrative, which is sequences of events linked through cause and effect, unfolding over time and driven by characters or agents.
Ranked #1 on
Video Captioning
on MSVD-CTN
An Effective-Efficient Approach for Dense Multi-Label Action Detection
In this paper, we address this issue by proposing a novel transformer-based network that (a) employs a non-hierarchical structure when modelling different ranges of temporal dependencies and (b) embeds relative positional encoding in its transformer layers.
Gaussian Splatting with Localized Points Management
To address these limitations, we propose a Localized Point Management (LPM) strategy, capable of identifying those error-contributing zones in the highest demand for both point addition and geometry calibration.
CoLeaF: A Contrastive-Collaborative Learning Framework for Weakly Supervised Audio-Visual Video Parsing
In this paper, we propose CoLeaF, a novel learning framework that optimizes the integration of cross-modal context in the embedding space such that the network explicitly learns to combine cross-modal information for audible-visible events while filtering them out for unaligned events.
ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D image
In this paper, we explore the benefits of incorporating depth observations in the reconstruction process by introducing ANIM, a novel method that reconstructs arbitrary 3D human shapes from single-view RGB-D images with an unprecedented level of accuracy.
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA
In the context of Audio Visual Question Answering (AVQA) tasks, the audio visual modalities could be learnt on three levels: 1) Spatial, 2) Temporal, and 3) Semantic.
Ranked #3 on
Audio-visual Question Answering
on MUSIC-AVQA
Audio-visual Question Answering
Audio-Visual Question Answering (AVQA)
+2
PAT: Position-Aware Transformer for Dense Multi-Label Action Detection
To address this issue, we (i) embed relative positional encoding in the self-attention mechanism and (ii) exploit multi-scale temporal relationships by designing a novel non hierarchical network, in contrast to the recent transformer-based approaches that use a hierarchical structure.
Ranked #1 on
Action Detection
on MultiTHUMOS
SEM-POS: Grammatically and Semantically Correct Video Captioning
Generating grammatically and semantically correct captions in video captioning is a challenging task.
Ranked #3 on
Video Captioning
on MSVD-CTN
Super-resolution 3D Human Shape from a Single Low-Resolution Image
The approach overcomes limitations of existing approaches that reconstruct 3D human shape from a single image, which require high-resolution images together with auxiliary data such as surface normal or a parametric model to reconstruct high-detail shape.
Visually Supervised Speaker Detection and Localization via Microphone Array
We propose to generate weak labels using a pre-trained active speaker detector on pre-extracted face tracks.
Super-Resolution Appearance Transfer for 4D Human Performances
Typically the requirement to frame cameras to capture the volume of a dynamic performance ($>50m^3$) results in the person occupying only a small proportion $<$ 10% of the field of view.
Attention-based Multi-Reference Learning for Image Super-Resolution
A novel hierarchical attention-based sampling approach is introduced to learn the similarity between low-resolution image features and multiple reference images based on a perceptual loss.
SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation
Estimating the pose of animals can facilitate the understanding of animal motion which is fundamental in disciplines such as biomechanics, neuroscience, ethology, robotics and the entertainment industry.
Ranked #1 on
Animal Pose Estimation
on StanfordExtra
Temporal Consistency Loss for High Resolution Textured and Clothed 3DHuman Reconstruction from Monocular Video
We present a novel method to learn temporally consistent 3D reconstruction of clothed people from a monocular video.
Multi-person Implicit Reconstruction from a Single Image
We present a new end-to-end learning framework to obtain detailed and spatially coherent reconstructions of multiple people from a single image.
Multi-View Consistency Loss for Improved Single-Image 3D Reconstruction of Clothed People
This paper introduces two advances to overcome this limitation: firstly a new synthetic dataset of realistic clothed people, 3DVH; and secondly, a novel multiple-view loss function for training of monocular volumetric shape estimation, which is demonstrated to significantly improve generalisation and reconstruction accuracy.
Spectral Analysis Network for Deep Representation Learning and Image Clustering
This paper proposes a new network structure for unsupervised deep representation learning based on spectral analysis, which is a popular technique with solid theory foundations.
Learning Dense Wide Baseline Stereo Matching for People
We show that it is possible to learn stereo matching from synthetic people dataset and improve performance on real datasets for stereo reconstruction of people from narrow and wide baseline stereo data.
Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras
We aim to simultaneously estimate the 3D articulated pose and high fidelity volumetric occupancy of human performance, from multiple viewpoint video (MVV) with as few as two views.
Ranked #176 on
3D Human Pose Estimation
on Human3.6M
EdgeNet: Semantic Scene Completion from a Single RGB-D Image
Semantic scene completion is the task of predicting a complete 3D representation of volumetric occupancy with corresponding semantic labels for a scene from a single point of view.
Ranked #22 on
3D Semantic Scene Completion
on NYUv2
U4D: Unsupervised 4D Dynamic Scene Understanding
We introduce the first approach to solve the challenging problem of unsupervised 4D visual scene understanding for complex dynamic scenes with multiple interacting people from multi-view video.
Temporally Coherent General Dynamic Scene Reconstruction
Existing techniques for dynamic scene reconstruction from multiple wide-baseline cameras primarily focus on reconstruction in controlled environments, with fixed calibrated cameras and strong prior constraints.
Volumetric performance capture from minimal camera viewpoints
We present a convolutional autoencoder that enables high fidelity volumetric reconstructions of human performance to be captured from multi-view video comprising only a small set of camera views.
Deep Autoencoder for Combined Human Pose Estimation and body Model Upscaling
We present a method for simultaneously estimating 3D human pose and body shape from a sparse set of wide-baseline camera views.
Ranked #9 on
3D Human Pose Estimation
on Total Capture
4D Temporally Coherent Light-field Video
Evaluation of the proposed light-field scene flow against existing multi-view dense correspondence approaches demonstrates a significant improvement in accuracy of temporal coherence.
Semantic Scene Completion Combining Colour and Depth: preliminary experiments
Semantic scene completion is the task of producing a complete 3D voxel representation of volumetric occupancy with semantic labels for a scene from a single-view observation.
Ranked #23 on
3D Semantic Scene Completion
on NYUv2
Total capture: 3D human pose estimation fusing video and inertial sensors
We incorporate this model within a dual stream network integrating pose embeddings derived from MVV and a forward kinematic solve of the IMU data.
Ranked #11 on
3D Human Pose Estimation
on Total Capture
Semantically Coherent Co-Segmentation and Reconstruction of Dynamic Scenes
Semantic co-segmentation exploits the coherence in semantic class labels both spatially, between views at a single time instant, and temporally, between widely spaced time instants of dynamic objects with similar shape and appearance.
Temporally coherent 4D reconstruction of complex dynamic scenes
Sparse-to-dense temporal correspondence is integrated with joint multi-view segmentation and reconstruction to obtain a complete 4D representation of static and dynamic objects.
FaceDirector: Continuous Control of Facial Performance in Video
We present a method to continuously blend between multiple facial performances of an actor, which can contain different facial expressions or emotional states.
General Dynamic Scene Reconstruction from Multiple View Video
The primary contributions of this paper are twofold: an automatic method for initial coarse dynamic scene segmentation and reconstruction without prior knowledge of background appearance or structure; and a general robust approach for joint segmentation refinement and dense reconstruction of dynamic scenes from multiple wide-baseline static or moving cameras.
