Search Results for author:
Found 80 papers, 34 papers with code
InstructHumans: Editing Animated 3D Human Textures with Instructions
We present InstructHumans, a novel framework for instruction-driven 3D human texture editing.
AID: Attention Interpolation of Text-to-Image Diffusion
To that end, we introduce a novel training-free technique named Attention Interpolation via Diffusion (AID).
Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects
We interact with the world with our hands and see it through our own (egocentric) perspective.
On the Utility of 3D Hand Poses for Action Recognition
3D hand poses are an under-explored modality for action recognition.
Ranked #1 on
3D Action Recognition
on Assembly101
Coherent Temporal Synthesis for Incremental Action Segmentation
Data replay is a successful incremental learning technique for images.
Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Models
Deep Neural Networks (DNNs) are powerful tools for various computer vision tasks, yet they often struggle with reliable uncertainty quantification - a critical requirement for real-world applications.
Learning Unorthogonalized Matrices for Rotation Estimation
Estimating 3D rotations is a common procedure for 3D computer vision.
On the Calibration of Human Pose Estimation
Most 2D human pose estimation frameworks estimate keypoint confidence in an ad-hoc manner, using heuristics such as the maximum value of heatmaps.
Deep Imbalanced Regression via Hierarchical Classification Adjustment
To improve regression performance over the entire range of data, we propose to construct hierarchical classifiers for solving imbalanced regression tasks.
Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement
The capacity of a modern deep learning system to determine if a sample falls within its realm of knowledge is fundamental and important.
Ranked #1 on
Out-of-Distribution Detection
on Far-OOD
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
The Robust Semantic Segmentation UNCV2023 Challenge Results
This paper outlines the winning solutions employed in addressing the MUAD uncertainty quantification challenge held at ICCV 2023.
Can I Trust Your Answer? Visually Grounded Video Question Answering
We study visually grounded VideoQA in response to the emerging trends of utilizing pretraining techniques for video-language understanding.
HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture
We present HiFiHR, a high-fidelity hand reconstruction approach that utilizes render-and-compare in the learning-based framework from a single image, capable of generating visually plausible and accurate 3D hand meshes while recovering realistic textures.
Opening the Vocabulary of Egocentric Actions
Given a set of verbs and objects observed during training, the goal is to generalize the verbs to an open vocabulary of actions with seen and novel objects.
Ranked #1 on
Open Vocabulary Action Recognition
on Assembly101
(using extra training data)
Learning to Generate Training Datasets for Robust Semantic Segmentation
We propose a novel approach to improve the robustness of semantic segmentation techniques by leveraging the synergy between label-to-image generators and image-to-label segmentation models.
Every Mistake Counts in Assembly
Our framework constructs a knowledge base with spatial and temporal beliefs based on observed mistakes.
Overcoming the Trade-off Between Accuracy and Plausibility in 3D Hand Shape Reconstruction
However, the reconstructed meshes are prone to artifacts and do not appear as plausible hand shapes.
Enhancing Video Super-Resolution via Implicit Resampling-based Alignment
We show that bilinear interpolation inherently attenuates high-frequency information while an MLP-based coordinate network can approximate more frequencies.
Ranked #1 on
Video Super-Resolution
on Vid4 - 4x upscaling
Contrastive Video Question Answering via Video Graph Transformer
CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning.
Ranked #11 on
Video Question Answering
on NExT-QA
(using extra training data)
Bias-Compensated Integral Regression for Human Pose Estimation
Experimental results on both the human body and hand benchmarks show that BCIR is faster to train and more accurate than the original integral regression, making it competitive with state-of-the-art detection methods.
Improving Deep Regression with Ordinal Entropy
In computer vision, it is often observed that formulating regression problems as a classification task often yields better performance.
Ranked #16 on
Crowd Counting
on ShanghaiTech B
Analyzing and Diagnosing Pose Estimation With Attributions
We present Pose Integrated Gradient (PoseIG), the first interpretability technique designed for pose estimation.
Cross-Domain 3D Hand Pose Estimation With Dual Modalities
To solve this problem, we present a framework for cross-domain semi-supervised hand pose estimation and target the challenging scenario of learning models from labelled multi-modal synthetic data and unlabelled real-world data.
MHEntropy: Entropy Meets Multiple Hypotheses for Pose and Shape Recovery
For monocular RGB-based 3D pose and shape estimation, multiple solutions are often feasible due to factors like occlusion and truncation.
C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation
For the task of temporal action segmentation, we propose an encoder-decoder-style architecture named C2F-TCN featuring a "coarse-to-fine" ensemble of decoder outputs.
UV-Based 3D Hand-Object Reconstruction with Grasp Optimization
We propose a novel framework for 3D hand shape reconstruction and hand-object grasp optimization from a single RGB image.
Temporal Action Segmentation: An Analysis of Modern Techniques
Temporal action segmentation (TAS) in videos aims at densely identifying video frames in minutes-long videos with multiple action classes.
Perception-Distortion Balanced ADMM Optimization for Single-Image Super-Resolution
In image super-resolution, both pixel-wise accuracy and perceptual fidelity are desirable.
Discrete-Constrained Regression for Local Counting Models
Through a series of experiments on carefully controlled synthetic data, we show that this counter-intuitive result is caused by imprecise ground truth local counts.
A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentation
In temporal action segmentation, Timestamp supervision requires only a handful of labelled frames per video sequence.
Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation
To this end, we propose two novel loss functions for the unlabelled data: an action affinity loss and an action continuity loss.
A Closer Look at Branch Classifiers of Multi-exit Architectures
Multi-exit architectures consist of a backbone and branch classifiers that offer shortened inference pathways to reduce the run-time of deep neural networks.
TemporalUV: Capturing Loose Clothing with Temporally Coherent UV Coordinates
We propose a novel approach to generate temporally coherent UV coordinates for loose clothing.
Multi-Scale Memory-Based Video Deblurring
Video deblurring has achieved remarkable progress thanks to the success of deep neural networks.
Ranked #3 on
Analog Video Restoration
on TAPE
Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities
Assembly101 is a new procedural activity dataset featuring 4321 videos of people assembling and disassembling 101 "take-apart" toy vehicles.
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training
A standard hardware bottleneck when training deep neural networks is GPU memory.
Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation
This paper addresses the 3D point cloud reconstruction and 3D pose estimation of the human hand from a single RGB image.
Video as Conditional Graph Hierarchy for Multi-Granular Question Answering
To align with the multi-granular essence of linguistic concepts in language queries, we propose to model video as a conditional graph hierarchy which weaves together visual facts of different granularity in a level-wise manner, with the guidance of corresponding textual cues.
Ranked #23 on
Video Question Answering
on NExT-QA
Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentation
Our method hinges on unsupervised representation learning, which, for temporal action segmentation, poses unique challenges.
Weakly-Supervised Dense Action Anticipation
We present a (semi-) weakly supervised method using only a small number of fully-labelled sequences and predominantly sequences in which only the (one) upcoming action is labelled.
Dive Deeper Into Integral Pose Regression
We do a deep dive on the inference and back-propagation of integral pose regression to better understand the causes behind the performance and training differences.
Temporal Action Segmentation with High-level Complex Activity Labels
Due to the lack of action-level supervision, we adopt the Hungarian matching algorithm to relate latent action prototypes to ground truth semantic classes for evaluation.
Robust Semantic Segmentation with Superpixel-Mix
Along with predictive performance and runtime speed, reliability is a key requirement for real-world semantic segmentation.
Accelerating Video Object Segmentation with Compressed Video
We propose an efficient plug-and-play acceleration framework for semi-supervised video object segmentation by exploiting the temporal redundancies in videos presented by the compressed bitstream.
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions
We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark to advance video understanding from describing to explaining the temporal actions.
Learning Deep Morphological Networks with Neural Architecture Search
Using a combination of linear and non-linear procedures is critical for generating a sufficiently deep feature space.
Technical Report: Temporal Aggregate Representations
At what temporal scale should they be derived?
Ranked #5 on
Action Anticipation
on EPIC-KITCHENS-100 (test)
Transferring Knowledge from Text to Video: Zero-Shot Anticipation for Procedural Actions
Can we teach a robot to recognize and make predictions for activities that it has never seen before?
Transformed ROIs for Capturing Visual Transformations in Videos
Modeling the visual changes that an action brings to a scene is critical for video understanding.
Towards Compact Single Image Super-Resolution via Contrastive Self-distillation
Convolutional neural networks (CNNs) are highly successful for super-resolution (SR) but often require sophisticated architectures with heavy memory cost and computational overhead, significantly restricts their practical deployments on resource-limited devices.
Coarse to Fine Multi-Resolution Temporal Convolutional Network
In this work, we propose a novel temporal encoder-decoder to tackle the problem of sequence fragmentation.
Ranked #3 on
Action Segmentation
on Assembly101
NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions
We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark to advance video understanding from describing to explaining the temporal actions.
Geospatial indicators of exposure, sensitivity, and adaptive capacity to assess neighbourhood variation in vulnerability to climate change-related health hazards
In addition to an overall vulnerability score, exposure, adaptive capacity, and sensitivity sub-scores were computed for each hazard.
Removing the Bias of Integral Pose Regression
Heatmap-based detection methods are dominant for 2D human pose estimation even though regression is more intuitive.
SemiHand: Semi-Supervised Hand Pose Estimation With Consistency
By design, we introduce data augmentation of differing difficulties, consistency regularizer, label correction and sample selection for RGB-based 3D hand pose estimation.
Multi-Stage Fusion for One-Click Segmentation
Segmenting objects of interest in an image is an essential building block of applications such as photo-editing and image analysis.
Localized Interactive Instance Segmentation
In current interactive instance segmentation works, the user is granted a free hand when providing clicks to segment an object; clicks are allowed on background pixels and other object instances far from the target object.
Rethinking CNN Models for Audio Classification
Besides, we show that even though we use the pretrained model weights for initialization, there is variance in performance in various output runs of the same model.
Environmental Sound Classification
General Classification
+2
Temporal Aggregate Representations for Long-Range Video Understanding
Future prediction, especially in long-range videos, requires reasoning from current and past observations.
Ranked #2 on
Action Anticipation
on Assembly101
Neural network compression via learnable wavelet transforms
Linear layers still occupy a significant portion of the parameters in recurrent neural networks (RNNs).
Object-centered Fourier Motion Estimation and Segment-Transformation Prediction
The ability to anticipate the future is essential for ac-tion planning in autonomous systems.
Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and Objects for 3D Hand Pose Estimation under Hand-Object Interaction
To address these issues, we designed a public challenge (HANDS'19) to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set.
Bonn Activity Maps: Dataset Description
The key prerequisite for accessing the huge potential of current machine learning techniques is the availability of large databases that capture the complex relations of interest.
Dual Grid Net: hand mesh vertex regression from single depth maps
In the first stage, the network estimates a dense correspondence field for every pixel on the depth map or image grid to the mesh grid.
Sequence Prediction using Spectral RNNs
Fourier methods have a long and proven track record as an excellent tool in data processing.
Supervised Deep Kriging for Single-Image Super-Resolution
We propose a novel single-image super-resolution approach based on the geostatistical method of kriging.
Learning Style Compatibility for Furniture
When judging style, a key question that often arises is whether or not a pair of objects are compatible with each other.
Scale-aware multi-level guidance for interactive instance segmentation
In interactive instance segmentation, users give feedback to iteratively refine segmentation masks.
Zero-Shot Anticipation for Instructional Activities
How can we teach a robot to predict what will happen next for an activity it has never seen before?
Disentangling Latent Hands for Image Synthesis and Pose Estimation
Hand image synthesis and pose estimation from RGB images are both highly challenging tasks due to the large discrepancy between factors of variation ranging from image background content to camera viewpoint.
HANDS18: Methods, Techniques and Applications for Hand Observation
The fourth instantiation of this workshop attracted significant interest from both academia and the industry.
Complex Gated Recurrent Neural Networks
Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures.
Unsupervised Learning and Segmentation of Complex Activities from Video
This paper presents a new method for unsupervised segmentation of complex activities from video into multiple steps, or sub-activities, without any textual input.
Dense 3D Regression for Hand Pose Estimation
Specifically, we decompose the pose parameters into a set of per-pixel estimations, i. e., 2D heat maps, 3D heat maps and unit 3D directional vector fields.
Ranked #4 on
Hand Pose Estimation
on MSRA Hands
Crossing Nets: Combining GANs and VAEs with a Shared Latent Space for Hand Pose Estimation
Regressing the hand pose can then be done by learning a discriminator to estimate the posterior of the latent pose given some depth maps.
A Two-Streamed Network for Estimating Fine-Scaled Depth Maps from Single RGB Images
Estimating depth from a single RGB image is an ill-posed and inherently ambiguous problem.
Ranked #69 on
Monocular Depth Estimation
on NYU-Depth V2
Direction matters: hand pose estimation from local surface normals
We present a hierarchical regression framework for estimating hand joint positions from single depth images based on local surface normals.
Efficient Unsupervised Temporal Segmentation of Motion Data
We introduce a method for automated temporal segmentation of human motion data into distinct actions and compositing motion primitives based on self-similar structures in the motion sequence.
Gesture Recognition Portfolios for Personalization
Human gestures, similar to speech and handwriting, are often unique to the individual.
Learning Probabilistic Non-Linear Latent Variable Models for Tracking Complex Activities
A common approach for handling the complexity and inherent ambiguities of 3D human pose estimation is to use pose priors learned from training data.
