Search Results for author:
Found 193 papers, 107 papers with code
Unleashing Text-to-Image Diffusion Models for Visual Perception
In this paper, we propose VPD (Visual Perception with a pre-trained Diffusion model), a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks.
Ranked #7 on
Referring Expression Segmentation
on RefCoCo val
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction
To lift image features to the 3D TPV space, we further propose a transformer-based TPV encoder (TPVFormer) to obtain the TPV features effectively.
Ranked #1 on
Prediction Of Occupancy Grid Maps
on nuScenes
HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions
In this paper, we show that the key ingredients behind the vision Transformers, namely input-adaptive, long-range and high-order spatial interactions, can also be efficiently implemented with a convolution-based framework.
Ranked #20 on
Semantic Segmentation
on ADE20K
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving
Towards a more comprehensive perception of a 3D scene, in this paper, we propose a SurroundOcc method to predict the 3D occupancy with multi-camera images.
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
Based on this observation, we propose a dynamic token sparsification framework to prune redundant tokens progressively and dynamically based on the input.
Ranked #3 on
Efficient ViTs
on ImageNet-1K (With LV-ViT-S)
Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks
We extend our method to hierarchical models including CNNs and hierarchical vision Transformers as well as more complex dense prediction tasks that require structured feature maps by formulating a more generic dynamic spatial sparsification framework with progressive sparsification and asymmetric computation for different spatial locations.
OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception
Towards a comprehensive benchmarking of surrounding perception algorithms, we propose OpenOccupancy, which is the first surrounding semantic occupancy perception benchmark.
PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers
In this paper, we present a new method that reformulates point cloud completion as a set-to-set translation problem and design a new model, called PoinTr that adopts a transformer encoder-decoder architecture for point cloud completion.
Ranked #1 on
Point Cloud Completion
on ShapeNet
(Chamfer Distance L2 metric)
AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers
In this paper, we present a new method that reformulates point cloud completion as a set-to-set translation problem and design a new model, called PoinTr, which adopts a Transformer encoder-decoder architecture for point cloud completion.
Ranked #2 on
Point Cloud Completion
on ShapeNet
Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
Inspired by BERT, we devise a Masked Point Modeling (MPM) task to pre-train point cloud Transformers.
Ranked #13 on
Few-Shot 3D Point Cloud Classification
on ModelNet40 5-way (10-shot)
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
+2
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting
In this work, we present a new framework for dense prediction by implicitly and explicitly leveraging the pre-trained knowledge from CLIP.
Structure-Preserving Super Resolution with Gradient Guidance
In this paper, we propose a structure-preserving super resolution method to alleviate the above issue while maintaining the merits of GAN-based methods to generate perceptual-pleasant details.
Ranked #46 on
Image Super-Resolution
on Urban100 - 4x upscaling
Structure-Preserving Image Super-Resolution
Firstly, we propose SPSR with gradient guidance (SPSR-G) by exploiting gradient maps of images to guide the recovery in two aspects.
NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo
In this work, we present a new multi-view depth estimation method that utilizes both conventional reconstruction and learning-based priors over the recently proposed neural radiance fields (NeRF).
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation
In this way, the proposed DiffTalk is capable of producing high-quality talking head videos in synchronization with the source audio, and more importantly, it can be naturally generalized across different identities without any further fine-tuning.
Global Filter Networks for Image Classification
Recent advances in self-attention and pure multi-layer perceptrons (MLP) models for vision have shown great potential in achieving promising performance with fewer inductive biases.
Ranked #9 on
Image Classification
on Stanford Cars
(using extra training data)
BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving
Specifically, BEVerse first performs shared feature extraction and lifting to generate 4D BEV representations from multi-timestamp and multi-view images.
Ranked #15 on
Robust Camera Only 3D Object Detection
on nuScenes-C
Objects are Different: Flexible Monocular 3D Object Detection
The precise localization of 3D objects from a single image without depth information is a highly challenging problem.
Ranked #8 on
Monocular 3D Object Detection
on KITTI Cars Moderate
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis
Thus the facial radiance field can be flexibly adjusted to the new identity with few reference images.
Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation
In this paper, we propose a deep face super-resolution (FSR) method with iterative collaboration between two recurrent networks which focus on facial image recovery and landmark estimation respectively.
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
In this paper, we explore a new framework of learning a world model, OccWorld, in the 3D Occupancy space to simultaneously predict the movement of the ego car and the evolution of the surrounding scenes.
OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments
As a fundamental task of vision-based perception, 3D occupancy prediction reconstructs 3D structures of surrounding environments.
SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation
In this paper, we propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras.
SelfOcc: Self-Supervised Vision-Based 3D Occupancy Prediction
Our SelfOcc outperforms the previous best method SceneRF by 58. 7% using a single frame as input on SemanticKITTI and is the first self-supervised work that produces reasonable 3D occupancy for surround cameras on nuScenes.
DSNet: A Flexible Detect-to-Summarize Network for Video Summarization
In this paper, we propose a Detect-to-Summarize network (DSNet) framework for supervised video summarization.
Ranked #2 on
Video Summarization
on TvSum
(using extra training data)
BiDet: An Efficient Binarized Object Detector
Conventional network binarization methods directly quantize the weights and activations in one-stage or two-stage detectors with constrained representational capacity, so that the information redundancy in the networks causes numerous false positives and degrades the performance significantly.
Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion
Human behavior has the nature of indeterminacy, which requires the pedestrian trajectory prediction system to model the multi-modality of future motion states.
Diffusion-SDF: Text-to-Shape via Voxelized Diffusion
With the rising industrial attention to 3D virtual modeling technology, generating novel 3D content based on specified conditions (e. g. text) has become a hot issue.
Multi-View Partial (MVP) Point Cloud Challenge 2021 on Completion and Registration: Methods and Results
Based on the MVP dataset, this paper reports methods and results in the Multi-View Partial Point Cloud Challenge 2021 on Completion and Registration.
LOGO: A Long-Form Video Dataset for Group Action Quality Assessment
Action quality assessment (AQA) has become an emerging topic since it can be extensively applied in numerous scenarios.
Segment and Caption Anything
We propose a method to efficiently equip the Segment Anything Model (SAM) with the ability to generate regional captions.
Hardness-Aware Deep Metric Learning
This paper presents a hardness-aware deep metric learning (HDML) framework.
Ranked #30 on
Metric Learning
on CUB-200-2011
(using extra training data)
Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification
Unlike most existing methods that learn visual attention based on conventional likelihood, we propose to learn the attention with counterfactual causality, which provides a tool to measure the attention quality and a powerful supervisory signal to guide the learning process.
Ranked #8 on
Vehicle Re-Identification
on VehicleID Medium
DensePoint: Learning Densely Contextual Representation for Efficient Point Cloud Processing
Point cloud processing is very challenging, as the diverse shapes formed by irregular points are often indistinguishable.
Ranked #22 on
3D Part Segmentation
on ShapeNet-Part
P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting
Nowadays, pre-training big models on large-scale datasets has become a crucial topic in deep learning.
Ranked #18 on
3D Point Cloud Classification
on ScanObjectNN
(using extra training data)
Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds
Based on this hypothesis, we propose to learn point cloud representation by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape without human supervision.
Embodied Task Planning with Large Language Models
Equipping embodied agents with commonsense is important for robots to successfully complete complex human instructions in general environments.
PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction
To address this, we propose a cylindrical tri-perspective view to represent point clouds effectively and comprehensively and a PointOcc model to process them efficiently.
Structure-Aware Face Clustering on a Large-Scale Graph with $\bf{10^{7}}$ Nodes
To address the dilemma of large-scale training and efficient inference, we propose the STructure-AwaRe Face Clustering (STAR-FC) method.
Structure-Aware Face Clustering on a Large-Scale Graph With 107 Nodes
To address the dilemma of large-scale training and efficient inference, we propose the STructure-AwaRe Face Clustering (STAR-FC) method.
LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection
In this paper, we propose the LiDAR Distillation to bridge the domain gap induced by different LiDAR beams for 3D object detection.
FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment
Most existing action quality assessment methods rely on the deep features of an entire video to predict the score, which is less reliable due to the non-transparent inference process and poor interpretability.
Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos
The generated text prompts are paired with corresponding video clips, and together co-train the text encoder and the video encoder via a contrastive approach.
Ranked #4 on
Action Segmentation
on GTEA
(using extra training data)
Towards Interpretable Deep Metric Learning with Structural Matching
Our method is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods.
Ranked #16 on
Metric Learning
on CUB-200-2011
3D Small Object Detection with Dynamic Spatial Pruning
Specifically, we theoretically derive a dynamic spatial pruning (DSP) strategy to prune the redundant spatial representation of 3D scene in a cascade manner according to the distribution of objects.
Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models
In the realm of vision-language understanding, the proficiency of models in interpreting and reasoning over visual content has become a cornerstone for numerous applications.
Ranked #42 on
Visual Question Answering
on MM-Vet
PCANet: A Simple Deep Learning Baseline for Image Classification?
In this work, we propose a very simple deep learning network for image classification which comprises only the very basic data processing components: cascaded principal component analysis (PCA), binary hashing, and block-wise histograms.
Ranked #46 on
Image Classification
on MNIST
DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion
Unlike previous work that relies on carefully designed network architectures and loss functions to fuse the information from the source and target faces, we reformulate the face swapping as a conditional inpainting task, performed by a powerful diffusion model guided by the desired face attributes (e. g., identity and landmarks).
Human Trajectory Prediction via Counterfactual Analysis
Most existing methods learn to predict future trajectories by behavior clues from history trajectories and interaction clues from environments.
LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images
In particular we adopt a learnable representation approach to the fusion task, in which the construction of the fusion network architecture is guided by the optimisation algorithm producing the learnable model.
PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds
In this paper, we propose a Point-Voxel Recurrent All-Pairs Field Transforms (PV-RAFT) method to estimate scene flow from point clouds.
Group-aware Contrastive Regression for Action Quality Assessment
Assessing action quality is challenging due to the subtle differences between videos and large variations in scores.
Ranked #2 on
Action Quality Assessment
on MTL-AQA
MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory
Due to the high price and heavy energy consumption of GPUs, deploying deep models on IoT devices such as microcontrollers makes significant contributions for ecological AI.
Cross-Modal Adapter for Text-Video Retrieval
However, as pre-trained models are scaling up, fully fine-tuning them on text-video retrieval datasets has a high risk of overfitting.
Learning Series-Parallel Lookup Tables for Efficient Image Super-Resolution
On the other hand, we propose a parallel network which includes two branches of cascaded lookup tables which process different components of the input low-resolution images.
Person Re-identification via Attention Pyramid
Experimental results demonstrate the superiority of our method, which outperforms the state-of-the-art methods by a large margin with limited computational cost.
Ranked #21 on
Person Re-Identification
on MSMT17
SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation
Most existing point cloud instance and semantic segmentation methods rely heavily on strong supervision signals, which require point-level labels for every point in the scene.
A Simple Baseline for Multi-Camera 3D Object Detection
First, we extract multi-scale features and generate the perspective object proposals on each monocular image.
SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation
Conventional point cloud semantic segmentation methods usually employ an encoder-decoder architecture, where mid-level features are locally aggregated to extract geometric information.
Introspective Deep Metric Learning for Image Retrieval
This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images.
Introspective Deep Metric Learning
This paper proposes an introspective deep metric learning (IDML) framework for uncertainty-aware comparisons of images.
Deep Relational Metric Learning
This paper presents a deep relational metric learning (DRML) framework for image clustering and retrieval.
FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection
To tackle this problem, we propose frustum-aware geometric reasoning (FGR) to detect vehicles in point clouds without any 3D annotations.
Inconsistency-aware Uncertainty Estimation for Semi-supervised Medical Image Segmentation
In semi-supervised medical image segmentation, most previous works draw on the common assumption that higher entropy means higher uncertainty.
Uncertainty-aware Score Distribution Learning for Action Quality Assessment
Assessing action quality from videos has attracted growing attention in recent years.
Ranked #4 on
Action Quality Assessment
on AQA-7
RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection
In particular, we propose to generate random layouts of a scene by making use of the objects in the synthetic CAD dataset and learn the 3D scene representation by applying object-level contrastive learning on two random scenes generated from the same set of synthetic objects.
Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement
In this paper, we propose a weakly-supervised approach for 3D object detection, which makes it possible to train a strong 3D detector with position-level annotations (i. e. annotations of object centers).
Attributable Visual Similarity Learning
This paper proposes an attributable visual similarity learning (AVSL) framework for a more accurate and explainable similarity measure between images.
Ranked #3 on
Metric Learning
on CARS196
(using extra training data)
Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression
An ordinal distribution constraint is proposed to exploit the ordinal nature of regression.
Ranked #2 on
Age Estimation
on Adience
Aesthetics Quality Assessment
Age And Gender Classification
+3
Personalized Trajectory Prediction via Distribution Discrimination
In this paper, we present a distribution discrimination (DisDis) method to predict personalized motion patterns by distinguishing the potential distributions.
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression
In this paper, we propose to learn the rank concepts from the rich semantic CLIP latent space.
Ranked #1 on
Few-shot Age Estimation
on MORPH Album2
OPERA: Omni-Supervised Representation Learning with Hierarchical Supervisions
The pretrain-finetune paradigm in modern computer vision facilitates the success of self-supervised learning, which tends to achieve better transferability than supervised learning.
Self-Supervised Video Hashing via Bidirectional Transformers
Most existing unsupervised video hashing methods are built on unidirectional models with less reliable training objectives, which underuse the correlations among frames and the similarity structure between videos.
Take-A-Photo: 3D-to-2D Generative Pre-training of Point Cloud Models
In this paper, we propose a novel 3D-to-2D generative pre-training method that is adaptable to any point cloud model.
Ranked #6 on
3D Part Segmentation
on ShapeNet-Part
Label2Label: A Language Modeling Framework for Multi-Attribute Learning
As each sample is annotated with multiple attribute labels, these "words" will naturally form an unordered but meaningful "sentence", which depicts the semantic information of the corresponding sample.
Generalizable Mixed-Precision Quantization via Attribution Rank Preservation
On the contrary, our GMPQ searches the mixed-quantization policy that can be generalized to largescale datasets with only a small amount of data, so that the search cost is significantly reduced without performance degradation.
Efficient Meshy Neural Fields for Animatable Human Avatars
Moreover, only minutes of optimization is enough for plausible reconstruction results.
Similarity-Aware Fusion Network for 3D Semantic Segmentation
To address this, we employ a late fusion strategy where we first learn the geometric and contextual similarities between the input and back-projected (from 2D pixels) point clouds and utilize them to guide the fusion of two modalities to further exploit complementary information.
Ranked #21 on
Semantic Segmentation
on ScanNet
Instance Similarity Learning for Unsupervised Feature Representation
In this paper, we propose an instance similarity learning (ISL) method for unsupervised feature representation.
Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search
Differentiable architecture search (DARTS) acquires the optimal architectures by optimizing the architecture parameters with gradient descent, which significantly reduces the search cost.
Token-Label Alignment for Vision Transformers
Data mixing strategies (e. g., CutMix) have shown the ability to greatly improve the performance of convolutional neural networks (CNNs).
Deep Compositional Metric Learning
In this paper, we propose a deep compositional metric learning (DCML) framework for effective and generalizable similarity measurement between images.
Graph-Based Social Relation Reasoning
Human beings are fundamentally sociable -- that we generally organize our social lives in terms of relations with other people.
Ranked #1 on
Visual Social Relationship Recognition
on PIPA
Deep Factorized Metric Learning
Learning a generalizable and comprehensive similarity metric to depict the semantic discrepancies between images is the foundation of many computer vision tasks.
An Improved Evaluation Framework for Generative Adversarial Networks
Unlike most existing evaluation frameworks which transfer the representation of ImageNet inception model to map images onto the feature space, our framework uses a specialized encoder to acquire fine-grained domain-specific representation.
Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantization
We first give a theoretical analysis that the diversity of synthetic samples is crucial for the data-free quantization, while in existing approaches, the synthetic data completely constrained by BN statistics experimentally exhibit severe homogenization at distribution and sample levels.
Narrative Action Evaluation with Prompt-Guided Multimodal Interaction
NAE is a more challenging task because it requires both narrative flexibility and evaluation rigor.
MADTP: Multimodal Alignment-Guided Dynamic Token Pruning for Accelerating Vision-Language Transformer
To this end, we propose a novel framework named Multimodal Alignment-Guided Dynamic Token Pruning (MADTP) for accelerating various VLTs.
Bort: Towards Explainable Neural Networks with Bounded Orthogonal Constraint
Deep learning has revolutionized human society, yet the black-box nature of deep neural networks hinders further application to reliability-demanded industries.
Separable Structure Modeling for Semi-supervised Video Object Segmentation
Specifically, we first compute a pixel-wise similarity matrix by using representations of reference and target pixels and then select top-rank reference pixels for target pixel classification.
Learning from Temporal Spatial Cubism for Cross-Dataset Skeleton-based Action Recognition
Different from the conventional adversarial learning-based approaches for UDA, we utilize a self-supervision scheme to reduce the domain shift between two skeleton-based action datasets.
TCOVIS: Temporally Consistent Online Video Instance Segmentation
The core of our method consists of a global instance assignment strategy and a spatio-temporal enhancement module, which improve the temporal consistency of the features from two aspects.
DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery
The recovery of occluded human meshes presents challenges for current methods due to the difficulty in extracting effective image features under severe occlusion.
MetaAge: Meta-Learning Personalized Age Estimators
Unlike most existing personalized methods that learn the parameters of a personalized estimator for each person in the training set, our method learns the mapping from identity information to age estimator parameters.
Ranked #1 on
Age Estimation
on ChaLearn 2015
Dense Hybrid Proposal Modulation for Lane Detection
In addition to the shape and location constraints, we design a quality-aware classification loss to adaptively supervise each positive proposal so that the discriminative power can be further boosted.
Automatic Data Augmentation by Learning the Deterministic Policy
By introducing an unified optimization goal, DeepAugNet intends to combine the data augmentation and the deep model training in an end-to-end training manner which is realized by simultaneously training a hybrid architecture of dueling deep Q-learning algorithm and a surrogate deep model.
Content-aware Warping for View Synthesis
To this end, we propose content-aware warping, which adaptively learns the interpolation weights for pixels of a relatively large neighborhood from their contextual information via a lightweight neural network.
Learning Accurate Performance Predictors for Ultrafast Automated Model Compression
On the contrary, we obtain the optimal efficient networks by directly optimizing the compression policy with an accurate performance predictor, where the ultrafast automated model compression for various computational cost constraint is achieved without complex compression policy search and evaluation.
Probabilistic Deep Metric Learning for Hyperspectral Image Classification
We propose to learn a global probabilistic distribution for each pixel in the patch and a probabilistic metric to model the distance between distributions.
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression
Recent Vision Transformer Compression (VTC) works mainly follow a two-stage scheme, where the importance score of each model unit is first evaluated or preset in each submodule, followed by the sparsity score evaluation according to the target sparsity constraint.
Dynamics-aware Adversarial Attack of 3D Sparse Convolution Network
It results in a serious issue of lagged gradient, making the learned attack at the current step ineffective due to the architecture changes afterward.
Dynamics-aware Adversarial Attack of Adaptive Neural Networks
To address this issue, we propose a Leaded Gradient Method (LGM) and show the significant effects of the lagged gradient.
Exploring Unified Perspective For Fast Shapley Value Estimation
Shapley values have emerged as a widely accepted and trustworthy tool, grounded in theoretical axioms, for addressing challenges posed by black-box models like deep neural networks.
Path Choice Matters for Clear Attribution in Path Methods
Rigorousness and clarity are both essential for interpretations of DNNs to engender human trust.
X-3D: Explicit 3D Structure Modeling for Point Cloud Recognition
However, we contend that such implicit high-dimensional structure modeling approch inadequately represents the local geometric structure of point clouds due to the absence of explicit structural information.
Deep Sparse Subspace Clustering
In this paper, we present a deep extension of Sparse Subspace Clustering, termed Deep Sparse Subspace Clustering (DSSC).
3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
Our method provides an automatic process that maps the raw data to the classification results.
Correlated and Individual Multi-Modal Deep Learning for RGB-D Object Recognition
Specifically, we construct a pair of deep convolutional neural networks (CNNs) for the RGB and depth data, and concatenate them at the top layer of the network with a loss function which learns a new feature space where both correlated part and the individual part of the RGB-D information are well modelled.
Automatic Subspace Learning via Principal Coefficients Embedding
In this paper, we address two challenging problems in unsupervised subspace learning: 1) how to automatically identify the feature dimension of the learned subspace (i. e., automatic subspace learning), and 2) how to learn the underlying subspace in the presence of Gaussian noise (i. e., robust subspace learning).
A Siamese Long Short-Term Memory Architecture for Human Re-Identification
Matching pedestrians across multiple camera views known as human re-identification (re-identification) is a challenging problem in visual surveillance.
Multi-task CNN Model for Attribute Prediction
Each CNN will generate attribute-specific feature representations, and then we apply multi-task learning on the features to predict their attributes.
Nonlinear Local Metric Learning for Person Re-identification
In this paper, we propose a nonlinear local metric learning (NLML) method to improve the state-of-the-art performance of person re-identification on public datasets.
Learning Invariant Color Features for Person Re-Identification
We model color feature generation as a learning problem by jointly learning a linear transformation and a dictionary to encode pixel values.
Face Recognition via Globality-Locality Preserving Projections
We present an improved Locality Preserving Projections (LPP) method, named Gloablity-Locality Preserving Projections (GLPP), to preserve both the global and local geometric structures of data.
Runtime Neural Pruning
In this paper, we propose a Runtime Neural Pruning (RNP) framework which prunes the deep neural network dynamically at the runtime.
Deep Adversarial Metric Learning
Learning an effective distance metric between image pairs plays an important role in visual analysis, where the training procedure largely relies on hard negative samples.
Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition
In this paper, we propose a deep progressive reinforcement learning (DPRL) method for action recognition in skeleton-based videos, which aims to distil the most informative frames and discard ambiguous frames in sequences for recognizing actions.
Ranked #3 on
Skeleton Based Action Recognition
on UT-Kinect
Learning Globally Optimized Object Detector via Policy Gradient
In this paper, we propose a simple yet effective method to learn globally optimized detector for object detection, which is a simple modification to the standard cross-entropy gradient inspired by the REINFORCE algorithm.
Deep Hashing via Discrepancy Minimization
This paper presents a discrepancy minimizing model to address the discrete optimization problem in hashing learning.
GraphBit: Bitwise Interaction Mining via Deep Reinforcement Learning
Specifically, we design a deep reinforcement learning model to learn the structure of the graph for bitwise interaction mining, reducing the uncertainty of binary codes by maximizing the mutual information with inputs and related bits, so that the ambiguous bits receive additional instruction from the graph for confident binarization.
Part-Activated Deep Reinforcement Learning for Action Prediction
In this paper, we propose a part-activated deep reinforcement learning (PA-DRL) for action prediction.
Dual-Agent Deep Reinforcement Learning for Deformable Face Tracking
In this paper, we propose a dual-agent deep reinforcement learning (DADRL) method for deformable face tracking, which generates bounding boxes and detects facial landmarks interactively from face videos.
Deep Reinforcement Learning with Iterative Shift for Visual Tracking
Visual tracking is confronted by the dilemma to locate a target both}accurately and efficiently, and make decisions online whether and how to adapt the appearance model or even restart tracking.
Collaborative Deep Reinforcement Learning for Multi-Object Tracking
To address this, we develop a deep prediction-decision network in our C-DRL, which simultaneously detects and predicts objects under a unified network via deep reinforcement learning.
Deep Variational Metric Learning
Deep metric learning has been extensively explored recently, which trains a deep neural network to produce discriminative embedding features.
Graininess-Aware Deep Feature Learning for Pedestrian Detection
In this paper, we propose a graininess-aware deep feature learning method for pedestrian detection.
Relaxation-Free Deep Hashing via Policy Gradient
In this paper, we propose a simple yet effective relaxation-free method to learn more effective binary codes via policy gradient for scalable image search.
Improving Sample-based Evaluation for Generative Adversarial Networks
Unlike most existing evaluation frameworks which transfer the representation of ImageNet inception model to map images onto the feature space, our framework uses a specialized encoder to acquire fine-grained domain-specific representation.
Discriminative Deep Metric Learning for Face Verification in the Wild
This paper presents a new discriminative deep metric learning (DDML) method for face verification in the wild.
Deep Transfer Metric Learning
Conventional metric learning methods usually assume that the training and test samples are captured in similar scenarios so that their distributions are assumed to be the same.
Multi-Manifold Deep Metric Learning for Image Set Classification
In this paper, we propose a multi-manifold deep metric learning (MMDML) method for image set classification, which aims to recognize an object of interest from a set of image instances captured from varying viewpoints or under varying illuminations.
Deep Hashing for Compact Binary Codes Learning
In this paper, we propose a new deep hashing (DH) approach to learn compact binary codes for large scale visual search.
Learning Compact Binary Descriptors With Unsupervised Deep Neural Networks
In this paper, we propose a new unsupervised deep learning approach called DeepBit to learn compact binary descriptor for efficient visual object matching.
Modality and Component Aware Feature Fusion For RGB-D Scene Classification
While convolutional neural networks (CNN) have been excellent for object recognition, the greater spatial variability in scene images typically meant that the standard full-image CNN features are suboptimal for scene classification.
Learning Deep Binary Descriptor With Multi-Quantization
In this paper, we propose an unsupervised feature learning method called deep binary descriptor with multi-quantization (DBD-MQ) for visual matching.
Consistent-Aware Deep Learning for Person Re-Identification in a Camera Network
In this paper, we propose a consistent-aware deep learning (CADL) framework for person re-identification in a camera network.
Multi-View Complementary Hash Tables for Nearest Neighbor Search
have enjoyed the benefits of complementary hash tables and information fusion over multiple views.
MMSS: Multi-Modal Sharable and Specific Feature Learning for RGB-D Object Recognition
We first construct deep CNN layers for color and depth separately, and then connect them with our carefully designed multi-modal layers, which fuse color and depth information by enforcing a common part to be shared by features of different modalities.
Multiple Feature Fusion via Weighted Entropy for Visual Tracking
It is desirable to combine multiple feature descriptors to improve the visual tracking performance because different features can provide complementary information to describe objects of interest.
Simultaneous Local Binary Feature Learning and Encoding for Face Recognition
In this paper, we propose a simultaneous local binary feature learning and encoding (SLBFLE) method for face recognition.
Local Subspace Collaborative Tracking
To address this, this paper presents a local subspace collaborative tracking method for robust visual tracking, where multiple linear and nonlinear subspaces are learned to better model the nonlinear relationship of object appearances.
Learning Discriminative Aggregation Network for Video-Based Face Recognition
In this paper, we propose a discriminative aggregation network (DAN) for video face recognition, which aims to integrate information from video frames effectively and efficiently.
Attention-Aware Deep Reinforcement Learning for Video Face Recognition
In this paper, we propose an attention-aware deep reinforcement learning (ADRL) method for video face recognition, which aims to discard the misleading and confounding frames and find the focuses of attention in face videos for person recognition.
Cross-Modal Deep Variational Hashing
In this paper, we propose a cross-modal deep variational hashing (CMDVH) method to learn compact binary codes for cross-modality multimedia retrieval.
COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis
There are substantial instructional videos on the Internet, which enables us to acquire knowledge for completing various tasks.
BridgeNet: A Continuity-Aware Probabilistic Network for Age Estimation
Existing methods for age estimation usually apply a divide-and-conquer strategy to deal with heterogeneous data caused by the non-stationary aging process.
Ranked #2 on
Age Estimation
on FGNET
Conditional Single-view Shape Generation for Multi-view Stereo Reconstruction
In this paper, we present a new perspective towards image-based shape generation.
Deep Fitting Degree Scoring Network for Monocular 3D Object Detection
In this paper, we propose to learn a deep fitting degree scoring network for monocular 3D object detection, which aims to score fitting degree between proposals and object conclusively.
Ranked #7 on
Vehicle Pose Estimation
on KITTI Cars Hard
P$^2$GNet: Pose-Guided Point Cloud Generating Networks for 6-DoF Object Pose Estimation
Humans are able to perform fast and accurate object pose estimation even under severe occlusion by exploiting learned object model priors from everyday life.
DotFAN: A Domain-transferred Face Augmentation Network for Pose and Illumination Invariant Face Recognition
With their aid, DotFAN can learn a disentangled face representation and effectively generate face images of various facial attributes while preserving the identity of augmented faces.
Comprehensive Instructional Video Analysis: The COIN Dataset and Performance Evaluation
We believe the introduction of the COIN dataset will promote the future in-depth research on instructional video analysis for the community.
Graph-based Kinship Reasoning Network
In this paper, we propose a graph-based kinship reasoning (GKR) network for kinship verification, which aims to effectively perform relational reasoning on the extracted features of an image pair.
Ranked #3 on
Kinship Verification
on KinFaceW-II
Latent Fingerprint Registration via Matching Densely Sampled Points
Given a pair of fingerprints to match, we bypass the minutiae extraction step and take uniformly sampled points as key points.
MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation
Knowledge Distillation (KD) has been one of the most popu-lar methods to learn a compact model.
Reinforced Axial Refinement Network for Monocular 3D Object Detection
Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image.
Ranked #16 on
Vehicle Pose Estimation
on KITTI Cars Hard
Deep Credible Metric Learning for Unsupervised Domain Adaptation Person Re-identification
Experimental results demonstrate that our DCML method explores credible and valuable training data and improves the performance of unsupervised domain adaptation.
Temporal Coherence or Temporal Motion: Which is More Critical for Video-based Person Re-identification?
Specifically, we disentangle the video representation into the temporal coherence and motion parts and randomly change the scale of the temporal motion features as the adversarial noise.
Structural Deep Metric Learning for Room Layout Estimation
We employ a metric model and a layout encoder to map the RGB images and the ground-truth layouts to the embedding space, respectively, and a layout decoder to map the embeddings to the corresponding layouts, where the whole framework is trained in an end-to-end manner.
Deep Hashing with Active Pairwise Supervision
n this paper, we propose a Deep Hashing method with Active Pairwise Supervision(DH-APS).
Rotation-robust Intersection over Union for 3D Object Detection
In this paper, we propose a Rotation-robust Intersection over Union ($ extit{RIoU}$) for 3D object detection, which aims to jointly learn the overlap of rotated bounding boxes.
Spatial Geometric Reasoning for Room Layout Estimation via Deep Reinforcement Learning
Unlike most existing works that define room layout on a 2D image, we model the layout in 3D as a configuration of the camera and the room.
SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular images
In this paper, we first introduce the concept of semantic objectness to exploit the geometric relationship of these two tasks through an analysis of the imaging process, then propose a Semantic Object Segmentation and Depth Estimation Network (SOSD-Net) based on the objectness assumption.
Ranked #81 on
Semantic Segmentation
on NYU Depth v2
Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search
As hashing becomes an increasingly appealing technique for large-scale image retrieval, multi-label hashing is also attracting more attention for the ability to exploit multi-level semantic contents.
WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition
In this paper, we contribute a new million-scale face benchmark containing noisy 4M identities/260M faces (WebFace260M) and cleaned 2M identities/42M faces (WebFace42M) training data, as well as an elaborately designed time-constrained evaluation protocol.
Ranked #1 on
Face Verification
on IJB-C
(training dataset metric)
Meta-Mining Discriminative Samples for Kinship Verification
In the end, the samples in the unbalanced train batch are re-weighted by the learned meta-miner to optimize the kinship models.
Ranked #1 on
Kinship Verification
on KinFaceW-II
SIMPLE: SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation
To make a better trade-off between accuracy and efficiency, we propose a novel multi-person pose estimation framework, SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation (SIMPLE).
Pseudo Facial Generation With Extreme Poses for Face Recognition
Many of them settle it by generating fake frontal faces from extreme ones, whereas they are tough to maintain the identity information with high computational consumption and uncontrolled disturbances.
Masked Face Recognition Challenge: The WebFace260M Track Report
There are second phase of the challenge till October 1, 2021 and on-going leaderboard.
Reasoning Graph Networks for Kinship Verification: from Star-shaped to Hierarchical
To address this, we propose a Star-shaped Reasoning Graph Network (S-RGN).
Ranked #1 on
Kinship Verification
on KinFaceW-I
Frequency-Aware Spatiotemporal Transformers for Video Inpainting Detection
In this paper, we propose a frequency-aware spatiotemporal transformers for deep In this paper, we propose a Frequency-Aware Spatiotemporal Transformer (FAST) for video inpainting detection, which aims to simultaneously mine the traces of video inpainting from spatial, temporal, and frequency domains.
Adaptive neighborhood Metric learning
In this paper, we reveal that metric learning would suffer from serious inseparable problem if without informative sample mining.
A Roadmap for Big Model
With the rapid development of deep learning, training Big Models (BMs) for multiple downstream tasks becomes a popular paradigm.
HyperDet3D: Learning a Scene-conditioned 3D Object Detector
A bathtub in a library, a sink in an office, a bed in a laundry room -- the counter-intuition suggests that scene provides important prior knowledge for 3D object detection, which instructs to eliminate the ambiguous detection of similar objects.
WebFace260M: A Benchmark for Million-Scale Deep Face Recognition
For a comprehensive evaluation of face matchers, three recognition tasks are performed under standard, masked and unbiased settings, respectively.
Gait Recognition in the Wild: A Large-scale Benchmark and NAS-based Baseline
To the best of our knowledge, this is the first large-scale dataset for gait recognition in the wild.
Relational Reasoning Over Spatial-Temporal Graphs for Video Summarization
Then, we construct a temporal graph by using the aggregated representations of spatial graphs.
Ranked #1 on
Video Summarization
on TvSum
(using extra training data)
Dimension Embeddings for Monocular 3D Object Detection
In this paper, we propose a general method to learn appropriate embeddings for dimension estimation in monocular 3D object detection.
Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value
Explaining deep convolutional neural networks has been recently drawing increasing attention since it helps to understand the networks' internal operations and why they make certain decisions.
Planning Irregular Object Packing via Hierarchical Reinforcement Learning
We compare our approach with existing robotic packing methods for irregular objects in a physics simulator.
FLAG3D: A 3D Fitness Activity Dataset with Language Instruction
With the continuously thriving popularity around the world, fitness activity analytic has become an emerging research topic in computer vision.
Category-level Shape Estimation for Densely Cluttered Objects
Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud.
Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis
In this paper, we propose binary sparse convolutional networks called BSC-Net for efficient point cloud analysis.
Towards Accurate Data-free Quantization for Diffusion Models
On the contrary, we design group-wise quantization functions for activation discretization in different timesteps and sample the optimal timestep for informative calibration image generation, so that our quantized diffusion model can reduce the discretization errors with negligible computational overhead.
DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
The established world model holds immense potential for the generation of high-quality driving videos, and driving policies for safe maneuvering.
CLIP-Cluster: CLIP-Guided Attribute Hallucination for Face Clustering
Furthermore, we develop a neighbor-aware proxy generator that fuses the features describing various attributes into a proxy feature to build a bridge among different sub-clusters and reduce the intra-class variance.
Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning
By this means, our model explores all sorts of reliable sub-relations within an action sequence in the condensed action space.
Anyview: Generalizable Indoor 3D Object Detection with Variable Frames
Existing methods only consider fixed frames of input data for a single detector, such as monocular RGB-D images or point clouds reconstructed from dense multi-view RGB-D images.
ThinkBot: Embodied Instruction Following with Thought Chain Reasoning
Embodied Instruction Following (EIF) requires agents to complete human instruction by interacting objects in complicated surrounding environments.
Neighborhood Repulsed Metric Learning for Kinship Verification
In this paper, we propose a new neighborhood repulsed metric learning (NRML) method for kinship verification.
Ranked #5 on
Kinship Verification
on KinFaceW-I
WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
World models play a crucial role in understanding and predicting the dynamics of the world, which is essential for video generation.
Memory-based Adapters for Online 3D Scene Perception
To this end, we propose an adapter-based plug-and-play module for the backbone of 3D scene perception model, which constructs memory to cache and aggregate the extracted RGB-D features to empower offline models with temporal learning ability.
ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
Then, we build a Gaussian world model to parameterize the distribution in our dynamic Gaussian Splatting framework, which provides informative supervision in the interactive environment via future scene reconstruction.
Learning Dual-Level Deformable Implicit Representation for Real-World Scale Arbitrary Super-Resolution
Scale arbitrary super-resolution based on implicit image function gains increasing popularity since it can better represent the visual world in a continuous manner.
