Search Results for author:
Found 262 papers, 162 papers with code
Preconditioning for Accelerated Iteratively Reweighted Least Squares in Structured Sparsity Reconstruction
In this paper, we propose a novel algorithm for structured sparsity reconstruction.
DeepID-Net: multi-stage and deformable deep convolutional neural networks for object detection
In the proposed new deep architecture, a new deformation constrained pooling (def-pooling) layer models the deformation of object parts with geometric constraint and penalty.
Fast Iteratively Reweighted Least Squares Algorithms for Analysis-Based Sparsity Reconstruction
The convergence rate of the proposed algorithm is almost the same as that of the traditional IRLS algorithms, that is, exponentially fast.
Highly Efficient Forward and Backward Propagation of Convolutional Neural Networks for Pixelwise Classification
The proposed algorithms eliminate all the redundant computation in convolution and pooling on images by introducing novel d-regularly sparse kernels.
DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection
In this paper, we propose deformable deep convolutional neural networks for generic object detection.
Saliency Detection by Multi-Context Deep Learning
Low-level saliency cues or priors do not produce good enough saliency detection results especially when the salient object presents in a low-contrast background with confusing visual appearance.
Understanding Pedestrian Behaviors From Stationary Crowd Groups
Pedestrian behavior modeling and analysis is important for crowd scene understanding and has various applications in video surveillance.
Cross-Scene Crowd Counting via Deep Convolutional Neural Networks
To address this problem, we propose a deep convolutional neural network (CNN) for crowd counting, and it is trained alternatively with two related learning objectives, crowd density and crowd count.
Ranked #15 on
Crowd Counting
on WorldExpo’10
Pedestrian Travel Time Estimation in Crowded Scenes
In this paper, we target on the problem of estimating the statistic of pedestrian travel time within a period from an entrance to a destination in a crowded scene.
Structured Feature Learning for Pose Estimation
In this paper, we propose a structured feature learning framework to reason the correlations among body joints at the feature level in human pose estimation.
T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
Temporal and contextual information of videos are not fully investigated and utilized.
Object Detection from Video Tubelets with Convolutional Neural Networks
Deep Convolution Neural Networks (CNNs) have shown impressive performance in various vision tasks such as image classification, object detection and semantic segmentation.
Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification
Learning generic and robust feature representations with data from multiple domains for the same problem is of great value, especially for the problems that have multiple datasets but none of them are large enough to provide abundant data variations.
End-To-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation
In this paper, we propose a novel end-to-end framework for human pose estimation that combines DCNNs with the expressive deformable mixture of parts.
Crafting GBD-Net for Object Detection
The effectiveness of GBD-Net is shown through experiments on three object detection datasets, ImageNet, Pascal VOC2007 and Microsoft COCO.
CRF-CNN: Modeling Structured Information in Human Pose Estimation
In a classical neural network, there is no message passing between neurons in the same layer.
StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks
Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications.
Ranked #3 on
Text-to-Image Generation
on Oxford 102 Flowers
(Inception score metric)
Person Search with Natural Language Description
Searching persons in large-scale image databases with the query of natural language description has important applications in video surveillance.
Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
Analysis of the learned SRN model demonstrates that it can effectively capture both semantic and spatial relations of labels for improving classification performance.
Ranked #6 on
Multi-Label Classification
on NUS-WIDE
Object Detection in Videos with Tubelet Proposal Networks
Object detection in videos has drawn increasing attention recently with the introduction of the large-scale ImageNet VID dataset.
Learning Deep Representations for Scene Labeling with Semantic Context Guided Supervision
The experiments show that our proposed method makes deep models learn more discriminative feature representations without increasing model size or complexity.
Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection
We propose a convolution neural network based algorithm for simultaneously diagnosing diabetic retinopathy and highlighting suspicious regions.
Learning Feature Pyramids for Human Pose Estimation
We investigate our method on two standard benchmarks for human pose estimation.
Ranked #6 on
Pose Estimation
on Leeds Sports Poses
Identity-Aware Textual-Visual Matching with Latent Co-attention
The stage-2 CNN-LSTM network refines the matching results with a latent co-attention mechanism.
Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism
The visibility map of the target is learned and used for inferring the spatial attention map.
Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
Vehicle re-identification is an important problem and has many applications in video surveillance and intelligent transportation.
Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-Identification
In our vehicle ReID framework, an orientation invariant feature embedding module and a spatial-temporal regularization module are proposed.
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks
In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) aiming at generating high-resolution photo-realistic images.
Ranked #5 on
Text-to-Image Generation
on Oxford 102 Flowers
Co-attending Free-form Regions and Detections with Multi-modal Multiplicative Feature Embedding for Visual Question Answering
Existing VQA methods mainly adopt the visual attention mechanism to associate the input question with corresponding image regions for effective question answering.
Single View Stereo Matching
The resulting model outperforms all the previous monocular depth estimation methods as well as the stereo block matching method in the challenging KITTI dataset by only using a small number of real training data.
Ranked #42 on
Monocular Depth Estimation
on KITTI Eigen split
Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data
The aim of image captioning is to generate captions by machine to describe image contents.
3D Human Pose Estimation in the Wild by Adversarial Learning
Instead of defining hard-coded rules to constrain the pose estimation results, we design a novel multi-source discriminator to distinguish the predicted 3D poses from the ground-truth, which helps to enforce the pose estimator to generate anthropometrically valid poses even with images in the wild.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Learnable Histogram: Statistical Context Features for Deep Neural Networks
Statistical features, such as histogram, Bag-of-Words (BoW) and Fisher Vector, were commonly used with hand-crafted features in conventional classification methods, but attract less attention since the popularity of deep learning methods.
Video Person Re-Identification With Competitive Snippet-Similarity Aggregation and Co-Attentive Snippet Embedding
The attention weights are obtained based on a query feature, which is learned from the whole probe snippet by an LSTM network, making the resulting embeddings less affected by noisy frames.
Ranked #4 on
Person Re-Identification
on PRID2011
Eliminating Background-Bias for Robust Person Re-Identification
State-of-the-art methods mainly utilize deep learning based approaches for learning visual features for describing person appearances.
Group Consistent Similarity Learning via Deep CRF for Person Re-Identification
Extensive experiments demonstrate the effectiveness of our model that combines DNN and CRF for learning robust multi-scale local similarities.
Deep Continuous Conditional Random Fields with Asymmetric Inter-object Constraints for Online Multi-object Tracking
In addition, inter-object relations are mostly modeled in a symmetric way, which we argue is not an optimal setting.
Person Re-identification with Deep Similarity-Guided Graph Neural Network
However, existing person re-identification models mostly estimate the similarities of different image pairs of probe and gallery images independently while ignores the relationship information between different probe-gallery pairs.
Ranked #2 on
Person Re-Identification
on CUHK03
End-to-End Deep Kronecker-Product Matching for Person Re-identification
Person re-identification aims to robustly measure similarities between person images.
Deep Group-shuffling Random Walk for Person Re-identification
Person re-identification aims at finding a person of interest in an image gallery by comparing the probe image of this person with all the gallery images.
Generative Adversarial Frontal View to Bird View Synthesis
Due to the large gap and severe deformation between the frontal view and bird view, generating a bird view image from a single frontal view is challenging.
Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association
Person re-identification is an important task that requires learning discriminative visual features for distinguishing different person identities.
Ranked #22 on
Text based Person Retrieval
on CUHK-PEDES
Question-Guided Hybrid Convolution for Visual Question Answering
Most state-of-the-art VQA methods fuse the high-level textual and visual features from the neural network and abandon the visual spatial information when learning multi-modal features. To address these problems, question-guided kernels generated from the input question are designed to convolute with visual features for capturing the textual and visual relationship in the early stage.
Ranked #14 on
Visual Question Answering (VQA)
on CLEVR
Learning Monocular Depth by Distilling Cross-domain Stereo Networks
Monocular depth estimation aims at estimating a pixelwise depth map for a single image, which has wide applications in scene understanding and autonomous driving.
HMS-Net: Hierarchical Multi-scale Sparsity-invariant Network for Sparse Depth Completion
Dense depth cues are important and have wide applications in various computer vision tasks.
Ranked #10 on
Depth Completion
on KITTI Depth Completion
FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification
Our proposed FD-GAN achieves state-of-the-art performance on three person reID datasets, which demonstrates that the effectiveness and robust feature distilling capability of the proposed FD-GAN.
Ranked #3 on
Person Re-Identification
on CUHK03
Efficient Attention: Attention with Linear Complexities
Dot-product attention has wide applications in computer vision and natural language processing.
Ranked #2 on
Extractive Text Summarization
on GovReport
PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
In this paper, we propose PointRCNN for 3D object detection from raw point cloud.
Ranked #2 on
Object Detection
on KITTI Cars Moderate
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
A^2-Net: Molecular Structure Estimation from Cryo-EM Density Volumes
Constructing of molecular structural models from Cryo-Electron Microscopy (Cryo-EM) density volumes is the critical last step of structure determination by Cryo-EM technologies.
Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
Referring expression grounding aims at locating certain objects or persons in an image with a referring expression, where the key challenge is to comprehend and align various types of information from visual and textual domain, such as visual attributes, location and interactions with surrounding regions.
Unsupervised Cross-spectral Stereo Matching by Learning to Synthesize
Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any supervision in the form of ground truth disparity or depth.
Group-wise Correlation Stereo Network
Previous works built cost volumes with cross-correlation or concatenation of left and right features across all disparity levels, and then a 2D or 3D convolutional neural network is utilized to regress the disparity maps.
Conditional Adversarial Generative Flow for Controllable Image Synthesis
Flow-based generative models show great potential in image synthesis due to its reversible pipeline and exact log-likelihood target, yet it suffers from weak ability for conditional image synthesis, especially for multi-label or unaware conditions.
Generalizing Monocular 3D Human Pose Estimation in the Wild
We observe that recent innovation in this area mainly focuses on new techniques that explicitly address the generalization issue when using this dataset, because this database is constructed in a highly controlled environment with limited human subjects and background variations.
Ranked #64 on
3D Human Pose Estimation
on Human3.6M
AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations
Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy.
Ranked #6 on
Face Verification
on MegaFace
P2SGrad: Refined Gradients for Optimizing Deep Face Models
Cosine-based softmax losses significantly improve the performance of deep face recognition networks.
From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications.
Signet Ring Cell Detection With a Semi-supervised Learning Framework
Our framework achieves accurate signet ring cell detection and can be readily applied in the clinical trails.
FocusNet: Imbalanced Large and Small Organ Segmentation with an End-to-End Deep Neural Network for Head and Neck CT Images
In this paper, we propose an end-to-end deep neural network for solving the problem of imbalanced large and small organ segmentation in head and neck (HaN) CT images.
Multi-modality Latent Interaction Network for Visual Question Answering
The proposed module learns the cross-modality relationships between latent visual and language summarizations, which summarize visual regions and question into a small number of latent representations to avoid modeling uninformative individual region-word relations.
Interpolated Convolutional Networks for 3D Point Cloud Understanding
Our InterpConv is shown to be permutation and sparsity invariant, and can directly handle irregular inputs.
Ranked #27 on
3D Part Segmentation
on ShapeNet-Part
CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval
Text-image cross-modal retrieval is a challenging task in the field of language and vision.
Ranked #9 on
Image Retrieval
on Flickr30K 1K test
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
Semantic image synthesis aims at generating photorealistic images from semantic layouts.
Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints
Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds.
Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification
In order to mitigate the effects of noisy pseudo labels, we propose to softly refine the pseudo labels in the target domain by proposing an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels in an alternative training manner.
Structure-Feature based Graph Self-adaptive Pooling
To solve these problems mentioned above, we propose a novel graph self-adaptive pooling method with the following objectives: (1) to construct a reasonable pooled graph topology, structure and feature information of the graph are considered simultaneously, which provide additional veracity and objectivity in node selection; and (2) to make the pooled nodes contain sufficiently effective graph information, node feature information is aggregated before discarding the unimportant nodes; thus, the selected nodes contain information from neighbor nodes, which can enhance the use of features of the unselected nodes.
Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimation
Monocular 3D object detection task aims to predict the 3D bounding boxes of objects based on monocular RGB images.
MagnifierNet: Towards Semantic Adversary and Fusion for Person Re-identification
Meanwhile, we introduce "Semantic Fusion Branch" to filter out irrelevant noises by selectively fusing semantic region information sequentially.
Structured Domain Adaptation with Online Relation Regularization for Unsupervised Person Re-ID
To tackle the challenges, we propose an end-to-end structured domain adaptation framework with an online relation-consistency regularization term.
Ranked #4 on
Unsupervised Domain Adaptation
on Market to MSMT
3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior
To this end, we first propose a novel 3D sketch-aware feature embedding to explicitly encode geometric information effectively and efficiently.
3D Semantic Scene Completion from a single RGB image
Hallucination
Learning to Predict Context-adaptive Convolution for Semantic Segmentation
Long-range contextual information is essential for achieving high-performance semantic segmentation.
StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching
Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias.
Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID
To solve these problems, we propose a novel self-paced contrastive learning framework with hybrid memory.
Ranked #3 on
Unsupervised Domain Adaptation
on Market to MSMT
Self-supervising Fine-grained Region Similarities for Large-scale Image Localization
The task of large-scale retrieval-based image localization is to estimate the geographical location of a query image by recognizing its nearest reference images from a city-scale dataset.
Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization
We propose to explicitly model the Actor-Context-Actor Relation, which is the relation between two actors based on their interactions with the context.
Ranked #2 on
Action Recognition
on AVA v2.1
1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020
This technical report introduces our winning solution to the spatio-temporal action localization track, AVA-Kinetics Crossover, in ActivityNet Challenge 2020.
Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers
Given an input video, its associated audio, and a brief caption, the audio-visual scene aware dialog (AVSD) task requires an agent to indulge in a question-answer dialog with a human about the audio-visual content.
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
Depth information has proven to be a useful cue in the semantic segmentation of RGB-D images for providing a geometric counterpart to the RGB representation.
Complementary Boundary Generator with Scale-Invariant Relation Modeling for Temporal Action Localization: Submission to ActivityNet Challenge 2020
Specifically, in order to generate high-quality proposals, we consider several factors including the video feature encoder, the proposal generator, the proposal-proposal relations, the scale imbalance, and ensemble strategy.
Balanced Meta-Softmax for Long-Tailed Visual Recognition
In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.
Ranked #7 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space.
Ranked #11 on
LIDAR Semantic Segmentation
on nuScenes
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
We propose a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions.
Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets
In most scenarios, one might obtain annotations of a single or a few organs from one training set, and obtain annotations of the the other organs from another set of training images.
EfficientFCN: Holistically-guided Decoding for Semantic Segmentation
State-of-the-art semantic segmentation algorithms are mostly based on dilated Fully Convolutional Networks (dilatedFCN), which adopt dilated convolutions in the backbone networks to extract high-resolution feature maps for achieving high-performance segmentation performance.
PV-RCNN: The Top-Performing LiDAR-only Solutions for 3D Detection / 3D Tracking / Domain Adaptation of Waymo Open Dataset Challenges
In this technical report, we present the top-performing LiDAR-only solutions for 3D detection, 3D tracking and domain adaptation three tracks in Waymo Open Dataset Challenges 2020.
SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.
End-to-End Object Detection with Adaptive Clustering Transformer
In this paper, a novel variant of transformer named Adaptive Clustering Transformer(ACT) has been proposed to reduce the computation cost for high-resolution input.
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited.
Ranked #2 on
3D Semantic Segmentation
on ScribbleKITTI
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
2) Dynamic Shifting for complex point distributions.
Ranked #2 on
Panoptic Segmentation
on SemanticKITTI
REFINE: Prediction Fusion Network for Panoptic Segmentation
Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results.
Ranked #11 on
Panoptic Segmentation
on COCO test-dev
A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection
With the proposed holistically-guided decoder, we implement the EfficientFCN architecture for semantic segmentation and HGD-FPN for object detection and instance segmentation.
Towards Overcoming False Positives in Visual Relationship Detection
We observe that during training, the relationship proposal distribution is highly imbalanced: most of the negative relationship proposals are easy to identify, e. g., the inaccurate object detection, which leads to the under-fitting of low-frequency difficult proposals.
Self-supervised Temporal Learning
Inspired by spatial-based contrastive SSL, we show that significant improvement can be achieved by a proposed temporal-based contrastive learning approach, which includes three novel and efficient modules: temporal augmentations, temporal memory bank and SSTL loss.
Progressive Correspondence Pruning by Consensus Learning
Correspondence selection aims to correctly select the consistent matches (inliers) from an initial set of putative correspondences.
Fast Convergence of DETR with Spatially Modulated Co-Attention
The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN.
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields.
Ranked #2 on
3D Object Detection
on KITTI Cars Easy val
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network, which can maintain the advantages of both unstructured fine-grained sparsity and structured coarse-grained sparsity simultaneously on specifically designed GPUs.
ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection
Then, the detector is iteratively improved on the target domain by alternatively conducting two steps, which are the pseudo label updating with the developed quality-aware triplet memory bank and the model training with curriculum data augmentation.
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network
Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse.
AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks
However, the automatic design of loss functions for generic tasks with various evaluation metrics remains under-investigated.
Fixing the Teacher-Student Knowledge Discrepancy in Distillation
To solve this problem, in this paper, we propose a novel student-dependent distillation method, knowledge consistent distillation, which makes teacher's knowledge more consistent with the student and provides the best suitable knowledge to different student networks for distillation.
FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
Radiotherapy is a treatment where radiation is used to eliminate cancer cells.
LIFE: Lighting Invariant Flow Estimation
However, local image contents are inevitably ambiguous and error-prone during the cross-image feature matching process, which hinders downstream tasks.
Semantic Scene Completion via Integrating Instances and Scene in-the-Loop
The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the overall scene.
Ranked #1 on
3D Semantic Scene Completion
on NYUv2
Decoupled Spatial-Temporal Transformer for Video Inpainting
Seamless combination of these two novel designs forms a better spatial-temporal attention scheme and our proposed model achieves better performance than state-of-the-art video inpainting approaches with significant boosted efficiency.
Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
In this way, our BAKE framework achieves online knowledge ensembling across multiple samples with only a single network.
Inverting Generative Adversarial Renderer for Face Reconstruction
Since the GAR learns to model the complicated real-world image, instead of relying on the simplified graphics rules, it is capable of producing realistic images, which essentially inhibits the domain-shift noise in training and optimization.
VS-Net: Voting with Segmentation for Visual Localization
To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks.
FNAS: Uncertainty-Aware Fast Neural Architecture Search
Further, an architecture knowledge pool together with a block similarity function is proposed to utilize parameter knowledge and reduces the searching time by 2 times.
Container: Context Aggregation Network
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
Ranked #462 on
Image Classification
on ImageNet
Scalable Transformers for Neural Machine Translation
In this paper, we propose a novel Scalable Transformers, which naturally contains sub-Transformers of different scales and have shared parameters.
Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification
Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations.
Hybrid Supervision Learning for Pathology Whole Slide Image Classification
To handle this problem, we propose a hybrid supervision learning framework for this kind of high resolution images with sufficient image-level coarse annotations and a few pixel-level fine labels.
Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification
With the contribution of the CCD and CRP, our CRCKD algorithm can distill the relational knowledge more comprehensively.
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
Fast Convergence of DETR with Spatially Modulated Co-Attention
However, DETR suffers from its slow convergence.
Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization
In this paper, we present a framework named FAC-Net based on the I3D backbone, on which three branches are appended, named class-wise foreground classification branch, class-agnostic attention branch and multiple instance learning branch.
ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection
These specific designs enable the detector to be trained on meticulously refined pseudo labeled target data with denoised training signals, and thus effectively facilitate adapting an object detector to a target domain without requiring annotations.
LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation
However, two major issues of the fusion between camera and LiDAR hinder its performance, \ie, how to effectively fuse these two modalities and how to precisely align them (suffering from the weak spatiotemporal synchronization problem).
LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector
Compared with the state-of-the-art stereo detector, our method has improved the 3D detection performance of cars, pedestrians, cyclists by 10. 44%, 5. 69%, 5. 97% mAP respectively on the official KITTI benchmark.
Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
With the effective spatio-temporal relationship modeling, it is possible not only to uncover contextual information in each frame but also to directly capture inter-time dependencies.
Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation
To this end, we propose Multi-level Attention Encoder-Decoder Network (MAED), including a Spatial-Temporal Encoder (STE) and a Kinematic Topology Decoder (KTD) to model multi-level attentions in a unified framework.
Ranked #40 on
3D Human Pose Estimation
on MPI-INF-3DHP
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
On the contrary, the soft composition operates by stitching different patches into a whole feature map where pixels in overlapping regions are summed up.
Ranked #3 on
Video Inpainting
on DAVIS
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
In this paper, we benchmark our model on these three tasks.
UniFormer: Unified Transformer for Efficient Spatial-Temporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 8% and 71. 4% top-1 accuracy respectively.
Ranked #8 on
Action Recognition
on Something-Something V1
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Recently, transformer and multi-layer perceptron (MLP) architectures have achieved impressive results on various vision tasks.
Ranked #236 on
Image Classification
on ImageNet
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Large-scale contrastive vision-language pre-training has shown significant progress in visual representation learning.
Rethinking Noise Synthesis and Modeling in Raw Denoising
However, the real raw image noise is contributed by many noise sources and varies greatly among different sensors.
Ranked #2 on
Image Denoising
on SID SonyA7S2 x100
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling
To further enhance CLIP's few-shot capability, CLIP-Adapter proposed to fine-tune a lightweight residual feature adapter and significantly improves the performance for few-shot classification.
IDR: Self-Supervised Image Denoising via Iterative Data Refinement
To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
A Simple Long-Tailed Recognition Baseline via Vision-Language Model
Recent advances in large-scale contrastive visual-language pretraining shed light on a new pathway for visual recognition.
Ranked #4 on
Long-tail Learning
on Places-LT
(using extra training data)
DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks
However, the existing N:M algorithms only address the challenge of how to train N:M sparse neural networks in a uniform fashion (i. e. every layer has the same N:M sparsity) and suffer from a significant accuracy drop for high sparsity (i. e. when sparsity > 80\%).
Container: Context Aggregation Networks
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks
The model is pre-trained on several uni-modal and multi-modal tasks, and evaluated on a variety of downstream tasks, including novel tasks that did not appear in the pre-training stage.
PointCLIP: Point Cloud Understanding by CLIP
On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.
Ranked #3 on
3D Open-Vocabulary Instance Segmentation
on STPLS3D
3D Open-Vocabulary Instance Segmentation
Few-Shot Learning
+6
Pyramid Fusion Transformer for Semantic Segmentation
The recently proposed MaskFormer gives a refreshed perspective on the task of semantic segmentation: it shifts from the popular pixel-level classification paradigm to a mask-level classification method.
UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 9% and 71. 2% top-1 accuracy respectively.
UniFormer: Unifying Convolution and Self-attention for Visual Recognition
Different from the typical transformer blocks, the relation aggregators in our UniFormer block are equipped with local and global token affinity respectively in shallow and deep layers, allowing to tackle both redundancy and dependency for efficient and effective representation learning.
Ranked #151 on
Image Classification
on ImageNet
Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning
Especially, we develop a variant of ViT for 3D point cloud feature extraction, which also achieves comparable results with existing backbones when combined with our framework, and visualization of the attention maps show that our model does understand the point cloud by combining the global shape information and multiple local structural information, which is consistent with the inspiration of our representation learning method.
Meta Knowledge Distillation
Recent studies pointed out that knowledge distillation (KD) suffers from two degradation problems, the teacher-student gap and the incompatibility with strong data augmentations, making it not applicable to training state-of-the-art models, which are trained with advanced augmentations.
Ranked #133 on
Image Classification
on ImageNet
Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling
The correct ego-motion estimation basically relies on the understanding of correspondences between adjacent LiDAR scans.
Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation
Our method seeks to mine the representative snippets in each video for propagating information between video snippets to generate better pseudo labels.
Pseudo Label
Weakly-supervised Temporal Action Localization
+1
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
MonoDETR: Depth-guided Transformer for Monocular 3D Object Detection
In this paper, we introduce the first DETR framework for Monocular DEtection with a depth-guided TRansformer, named MonoDETR.
3D Object Detection From Monocular Images
Autonomous Driving
+3
RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization
The correspondence field estimation and pose refinement are conducted alternatively in each iteration to recover the object poses.
Ranked #1 on
6D Pose Estimation using RGB
on LineMOD
Learning a Structured Latent Space for Unsupervised Point Cloud Completion
Specifically, we map a series of related partial point clouds into multiple complete shape and occlusion code pairs and fuse the codes to obtain their representations in the unified latent space.
FlowFormer: A Transformer Architecture for Optical Flow
We introduce optical Flow transFormer, dubbed as FlowFormer, a transformer-based neural network architecture for learning optical flow.
Ranked #1 on
Optical Flow Estimation
on Sintel-final
RBGNet: Ray-based Grouping for 3D Object Detection
In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers.
Ranked #13 on
3D Object Detection
on ScanNetV2
Simulating Fluids in Real-World Still Images
In this work, we tackle the problem of real-world fluid animation from a still image.
Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis
Specifically, we compress the light fields for novel view human rendering as conditional implicit neural radiance fields from both geometry and appearance aspects.
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers
In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention-based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency.
ConvMAE: Masked Convolution Meets Masked Autoencoders
Masked auto-encoding for feature pretraining and multi-scale hybrid convolution-transformer architectures can further unleash the potentials of ViT, leading to state-of-the-art performances on image classification, detection and semantic segmentation.
Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network
As for each sub-network, we propose an efficient multi-frequency denoising network to remove noise of different frequencies.
MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection
Accurate and reliable 3D detection is vital for many applications including autonomous driving vehicles and service robots.
MixMAE: Mixed and Masked Autoencoder for Efficient Pretraining of Hierarchical Vision Transformers
In this paper, we propose Mixed and Masked AutoEncoder (MixMAE), a simple but efficient pretraining method that is applicable to various hierarchical Vision Transformers.
Ranked #2 on
Image Classification
on Places205
Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training
By fine-tuning on downstream tasks, Point-M2AE achieves 86. 43% accuracy on ScanObjectNN, +3. 36% to the second-best, and largely benefits the few-shot classification, part segmentation and 3D object detection with the hierarchical pre-training scheme.
Ranked #4 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object Interaction detection
However, there is still room for improvement in video HOI detection performance.
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs
To mitigate such interference, we introduce the Conditional Mixture-of-Experts (Conditional MoEs) to generalist models.
Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
3D Object Detection for Autonomous Driving: A Comprehensive Survey
Autonomous driving, in recent years, has been receiving increasing attention for its potential to relieve drivers' burdens and improve the safety of driving.
A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift
In this study, we propose a simple yet effective framework for video restoration.
Ranked #1 on
Deblurring
on GoPro
(using extra training data)
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning
This has led to a new research direction in parameter-efficient transfer learning.
Ranked #23 on
Action Recognition
on Something-Something V2
(using extra training data)
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Finally, we integrate configurable operators and DSMs into a unified search space and search with a Reinforcement Learning-based search algorithm to fully explore the optimal combination of the operators.
Ranked #12 on
Neural Architecture Search
on ImageNet
TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers
In this paper, we propose a novel data augmentation technique TokenMix to improve the performance of vision transformers.
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
On top of that, the performance of Tip-Adapter can be further boosted to be state-of-the-art on ImageNet by fine-tuning the cache model for 10$\times$ fewer epochs than existing methods, which is both effective and efficient.
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer
Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns.
Ranked #2 on
Autonomous Driving
on CARLA Leaderboard
Frozen CLIP Models are Efficient Video Learners
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos.
Ranked #26 on
Action Classification
on Kinetics-400
(using extra training data)
Learning Degradation Representations for Image Deblurring
With the integration, MSDI-Net can handle various and complicated blurry patterns adaptively.
Ranked #13 on
Image Deblurring
on GoPro
Towards Robust Face Recognition with Comprehensive Search
To eliminate the bias of single-aspect research and provide an overall understanding of the face recognition model design, we first carefully design the search space for each aspect, then a comprehensive search method is introduced to jointly search optimal data cleaning, architecture, and loss function design.
Magnetic Resonance Fingerprinting with compressed sensing and distance metric learning
Adaptively learning a distance metric from the undersampled training data can significantly improve the matching accuracy of the query fingerprints.
NeuralMarker: A Framework for Learning General Marker Correspondence
We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker.
Collaboration of Pre-trained Models Makes Better Few-shot Learner
In this paper, we propose CoMo, a Collaboration of pre-trained Models that incorporates diverse prior knowledge from various pre-training paradigms for better few-shot learning.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(AP50 metric, using extra
training data)
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
In this paper, we propose Uni-Perceiver v2, which is the first generalist model capable of handling major large-scale vision and vision-language tasks with competitive performance.
Teach-DETR: Better Training DETR with Teachers
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors.
CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders
Pre-training by numerous image data has become de-facto for robust 2D representations.
Ranked #2 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
ConQueR: Query Contrast Voxel-DETR for 3D Object Detection
We thus propose a Query Contrast mechanism to explicitly enhance queries towards their best-matched GTs over all unmatched query predictions.
Starting From Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
Adaptive Zone-Aware Hierarchical Planner for Vision-Language Navigation
In this paper, we propose an Adaptive Zone-aware Hierarchical Planner (AZHP) to explicitly divides the navigation process into two heterogeneous phases, i. e., sub-goal setting via zone partition/selection (high-level action) and sub-goal executing (low-level action), for hierarchical planning.
SparseMAE: Sparse Training Meets Masked Autoencoders
In this paper, we aim to reduce model complexity from large vision transformers pretrained by MAE with assistant of sparse training.
FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
FlowFormer introduces a transformer architecture into optical flow estimation and achieves state-of-the-art performance.
FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR).
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners
Our CaFo incorporates CLIP's language-contrastive knowledge, DINO's vision-contrastive knowledge, DALL-E's vision-generative knowledge, and GPT-3's language-generative knowledge.
KBNet: Kernel Basis Network for Image Restoration
In this paper, we propose a kernel basis attention (KBA) module, which introduces learnable kernel bases to model representative image patterns for spatial information aggregation.
Ranked #1 on
Color Image Denoising
on McMaster sigma50
Mimic before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking
To alleviate this, previous methods simply replace the pixel reconstruction targets of 75% masked tokens by encoded features from pre-trained image-image (DINO) or image-language (CLIP) contrastive learning.
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
BlinkSim consists of a configurable rendering engine and a flexible engine for event data simulation.
Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
Ranked #1 on
Training-free 3D Part Segmentation
on ShapeNet-Part
3D Point Cloud Classification
Training-free 3D Part Segmentation
+1
VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation
We first propose a TRi-frame Optical Flow (TROF) module that estimates bi-directional optical flows for the center frame in a three-frame manner.
GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding
In this paper, we propose Geometry Enhanced Masked Image Modeling (GeoMIM) to transfer the knowledge of the LiDAR model in a pretrain-finetune paradigm for improving the multi-view camera-based 3D detection.
CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching
To overcome these obstacles, we propose CORA, a DETR-style framework that adapts CLIP for Open-vocabulary detection by Region prompting and Anchor pre-matching.
Ranked #6 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
Human Preference Score: Better Aligning Text-to-Image Models with Human Preference
To address this issue, we collect a dataset of human choices on generated images from the Stable Foundation Discord channel.
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model.
Ranked #2 on
Music Question Answering
on MusicQA
Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction
The HoP approach is straightforward: given the current timestamp t, we generate a pseudo Bird's-Eye View (BEV) feature of timestamp t-k from its adjacent frames and utilize this feature to predict the object set at timestamp t-k. Our approach is motivated by the observation that enforcing the detector to capture both the spatial location and temporal motion of objects occurring at historical timestamps can lead to more accurate BEV feature learning.
Ranked #3 on
3D Object Detection
on nuScenes Camera Only
Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels
Besides, the generated pseudo-labels can be fluctuating and inaccurate at the early stage of training.
Pseudo Label
Weakly-supervised Temporal Action Localization
+1
Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehicles
We propose a perception imitation method to simulate results of a certain perception model, and discuss a new heuristic route of autonomous driving simulator without data synthesis.
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
This strategy effectively alleviates the interference between the two tasks of image-text alignment and instruction following and achieves strong multi-modal reasoning with only a small-scale image-text and instruction dataset.
Ranked #6 on
Visual Question Answering (VQA)
on InfiMM-Eval
Instruction Following
Optical Character Recognition (OCR)
+7
