Search Results for author:
Found 194 papers, 112 papers with code
RBF-Softmax: Learning Deep Representative Prototypes with Radial Basis Function Softmax
To address this problem, this paper introduces a novel Radial Basis Function (RBF) distances to replace the commonly used inner products in the softmax loss function, such that it can adaptively assign losses to regularize the intra-class and inter-class distances by reshaping the relative differences, and thus creating more representative prototypes of classes to improve optimization.
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model.
Better Aligning Text-to-Image Models with Human Preference
Using the HPS, we propose a simple yet effective method to adapt Stable Diffusion to better align with human aesthetic preferences.
CORA: Adapting CLIP for Open-Vocabulary Detection with Region Prompting and Anchor Pre-Matching
To overcome these obstacles, we propose CORA, a DETR-style framework that adapts CLIP for Open-vocabulary detection by Region prompting and Anchor pre-matching.
Ranked #1 on
Open Vocabulary Object Detection
on MSCOCO
(using extra training data)
Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding
In this paper, we propose Geometry Enhanced Masked Image Modeling (GeoMIM) to transfer the knowledge of the LiDAR model in a pretrain-finetune paradigm for improving the multi-view camera-based 3D detection.
VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation
We first propose a TRi-frame Optical Flow (TROF) module that estimates bi-directional optical flows for the center frame in a three-frame manner.
Parameter is Not All You Need: Starting from Non-Parametric Networks for 3D Point Cloud Analysis
We present a Non-parametric Network for 3D point cloud analysis, Point-NN, which consists of purely non-learnable components: farthest point sampling (FPS), k-nearest neighbors (k-NN), and pooling operations, with trigonometric functions.
Ranked #1 on
Training-free 3D Part Segmentation
on ShapeNet-Part
3D Point Cloud Classification
Training-free 3D Part Segmentation
+1
PATS: Patch Area Transportation with Subdivision for Local Feature Matching
However, estimating scale differences between these patches is non-trivial since the scale differences are determined by both relative camera poses and scene structures, and thus spatially varying over image pairs.
BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
BlinkSim consists of a configurable rendering engine and a flexible engine for event data simulation.
Mimic before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking
To alleviate this, previous methods simply replace the pixel reconstruction targets of 75% masked tokens by encoded features from pre-trained image-image (DINO) or image-language (CLIP) contrastive learning.
KBNet: Kernel Basis Network for Image Restoration
In this paper, we propose a kernel basis attention (KBA) module, which introduces learnable kernel bases to model representative image patterns for spatial information aggregation.
Ranked #1 on
Color Image Denoising
on McMaster sigma50
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners
Our CaFo incorporates CLIP's language-contrastive knowledge, DINO's vision-contrastive knowledge, DALL-E's vision-generative knowledge, and GPT-3's language-generative knowledge.
FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
FlowFormer introduces a transformer architecture into optical flow estimation and achieves state-of-the-art performance.
FeatAug-DETR: Enriching One-to-Many Matching for DETRs with Feature Augmentation
The first method is One-to-many Matching via Data Augmentation (denoted as DataAug-DETR).
ConQueR: Query Contrast Voxel-DETR for 3D Object Detection
We thus propose a Query Contrast mechanism to explicitly enhance queries towards their best-matched GTs over all unmatched query predictions.
Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders
Pre-training by numerous image data has become de-facto for robust 2D representations.
Ranked #1 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
CGOF++: Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
Teach-DETR: Better Training DETR with Teachers
In this paper, we present a novel training scheme, namely Teach-DETR, to learn better DETR-based detectors from versatile teacher detectors.
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
In this paper, we propose Uni-Perceiver v2, which is the first generalist model capable of handling major large-scale vision and vision-language tasks with competitive performance.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Object Detection
on PASCAL VOC 2007
Collaboration of Pre-trained Models Makes Better Few-shot Learner
In this paper, we propose CoMo, a Collaboration of pre-trained Models that incorporates diverse prior knowledge from various pre-training paradigms for better few-shot learning.
Magnetic Resonance Fingerprinting with compressed sensing and distance metric learning
Adaptively learning a distance metric from the undersampled training data can significantly improve the matching accuracy of the query fingerprints.
NeuralMarker: A Framework for Learning General Marker Correspondence
We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker.
Towards Robust Face Recognition with Comprehensive Search
To eliminate the bias of single-aspect research and provide an overall understanding of the face recognition model design, we first carefully design the search space for each aspect, then a comprehensive search method is introduced to jointly search optimal data cleaning, architecture, and loss function design.
Learning Degradation Representations for Image Deblurring
With the integration, MSDI-Net can handle various and complicated blurry patterns adaptively.
Frozen CLIP Models are Efficient Video Learners
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos.
Ranked #14 on
Action Classification
on Kinetics-400
(using extra training data)
Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer
Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns.
Ranked #1 on
Autonomous Driving
on CARLA Leaderboard
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
On top of that, the performance of Tip-Adapter can be further boosted to be state-of-the-art on ImageNet by fine-tuning the cache model for 10$\times$ fewer epochs than existing methods, which is both effective and efficient.
TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformers
In this paper, we propose a novel data augmentation technique TokenMix to improve the performance of vision transformers.
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Finally, we integrate configurable operators and DSMs into a unified search space and search with a Reinforcement Learning-based search algorithm to fully explore the optimal combination of the operators.
Ranked #12 on
Neural Architecture Search
on ImageNet
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning
This has led to a new research direction in parameter-efficient transfer learning.
Ranked #13 on
Action Recognition
on Something-Something V2
(using extra training data)
No Attention is Needed: Grouped Spatial-temporal Shift for Simple and Efficient Video Restorers
The key of our framework is the grouped spatial-temporal shift, which is simple and lightweight, but can implicitly establish inter-frame correspondences and achieve multi-frame aggregation.
3D Object Detection for Autonomous Driving: A Review and New Outlooks
Autonomous driving, in recent years, has been receiving increasing attention for its potential to relieve drivers' burdens and improve the safety of driving.
Controllable 3D Face Synthesis with Conditional Generative Occupancy Fields
Capitalizing on the recent advances in image generation models, existing controllable face image synthesis methods are able to generate high-fidelity images with some levels of controllability, e. g., controlling the shapes, expressions, textures, and poses of the generated face images.
Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs
To mitigate such interference, we introduce the Conditional Mixture-of-Experts (Conditional MoEs) to generalist models.
Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object Interaction detection
However, there is still room for improvement in video HOI detection performance.
Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training
By fine-tuning on downstream tasks, Point-M2AE achieves 86. 43% accuracy on ScanObjectNN, +3. 36% to the second-best, and largely benefits the few-shot classification, part segmentation and 3D object detection with the hierarchical pre-training scheme.
Ranked #2 on
Few-Shot 3D Point Cloud Classification
on ModelNet40 5-way (20-shot)
(using extra training data)
MixMIM: Mixed and Masked Image Modeling for Efficient Visual Representation Learning
In this study, we propose Mixed and Masked Image Modeling (MixMIM), a simple but efficient MIM method that is applicable to various hierarchical Vision Transformers.
Ranked #1 on
Object Detection
on COCO 2017
(mAP metric)
MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection
Accurate and reliable 3D detection is vital for many applications including autonomous driving vehicles and service robots.
Efficient Burst Raw Denoising with Variance Stabilization and Multi-frequency Denoising Network
As for each sub-network, we propose an efficient multi-frequency denoising network to remove noise of different frequencies.
ConvMAE: Masked Convolution Meets Masked Autoencoders
Masked auto-encoding for feature pretraining and multi-scale hybrid convolution-transformer architectures can further unleash the potentials of ViT, leading to state-of-the-art performances on image classification, detection and semantic segmentation.
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers
In this work, pushing further along this under-studied direction we introduce EdgeViTs, a new family of light-weight ViTs that, for the first time, enable attention-based vision models to compete with the best light-weight CNNs in the tradeoff between accuracy and on-device efficiency.
Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis
Specifically, we compress the light fields for novel view human rendering as conditional implicit neural radiance fields from both geometry and appearance aspects.
Simulating Fluids in Real-World Still Images
In this work, we tackle the problem of real-world fluid animation from a still image.
RBGNet: Ray-based Grouping for 3D Object Detection
In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers.
Ranked #7 on
3D Object Detection
on ScanNetV2
FlowFormer: A Transformer Architecture for Optical Flow
We introduce optical Flow transFormer, dubbed as FlowFormer, a transformer-based neural network architecture for learning optical flow.
Ranked #1 on
Optical Flow Estimation
on Sintel-final
Learning a Structured Latent Space for Unsupervised Point Cloud Completion
Specifically, we map a series of related partial point clouds into multiple complete shape and occlusion code pairs and fuse the codes to obtain their representations in the unified latent space.
MonoDETR: Depth-guided Transformer for Monocular 3D Object Detection
In this paper, we introduce a novel framework for Monocular DEtection with a depth-guided TRansformer, named MonoDETR.
RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimization
The correspondence field estimation and pose refinement are conducted alternatively in each iteration to recover the object poses.
Ranked #1 on
6D Pose Estimation using RGB
on LineMOD
LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network
In this work, we address the task of LiDAR-based panoptic segmentation, which aims to parse both objects and scenes in a unified manner.
Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation
Our method seeks to mine the representative snippets in each video for propagating information between video snippets to generate better pseudo labels.
Pseudo Label
Weakly-supervised Temporal Action Localization
+1
Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modeling
The correct ego-motion estimation basically relies on the understanding of correspondences between adjacent LiDAR scans.
Meta Knowledge Distillation
Recent studies pointed out that knowledge distillation (KD) suffers from two degradation problems, the teacher-student gap and the incompatibility with strong data augmentations, making it not applicable to training state-of-the-art models, which are trained with advanced augmentations.
Ranked #120 on
Image Classification
on ImageNet
Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning
Especially, we develop a variant of ViT for 3D point cloud feature extraction, which also achieves comparable results with existing backbones when combined with our framework, and visualization of the attention maps show that our model does understand the point cloud by combining the global shape information and multiple local structural information, which is consistent with the inspiration of our representation learning method.
UniFormer: Unifying Convolution and Self-attention for Visual Recognition
Different from the typical transformer blocks, the relation aggregators in our UniFormer block are equipped with local and global token affinity respectively in shallow and deep layers, allowing to tackle both redundancy and dependency for efficient and effective representation learning.
Ranked #137 on
Image Classification
on ImageNet
UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 9% and 71. 2% top-1 accuracy respectively.
Pyramid Fusion Transformer for Semantic Segmentation
The recently proposed MaskFormer gives a refreshed perspective on the task of semantic segmentation: it shifts from the popular pixel-level classification paradigm to a mask-level classification method.
PointCLIP: Point Cloud Understanding by CLIP
On top of that, we design an inter-view adapter to better extract the global feature and adaptively fuse the few-shot knowledge learned from 3D into CLIP pre-trained in 2D.
Ranked #3 on
Training-free 3D Part Segmentation
on ShapeNet-Part
(using extra training data)
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks
The model is pre-trained on several uni-modal and multi-modal tasks, and evaluated on a variety of downstream tasks, including novel tasks that did not appear in the pre-training stage.
DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks
However, the existing N:M algorithms only address the challenge of how to train N:M sparse neural networks in a uniform fashion (i. e. every layer has the same N:M sparsity) and suffer from a significant accuracy drop for high sparsity (i. e. when sparsity > 80\%).
Container: Context Aggregation Networks
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
IDR: Self-Supervised Image Denoising via Iterative Data Refinement
To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
A Simple Long-Tailed Recognition Baseline via Vision-Language Model
Recent advances in large-scale contrastive visual-language pretraining shed light on a new pathway for visual recognition.
Ranked #2 on
Long-tail Learning
on Places-LT
(using extra training data)
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling
To further enhance CLIP's few-shot capability, CLIP-Adapter proposed to fine-tune a lightweight residual feature adapter and significantly improves the performance for few-shot classification.
Rethinking Noise Synthesis and Modeling in Raw Denoising
However, the real raw image noise is contributed by many noise sources and varies greatly among different sensors.
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Large-scale contrastive vision-language pre-training has shown significant progress in visual representation learning.
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP
Recently, transformer and multi-layer perceptron (MLP) architectures have achieved impressive results on various vision tasks.
Ranked #211 on
Image Classification
on ImageNet
UniFormer: Unified Transformer for Efficient Spatial-Temporal Representation Learning
For Something-Something V1 and V2, our UniFormer achieves new state-of-the-art performances of 60. 8% and 71. 4% top-1 accuracy respectively.
Ranked #3 on
Action Recognition
on Something-Something V1
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception
In this paper, we benchmark our model on these three tasks.
FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
On the contrary, the soft composition operates by stitching different patches into a whole feature map where pixels in overlapping regions are summed up.
Ranked #2 on
Video Inpainting
on DAVIS
Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimation
To this end, we propose Multi-level Attention Encoder-Decoder Network (MAED), including a Spatial-Temporal Encoder (STE) and a Kinematic Topology Decoder (KTD) to model multi-level attentions in a unified framework.
Ranked #1 on
3D Human Pose Estimation
on MPI-INF-3DHP
(PA-MPJPE metric)
Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition
With the effective spatio-temporal relationship modeling, it is possible not only to uncover contextual information in each frame but also to directly capture inter-time dependencies.
LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector
Compared with the state-of-the-art stereo detector, our method has improved the 3D detection performance of cars, pedestrians, cyclists by 10. 44%, 5. 69%, 5. 97% mAP respectively on the official KITTI benchmark.
LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation
However, two major issues of the fusion between camera and LiDAR hinder its performance, \ie, how to effectively fuse these two modalities and how to precisely align them (suffering from the weak spatiotemporal synchronization problem).
ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection
These specific designs enable the detector to be trained on meticulously refined pseudo labeled target data with denoised training signals, and thus effectively facilitate adapting an object detector to a target domain without requiring annotations.
Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localization
In this paper, we present a framework named FAC-Net based on the I3D backbone, on which three branches are appended, named class-wise foreground classification branch, class-agnostic attention branch and multiple instance learning branch.
Fast Convergence of DETR with Spatially Modulated Co-Attention
However, DETR suffers from its slow convergence.
Unsupervised Domain Adaptive 3D Detection with Multi-Level Consistency
In addition, existing 3D domain adaptive detection methods often assume prior access to the target domain annotations, which is rarely feasible in the real world.
Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification
With the contribution of the CCD and CRP, our CRCKD algorithm can distill the relational knowledge more comprehensively.
Hybrid Supervision Learning for Pathology Whole Slide Image Classification
To handle this problem, we propose a hybrid supervision learning framework for this kind of high resolution images with sufficient image-level coarse annotations and a few pixel-level fine labels.
Refining Pseudo Labels with Clustering Consensus over Generations for Unsupervised Object Re-identification
Unsupervised object re-identification targets at learning discriminative representations for object retrieval without any annotations.
Scalable Transformers for Neural Machine Translation
In this paper, we propose a novel Scalable Transformers, which naturally contains sub-Transformers of different scales and have shared parameters.
Container: Context Aggregation Network
Convolutional neural networks (CNNs) are ubiquitous in computer vision, with a myriad of effective and efficient variations.
Ranked #408 on
Image Classification
on ImageNet
FNAS: Uncertainty-Aware Fast Neural Architecture Search
Further, an architecture knowledge pool together with a block similarity function is proposed to utilize parameter knowledge and reduces the searching time by 2 times.
VS-Net: Voting with Segmentation for Visual Localization
To address this problem, we propose a novel visual localization framework that establishes 2D-to-3D correspondences between the query image and the 3D map with a series of learnable scene-specific landmarks.
Inverting Generative Adversarial Renderer for Face Reconstruction
Since the GAR learns to model the complicated real-world image, instead of relying on the simplified graphics rules, it is capable of producing realistic images, which essentially inhibits the domain-shift noise in training and optimization.
Self-distillation with Batch Knowledge Ensembling Improves ImageNet Classification
In this way, our BAKE framework achieves online knowledge ensembling across multiple samples with only a single network.
Decoupled Spatial-Temporal Transformer for Video Inpainting
Seamless combination of these two novel designs forms a better spatial-temporal attention scheme and our proposed model achieves better performance than state-of-the-art video inpainting approaches with significant boosted efficiency.
Semantic Scene Completion via Integrating Instances and Scene in-the-Loop
The key insight is that we decouple the instances from a coarsely completed semantic scene instead of a raw input image to guide the reconstruction of instances and the overall scene.
Ranked #1 on
3D Semantic Scene Completion
on NYUv2
LIFE: Lighting Invariant Flow Estimation
However, local image contents are inevitably ambiguous and error-prone during the cross-image feature matching process, which hinders downstream tasks.
FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images
Radiotherapy is a treatment where radiation is used to eliminate cancer cells.
Fixing the Teacher-Student Knowledge Discrepancy in Distillation
To solve this problem, in this paper, we propose a novel student-dependent distillation method, knowledge consistent distillation, which makes teacher's knowledge more consistent with the student and provides the best suitable knowledge to different student networks for distillation.
AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks
However, the automatic design of loss functions for generic tasks with various evaluation metrics remains under-investigated.
DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network
Conditional generative adversarial networks (cGANs) target at synthesizing diverse images given the input conditions and latent codes, but unfortunately, they usually suffer from the issue of mode collapse.
ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection
Then, the detector is iteratively improved on the target domain by alternatively conducting two steps, which are the pseudo label updating with the developed quality-aware triplet memory bank and the model training with curriculum data augmentation.
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
In this paper, we are the first to study training from scratch an N:M fine-grained structured sparse network, which can maintain the advantages of both unstructured fine-grained sparsity and structured coarse-grained sparsity simultaneously on specifically designed GPUs.
PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detection
3D object detection is receiving increasing attention from both industry and academia thanks to its wide applications in various fields.
Ranked #2 on
3D Object Detection
on KITTI Cars Easy val
Fast Convergence of DETR with Spatially Modulated Co-Attention
The recently proposed Detection Transformer (DETR) model successfully applies Transformer to objects detection and achieves comparable performance with two-stage object detection frameworks, such as Faster-RCNN.
Progressive Correspondence Pruning by Consensus Learning
Correspondence selection aims to correctly select the consistent matches (inliers) from an initial set of putative correspondences.
Self-supervised Temporal Learning
Inspired by spatial-based contrastive SSL, we show that significant improvement can be achieved by a proposed temporal-based contrastive learning approach, which includes three novel and efficient modules: temporal augmentations, temporal memory bank and SSTL loss.
Towards Overcoming False Positives in Visual Relationship Detection
We observe that during training, the relationship proposal distribution is highly imbalanced: most of the negative relationship proposals are easy to identify, e. g., the inaccurate object detection, which leads to the under-fitting of low-frequency difficult proposals.
A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection
With the proposed holistically-guided decoder, we implement the EfficientFCN architecture for semantic segmentation and HGD-FPN for object detection and instance segmentation.
REFINE: Prediction Fusion Network for Panoptic Segmentation
Panoptic segmentation aims at generating pixel-wise class and instance predictions for each pixel in the input image, which is a challenging task and far more complicated than naively fusing the semantic and instance segmentation results.
Ranked #10 on
Panoptic Segmentation
on COCO test-dev
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network
2) Dynamic Shifting for complex point distributions.
Ranked #1 on
Panoptic Segmentation
on SemanticKITTI
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
However, we found that in the outdoor point cloud, the improvement obtained in this way is quite limited.
Ranked #2 on
3D Semantic Segmentation
on ScribbleKITTI
End-to-End Object Detection with Adaptive Clustering Transformer
In this paper, a novel variant of transformer named Adaptive Clustering Transformer(ACT) has been proposed to reduce the computation cost for high-resolution input.
SelfVoxeLO: Self-supervised LiDAR Odometry with Voxel-based Deep Neural Networks
To suit our network to self-supervised learning, we design several novel loss functions that utilize the inherent properties of LiDAR point clouds.
PV-RCNN: The Top-Performing LiDAR-only Solutions for 3D Detection / 3D Tracking / Domain Adaptation of Waymo Open Dataset Challenges
In this technical report, we present the top-performing LiDAR-only solutions for 3D detection, 3D tracking and domain adaptation three tracks in Waymo Open Dataset Challenges 2020.
EfficientFCN: Holistically-guided Decoding for Semantic Segmentation
State-of-the-art semantic segmentation algorithms are mostly based on dilated Fully Convolutional Networks (dilatedFCN), which adopt dilated convolutions in the backbone networks to extract high-resolution feature maps for achieving high-performance segmentation performance.
Multi-organ Segmentation via Co-training Weight-averaged Models from Few-organ Datasets
In most scenarios, one might obtain annotations of a single or a few organs from one training set, and obtain annotations of the the other organs from another set of training images.
Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
A straightforward solution to tackle the issue of 3D-to-2D projection is to keep the 3D representation and process the points in the 3D space.
Ranked #9 on
LIDAR Semantic Segmentation
on nuScenes
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions
We propose a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions.
Balanced Meta-Softmax for Long-Tailed Visual Recognition
In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.
Ranked #5 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
Complementary Boundary Generator with Scale-Invariant Relation Modeling for Temporal Action Localization: Submission to ActivityNet Challenge 2020
Specifically, in order to generate high-quality proposals, we consider several factors including the video feature encoder, the proposal generator, the proposal-proposal relations, the scale imbalance, and ensemble strategy.
Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation
Depth information has proven to be a useful cue in the semantic segmentation of RGB-D images for providing a geometric counterpart to the RGB representation.
Ranked #4 on
Semantic Segmentation
on UrbanLF
Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers
Given an input video, its associated audio, and a brief caption, the audio-visual scene aware dialog (AVSD) task requires an agent to indulge in a question-answer dialog with a human about the audio-visual content.
1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020
This technical report introduces our winning solution to the spatio-temporal action localization track, AVA-Kinetics Crossover, in ActivityNet Challenge 2020.
Spatio-Temporal Action Localization
Temporal Action Localization
Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization
We propose to explicitly model the Actor-Context-Actor Relation, which is the relation between two actors based on their interactions with the context.
Ranked #1 on
Action Recognition
on AVA v2.1
Self-supervising Fine-grained Region Similarities for Large-scale Image Localization
The task of large-scale retrieval-based image localization is to estimate the geographical location of a query image by recognizing its nearest reference images from a city-scale dataset.
Self-paced Contrastive Learning with Hybrid Memory for Domain Adaptive Object Re-ID
To solve these problems, we propose a novel self-paced contrastive learning framework with hybrid memory.
Ranked #3 on
Unsupervised Domain Adaptation
on Market to MSMT
StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching
Large-scale synthetic datasets are beneficial to stereo matching but usually introduce known domain bias.
Learning to Predict Context-adaptive Convolution for Semantic Segmentation
Long-range contextual information is essential for achieving high-performance semantic segmentation.
3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior
To this end, we first propose a novel 3D sketch-aware feature embedding to explicitly encode geometric information effectively and efficiently.
Structured Domain Adaptation with Online Relation Regularization for Unsupervised Person Re-ID
To tackle the challenges, we propose an end-to-end structured domain adaptation framework with an online relation-consistency regularization term.
Ranked #4 on
Unsupervised Domain Adaptation
on Market to MSMT
MagnifierNet: Towards Semantic Adversary and Fusion for Person Re-identification
Meanwhile, we introduce "Semantic Fusion Branch" to filter out irrelevant noises by selectively fusing semantic region information sequentially.
Monocular 3D Object Detection with Decoupled Structured Polygon Estimation and Height-Guided Depth Estimation
Monocular 3D object detection task aims to predict the 3D bounding boxes of objects based on monocular RGB images.
Structure-Feature based Graph Self-adaptive Pooling
To solve these problems mentioned above, we propose a novel graph self-adaptive pooling method with the following objectives: (1) to construct a reasonable pooled graph topology, structure and feature information of the graph are considered simultaneously, which provide additional veracity and objectivity in node selection; and (2) to make the pooled nodes contain sufficiently effective graph information, node feature information is aggregated before discarding the unimportant nodes; thus, the selected nodes contain information from neighbor nodes, which can enhance the use of features of the unselected nodes.
Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification
In order to mitigate the effects of noisy pseudo labels, we propose to softly refine the pseudo labels in the target domain by proposing an unsupervised framework, Mutual Mean-Teaching (MMT), to learn better features from the target domain via off-line refined hard pseudo labels and on-line refined soft pseudo labels in an alternative training manner.
PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection
We present a novel and high-performance 3D object detection framework, named PointVoxel-RCNN (PV-RCNN), for accurate 3D object detection from point clouds.
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis
Semantic image synthesis aims at generating photorealistic images from semantic layouts.
Depth Completion from Sparse LiDAR Data with Depth-Normal Constraints
Most of existing methods directly train a network to learn a mapping from sparse depth inputs to dense depth maps, which has difficulties in utilizing the 3D geometric constraints and handling the practical sensor noises.
CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval
Text-image cross-modal retrieval is a challenging task in the field of language and vision.
Ranked #8 on
Image Retrieval
on Flickr30K 1K test
Interpolated Convolutional Networks for 3D Point Cloud Understanding
Our InterpConv is shown to be permutation and sparsity invariant, and can directly handle irregular inputs.
Ranked #21 on
3D Part Segmentation
on ShapeNet-Part
Multi-modality Latent Interaction Network for Visual Question Answering
The proposed module learns the cross-modality relationships between latent visual and language summarizations, which summarize visual regions and question into a small number of latent representations to avoid modeling uninformative individual region-word relations.
FocusNet: Imbalanced Large and Small Organ Segmentation with an End-to-End Deep Neural Network for Head and Neck CT Images
In this paper, we propose an end-to-end deep neural network for solving the problem of imbalanced large and small organ segmentation in head and neck (HaN) CT images.
Signet Ring Cell Detection With a Semi-supervised Learning Framework
Our framework achieves accurate signet ring cell detection and can be readily applied in the clinical trails.
From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network
3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications.
P2SGrad: Refined Gradients for Optimizing Deep Face Models
Cosine-based softmax losses significantly improve the performance of deep face recognition networks.
AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations
Our results show that training deep neural networks with the AdaCos loss is stable and able to achieve high face recognition accuracy.
Ranked #5 on
Face Verification
on MegaFace
Generalizing Monocular 3D Human Pose Estimation in the Wild
We observe that recent innovation in this area mainly focuses on new techniques that explicitly address the generalization issue when using this dataset, because this database is constructed in a highly controlled environment with limited human subjects and background variations.
Ranked #55 on
3D Human Pose Estimation
on Human3.6M
Conditional Adversarial Generative Flow for Controllable Image Synthesis
Flow-based generative models show great potential in image synthesis due to its reversible pipeline and exact log-likelihood target, yet it suffers from weak ability for conditional image synthesis, especially for multi-label or unaware conditions.
Group-wise Correlation Stereo Network
Previous works built cost volumes with cross-correlation or concatenation of left and right features across all disparity levels, and then a 2D or 3D convolutional neural network is utilized to regress the disparity maps.
Unsupervised Cross-spectral Stereo Matching by Learning to Synthesize
Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any supervision in the form of ground truth disparity or depth.
Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing
Referring expression grounding aims at locating certain objects or persons in an image with a referring expression, where the key challenge is to comprehend and align various types of information from visual and textual domain, such as visual attributes, location and interactions with surrounding regions.
A^2-Net: Molecular Structure Estimation from Cryo-EM Density Volumes
Constructing of molecular structural models from Cryo-Electron Microscopy (Cryo-EM) density volumes is the critical last step of structure determination by Cryo-EM technologies.
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud
In this paper, we propose PointRCNN for 3D object detection from raw point cloud.
Ranked #2 on
Object Detection
on KITTI Cars Easy
Efficient Attention: Attention with Linear Complexities
Dot-product attention has wide applications in computer vision and natural language processing.
Ranked #2 on
Extractive Text Summarization
on GovReport
FD-GAN: Pose-guided Feature Distilling GAN for Robust Person Re-identification
Our proposed FD-GAN achieves state-of-the-art performance on three person reID datasets, which demonstrates that the effectiveness and robust feature distilling capability of the proposed FD-GAN.
Ranked #3 on
Person Re-Identification
on CUHK03