Search Results for author:
Found 121 papers, 55 papers with code
SIMILE: Introducing Sequential Information towards More Effective Imitation Learning
Reinforcement learning (RL) is a metaheuristic aiming at teaching an agent to interact with an environment and maximizing the reward in a complex task.
Visual Tuning
With the surprising development of pre-trained visual foundation models, visual tuning jumped out of the standard modus operandi that fine-tunes the whole pre-trained model or just the fully connected layer.
Segment Anything in 3D with NeRFs
Recently, the Segment Anything Model (SAM) emerged as a powerful vision foundation model which is capable to segment anything in 2D images.
Pipeline MoE: A Flexible MoE Implementation with Pipeline Parallelism
The Mixture of Experts (MoE) model becomes an important choice of large language models nowadays because of its scalability with sublinear computational complexity for training and inference.
Focus on Your Target: A Dual Teacher-Student Framework for Domain-adaptive Semantic Segmentation
Currently, a popular UDA framework lies in self-training which endows the model with two-fold abilities: (i) learning reliable semantics from the labeled images in the source domain, and (ii) adapting to the target domain via generating pseudo labels on the unlabeled images.
USAGE: A Unified Seed Area Generation Paradigm for Weakly Supervised Semantic Segmentation
Seed area generation is usually the starting point of weakly supervised semantic segmentation (WSSS).
Multi-Label Classification
Weakly supervised Semantic Segmentation
+1
Integrally Pre-Trained Transformer Pyramid Networks
In this paper, we present an integral pre-training framework based on masked image modeling (MIM).
Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models
Pretrained vision-language models (VLMs) such as CLIP have shown impressive generalization capability in downstream vision tasks with appropriate text prompts.
Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast
In this paper, we present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast.
Learnable Distribution Calibration for Few-Shot Class-Incremental Learning
LDC is built upon a parameterized calibration unit (PCU), which initializes biased distributions for all classes based on classifier vectors (memory-free) and a single covariance matrix.
class-incremental learning
Few-Shot Class-Incremental Learning
+3
Fine-Grained Semantically Aligned Vision-Language Pre-Training
Large-scale vision-language pre-training has shown impressive advances in a wide range of downstream tasks.
Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction
To address the first issue, we propose adaptive graph scattering, which leverages multiple trainable band-pass graph filters to decompose pose features into richer graph spectrum bands.
Visual Recognition by Request
Humans have the ability of recognizing visual semantics in an unlimited granularity, but existing visual recognition algorithms cannot achieve this goal.
Active Pointly-Supervised Instance Segmentation
In this paper, we present an economic active learning setting, named active pointly-supervised instance segmentation (APIS), which starts with box-level annotations and iteratively samples a point within the box and asks if it falls on the object.
A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction
Next, we summarize the existing label-efficient image segmentation methods from a unified perspective that discusses an important question: how to bridge the gap between weak supervision and dense prediction -- the current methods are mostly based on heuristic priors, such as cross-pixel similarity, cross-label constraint, cross-view consistency, and cross-image relation.
Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
A multi-distance interpolation method is proposed and applied on voxel features to model both small and large motions.
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling
A key idea of efficient implementation is to discard the masked image patches (or tokens) throughout the target network (encoder), which requires the encoder to be a plain vision transformer (e. g., ViT), albeit hierarchical vision transformers (e. g., Swin Transformer) have potentially better properties in formulating vision inputs.
CenterNet++ for Object Detection
Our approach, named CenterNet, detects each object as a triplet keypoints (top-left and bottom-right corners and the center keypoint).
Ranked #32 on
Object Detection
on COCO test-dev
Domain-Agnostic Prior for Transfer Semantic Segmentation
Unsupervised domain adaptation (UDA) is an important topic in the computer vision community.
Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformers
The past year has witnessed a rapid development of masked image modeling (MIM).
TAPE: Task-Agnostic Prior Embedding for Image Restoration
In this paper, we propose a novel approach that embeds a task-agnostic prior into a transformer.
MVP: Multimodality-guided Visual Pre-training
Recently, masked image modeling (MIM) has become a promising direction for visual pre-training.
One-Bit Active Query With Contrastive Pairs
The Yes query is treated as positive pairs of the queried category for contrastive pulling, while the No query is treated as hard negative pairs for contrastive repelling.
Exploring Complicated Search Spaces with Interleaving-Free Sampling
In this paper, we build the search algorithm upon a complicated search space with long-distance connections, and show that existing weight-sharing search algorithms mostly fail due to the existence of \textbf{interleaved connections}.
NeuSample: Neural Sample Field for Efficient View Synthesis
Neural radiance fields (NeRF) have shown great potentials in representing 3D scenes and synthesizing novel views, but the computational overhead of NeRF at the inference stage is still heavy.
Semantic-Aware Generation for Self-Supervised Visual Representation Learning
In this paper, we propose a self-supervised visual representation learning approach which involves both generative and discriminative proxies, where we focus on the former part by requiring the target network to recover the original image based on the mid-level features.
Ranked #58 on
Semantic Segmentation
on Cityscapes test
Consensus Synergizes with Memory: A Simple Approach for Anomaly Segmentation in Urban Scenes
Anomaly segmentation is a crucial task for safety-critical applications, such as autonomous driving in urban scenes, where the goal is to detect out-of-distribution (OOD) objects with categories which are unseen during training.
Ranked #6 on
Anomaly Detection
on Fishyscapes L&F
CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis
The style-based GAN (StyleGAN) architecture achieved state-of-the-art results for generating high-quality images, but it lacks explicit and precise control over camera poses.
Ranked #1 on
3D-Aware Image Synthesis
on FFHQ 256 x 256
Vibration-based Uncertainty Estimation for Learning from Limited Supervision
We investigate the problem of estimating uncertainty for training data, so that deep neural networks can make use of the results for learning from limited supervision.
Deep Encryption: Protecting Pre-Trained Neural Networks with Confusion Neurons
Having consumed huge amounts of training data and computational resource, large-scale pre-trained models are often considered key assets of AI service providers.
Rectifying the Shortcut Learning of Background for Few-Shot Learning
The category gap between training and evaluation has been characterised as one of the main obstacles to the success of Few-Shot Learning (FSL).
Bag of Instances Aggregation Boosts Self-supervised Distillation
Here bag of instances indicates a set of similar samples constructed by the teacher and are grouped within a bag, and the goal of distillation is to aggregate compact representations over the student with respect to instances in a bag.
ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Image Segmentation
Semi-supervised learning is a useful tool for image segmentation, mainly due to its ability in extracting knowledge from unlabeled data to assist learning from labeled data.
Exploring the Diversity and Invariance in Yourself for Visual Pre-Training Task
By simply pulling the different augmented views of each image together or other novel mechanisms, they can learn much unsupervised knowledge and significantly improve the transfer performance of pre-training models.
MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokens
Transformers have offered a new methodology of designing neural networks for visual recognition.
Conformer: Local Features Coupling Global Representations for Visual Recognition
Within Convolutional Neural Network (CNN), the convolution operations are good at extracting local features but experience difficulty to capture global representations.
Ranked #292 on
Image Classification
on ImageNet
Visformer: The Vision-friendly Transformer
The past year has witnessed the rapid development of applying the Transformer module to vision problems.
Ranked #454 on
Image Classification
on ImageNet
Location-Sensitive Visual Recognition with Cross-IOU Loss
Object detection, instance segmentation, and pose estimation are popular visual recognition tasks which require localizing the object by internal or boundary landmarks.
Ranked #54 on
Object Detection
on COCO test-dev
Spatiotemporal Transformer for Video-based Person Re-identification
Recently, the Transformer module has been transplanted from natural language processing to computer vision.
MagDR: Mask-guided Detection and Reconstruction for Defending Deepfakes
This paper presents MagDR, a mask-guided detection and reconstruction pipeline for defending deepfakes from adversarial attacks.
Interactive Fusion of Multi-level Features for Compositional Activity Recognition
To understand a complex action, multiple sources of information, including appearance, positional, and semantic features, need to be integrated.
UnrealPerson: An Adaptive Pipeline towards Costless Person Re-identification
The main difficulty of person re-identification (ReID) lies in collecting annotated data and transferring the model across different domains.
Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning
In this paper, we propose a hierarchical semantic alignment strategy via expanding the views generated by a single image to \textbf{Cross-samples and Multi-level} representation, and models the invariance to semantically similar images in a hierarchical way.
Batch Normalization with Enhanced Linear Transformation
Batch normalization (BN) is a fundamental unit in modern deep networks, in which a linear transformation module was designed for improving BN's flexibility of fitting complex data distributions.
Omni-GAN: On the Secrets of cGANs and Beyond
The conditional generative adversarial network (cGAN) is a powerful tool of generating high-quality images, but existing approaches mostly suffer unsatisfying performance or the risk of mode collapse.
Ranked #5 on
Conditional Image Generation
on ImageNet 128x128
Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations
Contrastive learning has achieved great success in self-supervised visual representation learning, but existing approaches mostly ignored spatial information which is often crucial for visual representation.
Privileged Knowledge Distillation for Online Action Detection
Knowledge distillation is employed to transfer the privileged information from the offline teacher to the online student.
Ranked #5 on
Online Action Detection
on TVSeries
Can Semantic Labels Assist Self-Supervised Visual Representation Learning?
Recently, contrastive learning has largely advanced the progress of unsupervised visual representation learning.
One-bit Supervision for Image Classification
Instead of training a model upon the accurate label of each sample, our setting requires the model to query with a predicted label of each sample and learn from the answer whether the guess is correct.
Reinforced Axial Refinement Network for Monocular 3D Object Detection
Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image.
Ranked #16 on
Vehicle Pose Estimation
on KITTI Cars Hard
Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
Neural architecture search (NAS) has attracted increasing attentions in both academia and industry.
Corner Proposal Network for Anchor-free, Two-stage Object Detection
On the MS-COCO dataset, CPN achieves an AP of 49. 2% which is competitive among state-of-the-art object detection methods.
Ranked #96 on
Object Detection
on COCO test-dev
Polar Relative Positional Encoding for Video-Language Segmentation
In this paper, we propose a novel Polar Relative Positional Encoding (PRPE) mechanism that represents spatial relations in a ``linguistic'' way, i. e., in terms of direction and range.
Ranked #10 on
Referring Expression Segmentation
on J-HMDB
Social Adaptive Module for Weakly-supervised Group Activity Recognition
This paper presents a new task named weakly-supervised group activity recognition (GAR) which differs from conventional GAR tasks in that only video-level labels are available, yet the important persons within each frame are not provided even in the training data.
Universal-to-Specific Framework for Complex Action Recognition
The U2S framework is composed of three subnetworks: a universal network, a category-specific network, and a mask network.
Discretization-Aware Architecture Search
The search cost of neural architecture search (NAS) has been largely reduced by weight-sharing methods.
GOLD-NAS: Gradual, One-Level, Differentiable
There has been a large literature of neural architecture search, but most existing work made use of heuristic rules that largely constrained the search flexibility.
Searching towards Class-Aware Generators for Conditional Generative Adversarial Networks
To learn the sampling policy, a Markov decision process is embedded into the search algorithm and a moving average is applied for better stability.
ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Medical Image Segmentation
This paper focuses on a popular pipeline known as self learning, and points out a weakness named lazy learning that refers to the difficulty for a model to learn from the pseudo labels generated by itself.
Fitting the Search Space of Weight-sharing NAS with Graph Convolutional Networks
We alleviate this issue by training a graph convolutional network to fit the performance of sampled sub-networks so that the impact of random errors becomes minimal.
Unsupervised Person Re-identification via Softened Similarity Learning
Person re-identification (re-ID) is an important topic in computer vision.
Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio
Automatic designing computationally efficient neural networks has received much attention in recent years.
Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing
In recent years, cross-modal hashing (CMH) has attracted increasing attentions, mainly because its potential ability of mapping contents from different modalities, especially in vision and language, into the same space, so that it becomes efficient in cross-modal data retrieval.
Circumventing Outliers of AutoAugment with Knowledge Distillation
AutoAugment has been a powerful algorithm that improves the accuracy of many vision tasks, yet it is sensitive to the operator space as well as hyper-parameters, and an improper setting may degenerate network optimization.
Ranked #170 on
Image Classification
on ImageNet
Bottom-Up Temporal Action Localization with Mutual Regularization
To alleviate this problem, we introduce two regularization terms to mutually regularize the learning procedure: the Intra-phase Consistency (IntraC) regularization is proposed to make the predictions verified inside each phase; and the Inter-phase Consistency (InterC) regularization is proposed to keep consistency between these phases.
Rethinking the Distribution Gap of Person Re-identification with Camera-based Batch Normalization
The fundamental difficulty in person re-identification (ReID) lies in learning the correspondence among individual cameras.
Ranked #15 on
Unsupervised Domain Adaptation
on Duke to Market
Direct Transfer Person Re-identification
Domain Adaptive Person Re-Identification
+2
Latency-Aware Differentiable Neural Architecture Search
However, these methods suffer the difficulty in optimizing network, so that the searched network is often unfriendly to hardware.
Wasserstein-Bounded Generative Adversarial Networks
In the field of Generative Adversarial Networks (GANs), how to design a stable training strategy remains an open problem.
Scalable NAS with Factorizable Architectural Parameters
Neural Architecture Search (NAS) is an emerging topic in machine learning and computer vision.
Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild
With the rapid development of neural architecture search (NAS), researchers found powerful network architectures for a wide range of vision tasks.
Appending Adversarial Frames for Universal Video Attack
There have been many efforts in attacking image classification models with adversarial perturbations, but the same topic on video classification has not yet been thoroughly studied.
Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameters
Our approach bridges the gap from two aspects, namely, amending the estimation on the architectural gradients, and unifying the hyper-parameter settings in the search and re-training stages.
Fast Non-Local Neural Networks with Spectral Residual Learning
We show its equivalence to conducting residual learning in some spectral domain and carefully re-formulate a variety of neural layers into their spectral forms, such as ReLU or convolutions.
Pruning from Scratch
Network pruning is an important research field aiming at reducing computational costs of neural networks.
Single Camera Training for Person Re-identification
Differently, this paper investigates ReID in an unexplored single-camera-training (SCT) setting, where each person in the training set appears in only one camera.
Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data
Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks.
PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search
Differentiable architecture search (DARTS) provided a fast solution in finding effective network architectures, but suffered from large memory and computing overheads in jointly training a super-network and searching for an optimal architecture.
Ranked #20 on
Neural Architecture Search
on CIFAR-10
Defending Adversarial Attacks by Correcting logits
Generating and eliminating adversarial examples has been an intriguing topic in the field of deep learning.
Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation
Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search.
CenterNet: Keypoint Triplets for Object Detection
In object detection, keypoint-based approaches often suffer a large number of incorrect object bounding boxes, arguably due to the lack of an additional look into the cropped regions.
Ranked #118 on
Object Detection
on COCO test-dev
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval
Sketch-based image retrieval (SBIR) is widely recognized as an important vision problem which implies a wide range of real-world applications.
Thickened 2D Networks for Efficient 3D Medical Image Segmentation
With this design, we achieve a higher performance while maintaining a lower inference latency on a few abdominal organs from CT scans, in particular when the organ has a peculiar 3D shape and thus strongly requires contextual information, demonstrating our method's effectiveness and ability in capturing 3D information.
SIXray : A Large-scale Security Inspection X-ray Benchmark for Prohibited Item Discovery in Overlapping Images
In particular, the advantage of CHR is more significant in the scenarios with fewer positive training samples, which demonstrates its potential application in real-world security inspection.
Identity-Enhanced Network for Facial Expression Recognition
Facial expression recognition is a challenging task, arguably because of large intra-class variations and high inter-class similarities.
Attention-guided Unified Network for Panoptic Segmentation
This paper studies panoptic segmentation, a recently proposed task which segments foreground (FG) objects at the instance level as well as background (BG) contents at the semantic level.
Ranked #23 on
Panoptic Segmentation
on COCO test-dev
Elastic Boundary Projection for 3D Medical Image Segmentation
The key observation is that, although the object is a 3D volume, what we really need in segmentation is to find its boundary which is a 2D surface.
CRAVES: Controlling Robotic Arm with a Vision-based Economic System
We also construct a vision-based control system for task accomplishment, for which we train a reinforcement learning agent in a virtual environment and apply it to the real-world.
Iterative Reorganization with Weak Spatial Constraints: Solving Arbitrary Jigsaw Puzzles for Unsupervised Representation Learning
We consider spatial contexts, for which we solve so-called jigsaw puzzles, i. e., each image is cut into grids and then disordered, and the goal is to recover the correct configuration.
Snapshot Distillation: Teacher-Student Optimization in One Generation
Optimizing a deep neural network is a fundamental task in computer vision, yet direct training methods often suffer from over-fitting.
Generalized Coarse-to-Fine Visual Recognition with Progressive Training
Computer vision is difficult, partly because the desired mathematical function connecting input and output data is often complex, fuzzy and thus hard to learn.
Phase Collaborative Network for Two-Phase Medical Image Segmentation
However, in medical image analysis, fusing prediction from two phases is often difficult, because (i) there is a domain gap between two phases, and (ii) the semantic labels are not pixel-wise corresponded even for images scanned from the same patient.
Semantic Part Detection via Matching: Learning to Generalize to Novel Viewpoints from Limited Training Data
In particular, this enables images in the training dataset to be matched to a virtual 3D model of the object (for simplicity, we assume that the object viewpoint can be estimated by standard techniques).
Accelerating Deep Neural Networks with Spatial Bottleneck Modules
This paper presents an efficient module named spatial bottleneck for accelerating the convolutional layers in deep neural networks.
Infinite Curriculum Learning for Efficiently Detecting Gastric Ulcers in WCE Images
This paper considers WCE-based gastric ulcer detection, in which the major challenge is to detect the lesions in a local region.
Attention-based Pyramid Aggregation Network for Visual Place Recognition
Visual place recognition is challenging in the urban environment and is usually viewed as a large scale image retrieval task.
Multi-Scale Coarse-to-Fine Segmentation for Screening Pancreatic Ductal Adenocarcinoma
We propose an intuitive approach of detecting pancreatic ductal adenocarcinoma (PDAC), the most common type of pancreatic cancer, by checking abdominal CT scans.
G2C: A Generator-to-Classifier Framework Integrating Multi-Stained Visual Cues for Pathological Glomerulus Classification
Given an input image from a specified stain, several generators are first applied to estimate its appearances in other staining methods, and a classifier follows to combine visual cues from different stains for prediction (whether it is pathological, or which type of pathology it has).
Knowledge Distillation in Generations: More Tolerant Teachers Educate Better Students
We focus on the problem of training a deep neural network in generations.
Joint Shape Representation and Classification for Detecting PDAC
Shape representation and classification are performed in a joint manner, both to exploit the knowledge that PDAC often changes the shape of the pancreas and to prevent over-fitting.
Multi-Scale Spatially-Asymmetric Recalibration for Image Classification
Convolution is spatially-symmetric, i. e., the visual features are independent of its position in the image, which limits its ability to utilize contextual cues for visual recognition.
Bridging the Gap Between 2D and 3D Organ Segmentation with Volumetric Fusion Net
There has been a debate on whether to use 2D or 3D deep neural networks for volumetric organ segmentation.
SampleAhead: Online Classifier-Sampler Communication for Learning from Synthesized Data
But, this raises an important problem in active vision: given an {\bf infinite} data space, how to effectively sample a {\bf finite} subset to train a visual classifier?
Adversarial Attacks Beyond the Image Space
Though image-space adversaries can be interpreted as per-pixel albedo change, we verify that they cannot be well explained along these physically meaningful dimensions, which often have a non-local effect.
Visual Concepts and Compositional Voting
We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles.
DeepVoting: A Robust and Explainable Deep Network for Semantic Part Detection under Partial Occlusion
The first layer extracts the evidence of local visual cues, and the second layer performs a voting mechanism by utilizing the spatial relationship between visual cues and semantic parts.
Recurrent Saliency Transformation Network: Incorporating Multi-Stage Visual Cues for Small Organ Segmentation
The key innovation is a saliency transformation module, which repeatedly converts the segmentation probability map from the previous iteration as spatial weights and applies these weights to the current iteration.
Ranked #1 on
Pancreas Segmentation
on TCIA Pancreas-CT Dataset
Detecting Semantic Parts on Partially Occluded Objects
Our approach detects semantic parts by accumulating the confidence of local visual cues.
Deep Supervision for Pancreatic Cyst Segmentation in Abdominal CT Scans
Inspired by the high relevance between the location of a pancreas and its cystic region, we introduce extra deep supervision into the segmentation network, so that cyst segmentation can be improved with the help of relatively easier pancreas segmentation.
Adversarial Examples for Semantic Segmentation and Object Detection
Our observation is that both segmentation and detection are based on classifying multiple targets on an image (e. g., the basic target is a pixel or a receptive field in segmentation, and an object proposal in detection), which inspires us to optimize a loss function over a set of pixels/proposals for generating adversarial perturbations.
SORT: Second-Order Response Transform for Visual Recognition
In this paper, we reveal the importance and benefits of introducing second-order operations into deep neural networks.
Genetic CNN
The deep Convolutional Neural Network (CNN) is the state-of-the-art solution for large-scale visual recognition.
Deep Collaborative Learning for Visual Recognition
We formulate the function of a convolutional layer as learning a large visual vocabulary, and propose an alternative way, namely Deep Collaborative Learning (DCL), to reduce the computational complexity.
A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans
Deep neural networks have been widely adopted for automatic organ segmentation from abdominal CT scans.
Object Recognition with and without Objects
While recent deep neural networks have achieved a promising performance on object recognition, they rely implicitly on the visual contents of the whole image.
Geometric Neural Phrase Pooling: Modeling the Spatial Co-occurrence of Neurons
For this, we consider the neurons in the hidden layer as neural words, and construct a set of geometric neural phrases on top of them.
DisturbLabel: Regularizing CNN on the Loss Layer
During a long period of time we are combating over-fitting in the CNN training process with model regularization, including weight decay, model averaging, data augmentation, etc.
InterActive: Inter-Layer Activeness Propagation
An increasing number of computer vision tasks can be tackled with deep features, which are the intermediate outputs of a pre-trained Convolutional Neural Network.
RIDE: Reversal Invariant Descriptor Enhancement
In many fine-grained object recognition datasets, image orientation (left/right) might vary from sample to sample.
Fidelity-Naturalness Evaluation of Single Image Super Resolution
In this paper, we proposed to use both fidelity (the difference with original images) and naturalness (human visual perception of super resolved images) for evaluation.
Orientational Pyramid Matching for Recognizing Indoor Scenes
The novelty lies in that OPM uses the 3D orientations to form the pyramid and produce the pooling regions, which is unlike SPM that uses the spatial positions to form the pyramid.
