Search Results for author:
Found 277 papers, 110 papers with code
Global Knowledge Calibration for Fast Open-Vocabulary Segmentation
Recent advancements in pre-trained vision-language models, such as CLIP, have enabled the segmentation of arbitrary concepts solely from textual inputs, a process commonly referred to as open-vocabulary semantic segmentation (OVS).
Knowledge Distillation
Open Vocabulary Semantic Segmentation
+3
Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring
In this paper, based on the CLIP model, we revisit temporal modeling in the context of image-to-video knowledge transferring, which is the key point for extending image-text pretrained models to the video domain.
Ranked #5 on
Video Retrieval
on MSR-VTT-1kA
(using extra training data)
Multimodal Video Adapter for Parameter Efficient Video Text Retrieval
State-of-the-art video-text retrieval (VTR) methods usually fully fine-tune the pre-trained model (e. g.
Temporal Perceiving Video-Language Pre-training
Specifically, text-video localization consists of moment retrieval, which predicts start and end boundaries in videos given the text description, and text localization which matches the subset of texts with the video features.
CMAE-V: Contrastive Masked Autoencoders for Video Action Recognition
Contrastive Masked Autoencoder (CMAE), as a new self-supervised framework, has shown its potential of learning expressive feature representations in visual image recognition.
Class Prototype-based Cleaner for Label Noise Learning
Semi-supervised learning based methods are current SOTA solutions to the noisy-label learning problem, which rely on learning an unsupervised label cleaner first to divide the training samples into a labeled set for clean data and an unlabeled set for noise data.
Ranked #2 on
Image Classification
on Clothing1M
PV3D: A 3D Generative Model for Portrait Video Generation
While some prior works have applied such image GANs to unconditional 2D portrait video generation and static 3D portrait synthesis, there are few works successfully extending GANs for generating 3D-aware portrait videos.
Diffusion Probabilistic Model Made Slim
Despite the recent visually-pleasing results achieved, the massive computational cost has been a long-standing flaw for diffusion probabilistic models (DPMs), which, in turn, greatly limits their applications on resource-limited platforms.
AvatarGen: A 3D Generative Model for Animatable Human Avatars
Specifically, we decompose the generative 3D human synthesis into pose-guided mapping and canonical representation with predefined human pose and shape, such that the canonical representation can be explicitly driven to different poses and shapes with the guidance of a 3D parametric human model SMPL.
Expanding Small-Scale Datasets with Guided Imagination
The two criteria are verified to be essential for effective dataset expansion: GIF-SD obtains 13. 5\% higher model accuracy on natural image datasets than unguided expansion with SD.
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition
This paper does not attempt to design a state-of-the-art method for visual recognition but investigates a more efficient way to make use of convolutions to encode spatial features.
MagicVideo: Efficient Video Generation With Latent Diffusion Models
With the proposed efficient latent 3D U-Net design, MagicVideo can generate video clips with 256x256 spatial resolution on a single GPU card, which is 64x faster than the recent video diffusion model (VDM).
MagicMix: Semantic Mixing with Diffusion Models
Unlike style transfer, where an image is stylized according to the reference style without changing the image content, semantic blending mixes two different concepts in a semantic manner to synthesize a novel concept while preserving the spatial layout and geometry.
Reachability-Aware Laplacian Representation in Reinforcement Learning
In Reinforcement Learning (RL), Laplacian Representation (LapRep) is a task-agnostic state representation that encodes the geometry of the environment.
MetaFormer Baselines for Vision
By simply applying depthwise separable convolutions as token mixer in the bottom stages and vanilla self-attention in the top stages, the resulting model CAFormer sets a new record on ImageNet-1K: it achieves an accuracy of 85. 5% at 224x224 resolution, under normal supervised training without external data or distillation.
Ranked #57 on
Image Classification
on ImageNet
(using extra training data)
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning
With the proposed SSF, our model obtains 2. 46% (90. 72% vs. 88. 54%) and 11. 48% (73. 10% vs. 65. 57%) performance improvement on FGVC and VTAB-1k in terms of Top-1 accuracy compared to the full fine-tuning but only fine-tuning about 0. 3M parameters.
ManiCLIP: Multi-Attribute Face Manipulation from Text
In this paper we present a novel multi-attribute face manipulation method based on textual descriptions.
Global Prototype Encoding for Incremental Video Highlights Detection
To the best of our knowledge, this is the first time the incremental learning settings are introduced to video highlights detection, which in turn relieves the burden of training video inputs and promotes the scalability of conventional neural networks in proportion to both the size of the dataset and the quantity of domains.
AvatarGen: a 3D Generative Model for Animatable Human Avatars
Unsupervised generation of clothed virtual humans with various appearance and animatable poses is important for creating 3D human avatars and other AR/VR applications.
Contrastive Masked Autoencoders are Stronger Vision Learners
The target encoder, fed with the full images, enhances the feature discriminability via contrastive learning with its online counterpart.
Clover: Towards A Unified Video-Language Alignment and Fusion Model
We then introduce \textbf{Clover}\textemdash a Correlated Video-Language pre-training method\textemdash towards a universal Video-Language model for solving multiple video understanding tasks with neither performance nor efficiency compromise.
Ranked #1 on
Video Question Answering
on LSMDC-FiB
Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors
Domain Adaptation of Black-box Predictors (DABP) aims to learn a model on an unlabeled target domain supervised by a black-box predictor trained on a source domain.
Sharpness-Aware Training for Free
Intuitively, SAF achieves this by avoiding sudden drops in the loss in the sharp local minima throughout the trajectory of the updates of the weights.
Tyger: Task-Type-Generic Active Learning for Molecular Property Prediction
However, existing best deep AL methods are mostly developed for a single type of learning task (e. g., single-label classification), and hence may not perform well in molecular property prediction that involves various task types.
Understanding The Robustness in Vision Transformers
Our study is motivated by the intriguing properties of the emerging visual grouping in Vision Transformers, which indicates that self-attention may promote robustness through improved mid-level representations.
Ranked #3 on
Domain Generalization
on ImageNet-C
(using extra training data)
Generalizing Few-Shot NAS with Gradient Matching
Efficient performance estimation of architectures drawn from large search spaces is essential to Neural Architecture Search.
PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision
Existing self-supervised 3D human pose estimation schemes have largely relied on weak supervisions like consistency loss to guide the learning, which, inevitably, leads to inferior results in real-world scenarios with unseen poses.
Ranked #15 on
3D Human Pose Estimation
on MPI-INF-3DHP
SODAR: Segmenting Objects by DynamicallyAggregating Neighboring Mask Representations
Unlike the original per grid cell object masks, SODAR is implicitly supervised to learn mask representations that encode geometric structure of nearby objects and complement adjacent representations with context.
The Geometry of Robust Value Functions
The key of this perspective is to decompose the value space, in a state-wise manner, into unions of hypersurfaces.
Towards Adversarially Robust Deep Image Denoising
Secondly, to robustify DIDs, we propose an adversarial training strategy, hybrid adversarial training ({\sc HAT}), that jointly trains DIDs with adversarial and non-adversarial noisy data to ensure that the reconstruction quality is high and the denoisers around non-adversarial data are locally smooth.
UMAD: Universal Model Adaptation under Domain and Category Shift
To achieve bilateral adaptation in the target domain, we further maximize localized mutual information to align known samples with the source classifier and employ an entropic loss to push unknown samples far away from the source classification boundary, respectively.
Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning
Specifically, we experimentally show that directly encouraging CIL Learner at the initial phase to output similar representations as the model jointly trained on all classes can greatly boost the CIL performance.
Geometry-Guided Progressive NeRF for Generalizable and Efficient Neural Human Rendering
Meanwhile, for achieving higher rendering efficiency, we introduce a progressive rendering pipeline through geometry guidance, which leverages the geometric feature volume and the predicted density values to progressively reduce the number of sampling points and speed up the rendering process.
Towards Understanding Why Lookahead Generalizes Better Than SGD and Beyond
Specifically, we prove that lookahead using SGD as its inner-loop optimizer can better balance the optimization error and generalization error to achieve smaller excess risk error than vanilla SGD on (strongly) convex problems and nonconvex problems with Polyak-{\L}ojasiewicz condition which has been observed/proved in neural networks.
Shunted Self-Attention via Multi-Scale Token Aggregation
This novel merging scheme enables the self-attention to learn relationships between objects with different sizes and simultaneously reduces the token numbers and the computational cost.
MetaFormer Is Actually What You Need for Vision
Based on this observation, we hypothesize that the general architecture of the Transformers, instead of the specific token mixer module, is more essential to the model's performance.
Ranked #9 on
Semantic Segmentation
on DensePASS
Direct Multi-view Multi-person 3D Pose Estimation
Instead of estimating 3D joint locations from costly volumetric representation or reconstructing the per-person 3D pose from multiple detected 2D poses as in previous methods, MvP directly regresses the multi-person 3D poses in a clean and efficient way, without relying on intermediate tasks.
Ranked #3 on
3D Multi-Person Pose Estimation
on Panoptic
(using extra training data)
Deep Long-Tailed Learning: A Survey
Deep long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing deep models from a large number of images that follow a long-tailed class distribution.
Efficient Sharpness-aware Minimization for Improved Training of Neural Networks
Recently, the relation between the sharpness of the loss landscape and the generalization error has been established by Foret et al. (2020), in which the Sharpness Aware Minimizer (SAM) was proposed to mitigate the degradation of the generalization.
How Well Does Self-Supervised Pre-Training Perform with Streaming ImageNet?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
PnP-DETR: Towards Efficient Visual Analysis with Transformers
Recently, DETR pioneered the solution of vision tasks with transformers, it directly translates the image feature map into the object detection result.
DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture
In view of these, we propose DHA, which achieves joint optimization of Data augmentation policy, Hyper-parameter and Architecture.
Voxel Transformer for 3D Object Detection
We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds.
Ranked #3 on
3D Object Detection
on waymo vehicle
(L1 mAP metric)
Triplet Contrastive Learning for Brain Tumor Classification
Brain tumor is a common and fatal form of cancer which affects both adults and children.
Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition
Existing long-tailed recognition methods, aiming to train class-balanced models from long-tailed data, generally assume the models would be evaluated on the uniform test class distribution.
Ranked #4 on
Long-tail Learning
on CIFAR-10-LT (ρ=100)
Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawing
It enables learning high-quality Laplacian representations that faithfully approximate the ground truth.
Recovering the Unbiased Scene Graphs from the Biased Ones
Given input images, scene graph generation (SGG) aims to produce comprehensive, graphical representations describing visual relationships among salient objects.
Ranked #2 on
Unbiased Scene Graph Generation
on Visual Genome
VOLO: Vision Outlooker for Visual Recognition
Though recently the prevailing vision transformers (ViTs) have shown great potential of self-attention based models in ImageNet classification, their performance is still inferior to that of the latest SOTA CNNs if no extra data are provided.
Ranked #1 on
Semantic Segmentation
on Graz-02
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition
By realizing the importance of the positional information carried by 2D feature representations, unlike recent MLP-like models that encode the spatial information along the flattened spatial dimensions, Vision Permutator separately encodes the feature representations along the height and width dimensions with linear projections.
LV-BERT: Exploiting Layer Variety for BERT
In this paper, beyond this stereotyped layer pattern, we aim to improve pre-trained models by exploiting layer variety from two aspects: the layer type set and the layer order.
No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
Motivated by the above findings, we propose a novel and simple algorithm called Classifier Calibration with Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated gaussian mixture model.
Refiner: Refining Self-attention for Vision Transformers
Vision Transformers (ViTs) have shown competitive accuracy in image classification tasks compared with CNNs.
Ranked #151 on
Image Classification
on ImageNet
Image-to-Video Generation via 3D Facial Dynamics
Video generation from a single face image is an interesting problem and usually tackled by utilizing Generative Adversarial Networks (GANs) to integrate information from the input face image and a sequence of sparse facial landmarks.
PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively.
ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos
Detecting human-object interactions (HOI) is an important step toward a comprehensive visual understanding of machines.
Joint Face Image Restoration and Frontalization for Recognition
First, MDFR is a well-designed encoder-decoder architecture which extracts feature representation from an input face image with arbitrary low-quality factors and restores it to a high-quality counterpart.
Continual Learning via Bit-Level Information Preserving
Continual learning tackles the setting of learning different tasks sequentially.
PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation
To address this problem, we present PoseAug, a new auto-augmentation framework that learns to augment the available training poses towards a greater diversity and thus improve generalization of the trained 2D-to-3D pose estimator.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Body Meshes as Points
In this work, we present a single-stage model, Body Meshes as Points (BMP), to simplify the pipeline and lift both efficiency and performance.
Ranked #6 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
3D Human Shape Estimation
3D Multi-Person Pose Estimation
+1
How Well Does Self-Supervised Pre-Training Perform with Streaming Data?
Prior works on self-supervised pre-training focus on the joint training scenario, where massive unlabeled data are assumed to be given as input all at once, and only then is a learner trained.
All Tokens Matter: Token Labeling for Training Better Vision Transformers
In this paper, we present token labeling -- a new training objective for training high-performance vision transformers (ViTs).
Ranked #62 on
Semantic Segmentation
on ADE20K
DINE: Domain Adaptation from Single and Multiple Black-box Predictors
To ease the burden of labeling, unsupervised domain adaptation (UDA) aims to transfer knowledge in previous and related labeled datasets (sources) to a new unlabeled dataset (target).
Augmented Transformer with Adaptive Graph for Temporal Action Proposal Generation
Temporal action proposal generation (TAPG) is a fundamental and challenging task in video understanding, especially in temporal action detection.
AutoSpace: Neural Architecture Search with Less Human Interference
Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction.
DeepViT: Towards Deeper Vision Transformer
In this paper, we show that, unlike convolution neural networks (CNNs)that can be improved by stacking more convolutional layers, the performance of ViTs saturate fast when scaled to be deeper.
Ranked #364 on
Image Classification
on ImageNet
Coordinate Attention for Efficient Mobile Network Design
Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e. g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps.
Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning
In this paper, we investigate whether applying contrastive learning to fine-tuning would bring further benefits, and analytically find that optimizing the contrastive loss benefits both discriminative representation learning and model optimization during fine-tuning.
CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
By comparing \textit{non-robust} (normally trained) and \textit{robustified} (adversarially trained) models, we observe that adversarial training (AT) robustifies CNNs by aligning the channel-wise activations of adversarial data with those of their natural counterparts.
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
To overcome such limitations, we propose a new Tokens-To-Token Vision Transformer (T2T-ViT), which incorporates 1) a layer-wise Tokens-to-Token (T2T) transformation to progressively structurize the image to tokens by recursively aggregating neighboring Tokens into one Token (Tokens-to-Token), such that local structure represented by surrounding tokens can be modeled and tokens length can be reduced; 2) an efficient backbone with a deep-narrow structure for vision transformer motivated by CNN architecture design after empirical study.
Ranked #343 on
Image Classification
on ImageNet
ORDNet: Capturing Omni-Range Dependencies for Scene Parsing
Our ORDNet is able to extract more comprehensive context information and well adapt to complex spatial variance in scene images.
DiffAutoML: Differentiable Joint Optimization for Efficient End-to-End Automated Machine Learning
Our method performs co-optimization of the neural architectures, training hyper-parameters and data augmentation policies in an end-to-end fashion without the need of model retraining.
Exploring Balanced Feature Spaces for Representation Learning
Motivated by this question, we conduct a series of studies on the performance of self-supervised contrastive learning and supervised learning methods over multiple datasets where training instance distributions vary from a balanced one to a long-tailed one.
Ranked #30 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
AggMask: Exploring locally aggregated learning of mask representations for instance segmentation
Recently proposed one-stage instance segmentation models (\emph{e. g.}, SOLO) learn to directly predict location-specific object mask with fully-convolutional networks.
Learning Safe Policies with Cost-sensitive Advantage Estimation
Reinforcement Learning (RL) with safety guarantee is critical for agents performing tasks in risky environments.
Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
Furthermore, we propose a new labeling transfer strategy, which separates the target data into two splits based on the confidence of predictions (labeling information), and then employ semi-supervised learning to improve the accuracy of less-confident predictions in the target domain.
Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts
In particular, we propose a novel joint-training framework to train plain CNN by leveraging the gradients of the ResNet counterpart.
Fooling the primate brain with minimal, targeted image manipulation
Here we propose an array of methods for creating minimal, targeted image perturbations that lead to changes in both neuronal activity and perception as reflected in behavior.
Improving Generalization in Reinforcement Learning with Mixture Regularization
Deep reinforcement learning (RL) agents trained in a limited set of environments tend to suffer overfitting and fail to generalize to unseen testing environments.
Toward Accurate Person-level Action Recognition in Videos of Crowded Scenes
Prior works always fail to deal with this problem in two aspects: (1) lacking utilizing information of the scenes; (2) lacking training data in the crowd and complex scenes.
A Simple Baseline for Pose Tracking in Videos of Crowded Scenes
This paper presents our solution to ACM MM challenge: Large-scale Human-centric Video Analysis in Complex Events\cite{lin2020human}; specifically, here we focus on Track3: Crowd Pose Tracking in Complex Events.
Towards Accurate Human Pose Estimation in Videos of Crowded Scenes
In this paper, we focus on improving human pose estimation in videos of crowded scenes from the perspectives of exploiting temporal context and collecting new data.
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
The result shows that (1) the escaping time of both SGD and ADAM~depends on the Radon measure of the basin positively and the heaviness of gradient noise negatively; (2) for the same basin, SGD enjoys smaller escaping time than ADAM, mainly because (a) the geometry adaptation in ADAM~via adaptively scaling each gradient coordinate well diminishes the anisotropic structure in gradient noise and results in larger Radon measure of a basin; (b) the exponential gradient average in ADAM~smooths its gradient and leads to lighter gradient noise tails than SGD.
Visual Relationship Detection with Visual-Linguistic Knowledge from Multimodal Representations
Visual relationship detection aims to reason over relationships among salient objects in images, which has drawn increasing attention over the past few years.
Dual Adversarial Auto-Encoders for Clustering
By performing variational inference on the objective function of Dual-AAE, we derive a new reconstruction loss which can be optimized by training a pair of Auto-encoders.
ConvBERT: Improving BERT with Span-based Dynamic Convolution
The novel convolution heads, together with the rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context learning.
Few-shot Classification via Adaptive Attention
To be specific, we devise a simple and efficient meta-reweighting strategy to adapt the sample representations and generate soft attention to refine the representation such that the relevant features from the query and support samples can be extracted for a better few-shot classification.
The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation
Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset, and unveil that a major cause is the inaccurate classification of object proposals.
Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognition
Self-supervised learning (SSL) has been proved very effective at learning representations from unlabeled data in the image domain.
Domain Adaptation with Auxiliary Target Domain-Oriented Classifier
ATDOC alleviates the classifier bias by introducing an auxiliary classifier for target data only, to improve the quality of pseudo labels.
Rethinking Bottleneck Structure for Efficient Mobile Network Design
In this paper, we rethink the necessity of such design changes and find it may bring risks of information loss and gradient confusion.
Local Grid Rendering Networks for 3D Object Detection in Point Clouds
The performance of 3D object detection models over point clouds highly depends on their capability of modeling local geometric patterns.
Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation
In this work, we propose a novel framework, Inference Stage Optimization (ISO), for improving the generalizability of 3D pose models when source and target data come from different pose distributions.
Ranked #74 on
3D Human Pose Estimation
on 3DPW
Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
Solving long-tail large vocabulary object detection with deep learning based models is a challenging and demanding task, which is however under-explored. In this work, we provide the first systematic analysis on the underperformance of state-of-the-art models in front of long-tail distribution.
Multi-Miner: Object-Adaptive Region Mining for Weakly-Supervised Semantic Segmentation
In this paper, we propose a novel multi-miner framework to perform a region mining process that adapts to diverse object sizes and is thus able to mine more integral and finer object regions.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Understanding and Resolving Performance Degradation in Graph Convolutional Networks
In this work, we study performance degradation of GCNs by experimentally examining how stacking only TRANs or PROPs works.
Boosting Few-Shot Learning With Adaptive Margin Loss
Few-shot learning (FSL) has attracted increasing attention in recent years but remains challenging, due to the intrinsic difficulty in learning to generalize from a few examples.
Ranked #1 on
Few-Shot Image Classification
on ImageNet (1-shot)
RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
However, similar to existing preprocessing-based methods, the randomized process will degrade the prediction accuracy.
Semantic Domain Adversarial Networks for Unsupervised Domain Adaptation
To successfully align the multi-modal data structures across domains, the following works exploit discriminative information in the adversarial training process, e. g., using multiple class-wise discriminators and introducing conditional information in input or output of the domain discriminator.
Strip Pooling: Rethinking Spatial Pooling for Scene Parsing
Spatial pooling has been proven highly effective in capturing long-range contextual information for pixel-wise prediction tasks, such as scene parsing.
Ranked #26 on
Semantic Segmentation
on Cityscapes test
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation
On one hand, negative transfer results in misclassification of target samples to the classes only present in the source domain.
Ranked #2 on
Partial Domain Adaptation
on DomainNet
Cross-layer Feature Pyramid Network for Salient Object Detection
Feature pyramid network (FPN) based models, which fuse the semantics and salient details in a progressive manner, have been proven highly effective in salient object detection.
Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain.
Ranked #1 on
Domain Adaptation
on USPS-to-MNIST
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
Empirical results show that state-of-the-art models have an outstanding ability to capture biases contained in the dataset with high accuracy on EASY set.
Ranked #1 on
Logical Reasoning Question Answering
on ReClor
The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up
TADPOLE's unique results suggest that current prediction algorithms provide sufficient accuracy to exploit biomarkers related to clinical diagnosis and ventricle volume, for cohort refinement in clinical trials for Alzheimer's disease.
MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection
Recommender systems often face heterogeneous datasets containing highly personalized historical data of users, where no single model could give the best recommendation for every user.
PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection
Human and object points are the center of the detection boxes, and the interaction point is the midpoint of the human and object points.
Ranked #21 on
Human-Object Interaction Detection
on V-COCO
RC-DARTS: Resource Constrained Differentiable Architecture Search
In this paper, we propose the resource constrained differentiable architecture search (RC-DARTS) method to learn architectures that are significantly smaller and faster while achieving comparable accuracy.
Zoom in to where it matters: a hierarchical graph based model for mammogram analysis
In clinical practice, human radiologists actually review medical images with high resolution monitors and zoom into region of interests (ROIs) for a close-up examination.
Efficient Differentiable Neural Architecture Search with Meta Kernels
We demonstrate the effectiveness of our search strategy by conducting extensive experiments.
Efficient Meta Learning via Minibatch Proximal Update
We address the problem of meta-learning which learns a prior over hypothesis from a sample of meta-training tasks for fast adaptation on meta-testing tasks.
Classification Calibration for Long-tail Instance Segmentation
In this report, we investigate the performance drop phenomenon of state-of-the-art two-stage instance segmentation models when processing extreme long-tail training data based on the LVIS [5] dataset, and find a major cause is the inaccurate classification of object proposals.
Decoupling Representation and Classifier for Long-Tailed Recognition
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem.
Ranked #3 on
Long-tail learning with class descriptors
on CUB-LT
On Robustness of Neural Ordinary Differential Equations
We then provide an insightful understanding of this phenomenon by exploiting a certain desirable property of the flow of a continuous-time ODE, namely that integral curves are non-intersecting.
Compressed Video Action Recognition with Refined Motion Vector
Although CNN has reached satisfactory performance in image-related tasks, using CNN to process videos is much more challenging due to the enormous size of raw video streams.
Adaptive ROI Generation for Video Object Segmentation Using Reinforcement Learning
In this paper, we aim to tackle the task of semi-supervised video object segmentation across a sequence of frames where only the ground-truth segmentation of the first frame is provided.
Hierarchical Neural Architecture Search via Operator Clustering
Recently, the efficiency of automatic neural architecture design has been significantly improved by gradient-based search methods such as DARTS.
Towards Disentangling Non-Robust and Robust Components in Performance Metric
Then, we show by experiments that DNNs under standard training rely heavily on optimizing the non-robust component in achieving decent performance.
Prototype Recalls for Continual Learning
Without tampering with the performance on initial tasks, our method learns novel concepts given a few training examples of each class in new tasks.
Revisiting Knowledge Distillation via Label Smoothing Regularization
Without any extra computation cost, Tf-KD achieves up to 0. 65\% improvement on ImageNet over well-established baseline models, which is superior to label smoothing regularization.
PROTOTYPE-ASSISTED ADVERSARIAL LEARNING FOR UNSUPERVISED DOMAIN ADAPTATION
Previous adversarial learning methods condition domain alignment only on pseudo labels, but noisy and inaccurate pseudo labels may perturb the multi-class distribution embedded in probabilistic predictions, hence bringing insufficient alleviation to the latent mismatch problem.
Hierarchic Neighbors Embedding
Manifold learning now plays a very important role in machine learning and many relevant applications.
PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer
In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image.
Dynamic Kernel Distillation for Efficient Pose Estimation in Videos
Existing video-based human pose estimation methods extensively apply large networks onto every frame in the video to localize body joints, which suffer high computational cost and hardly meet the low-latency requirement in realistic applications.
Ranked #3 on
2D Human Pose Estimation
on JHMDB (2D poses only)
Single-Stage Multi-Person Pose Machines
Based on SPR, we develop the SPM model that can directly predict structured poses for multiple persons in a single stage, and thus offer a more compact pipeline and attractive efficiency advantage over two-stage methods.
Ranked #3 on
Keypoint Detection
on MPII Multi-Person
PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment
In this paper, we tackle the challenging few-shot segmentation problem from a metric learning perspective and present PANet, a novel prototype alignment network to better utilize the information of the support set.
Ranked #66 on
Few-Shot Semantic Segmentation
on PASCAL-5i (5-Shot)
Central Similarity Quantization for Efficient Image and Video Retrieval
In this work, we propose a new \emph{global} similarity metric, termed as \emph{central similarity}, with which the hash codes of similar data pairs are encouraged to approach a common center and those for dissimilar pairs to converge to different centers, to improve hash learning efficiency and retrieval accuracy.
Neural Epitome Search for Architecture-Agnostic Network Compression
The recent WSNet [1] is a new model compression method through sampling filterweights from a compact set and has demonstrated to be effective for 1D convolutionneural networks (CNNs).
Delving into 3D Action Anticipation from Streaming Videos
Action anticipation, which aims to recognize the action with a partial observation, becomes increasingly popular due to a wide range of applications.
PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction
We therefore propose a novel Position-Velocity Recurrent Encoder-Decoder (PVRED) for human motion prediction, which makes full use of pose velocities and temporal positional information.
Unsupervised Image Noise Modeling with Self-Consistent GAN
They jointly facilitate unsupervised learning of a noise model for various noise types.
Distilling Object Detectors with Fine-grained Feature Imitation
To address the challenge of distilling knowledge in detection model, we propose a fine-grained feature imitation method exploiting the cross-location discrepancy of feature response.
Understanding Adversarial Behavior of DNNs by Disentangling Non-Robust and Robust Components in Performance Metric
Despite many previous works studying the reason behind such adversarial behavior, the relationship between the generalization performance and adversarial behavior of DNNs is still little understood.
Query-efficient Meta Attack to Deep Neural Networks
Black-box attack methods aim to infer suitable attack patterns to targeted DNN models by only using output feedback of the models and the corresponding input queries.
Panoptic Edge Detection
In this work, we address a new finer-grained task, termed panoptic edge detection (PED), which aims at predicting semantic-level boundaries for stuff categories and instance-level boundaries for instance categories, in order to provide more comprehensive and unified scene understanding from the perspective of edges. We then propose a versatile framework, Panoptic Edge Network (PEN), which aggregates different tasks of object detection, semantic and instance edge detection into a single holistic network with multiple branches.
Deep Face Recognition Model Compression via Knowledge Transfer and Distillation
However, deploying such high performing models to resource constraint devices or real-time applications is challenging.
Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
Recent deep learning based face recognition methods have achieved great performance, but it still remains challenging to recognize very low-resolution query face like 28x28 pixels when CCTV camera is far from the captured subject.
Variational Prototype Replays for Continual Learning
In each classification task, our method learns a set of variational prototypes with their means and variances, where embedding of the samples from the same class can be represented in a prototypical distribution and class-representative prototypes are separated apart.
A Simple Pooling-Based Design for Real-Time Salient Object Detection
We further design a feature aggregation module (FAM) to make the coarse-level semantic information well fused with the fine-level features from the top-down pathway.
Ranked #1 on
RGB Salient Object Detection
on SOD
Hierarchical Meta Learning
Extensive experiments on few-shot classification and regression problems clearly demonstrate the superiority of HML over fine-tuning and state-of-the-art meta learning approaches in terms of generalization across heterogeneous tasks.
Cycle-SUM: Cycle-consistent Adversarial LSTM Networks for Unsupervised Video Summarization
The evaluator defines a learnable information preserving metric between original video and summary video and "supervises" the selector to identify the most informative frames to form the summary video.
Ranked #6 on
Unsupervised Video Summarization
on TvSum
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
Similarly, the output feature maps of a convolution layer can also be seen as a mixture of information at different frequencies.
Ranked #120 on
Action Classification
on Kinetics-400
Foreground-aware Pyramid Reconstruction for Alignment-free Occluded Person Re-identification
FPR uses the error from robust reconstruction over spatial pyramid features to measure similarities between two persons.
Few-shot Adaptive Faster R-CNN
To address these challenges, we first introduce a pairing mechanism over source and target features to alleviate the issue of insufficient target domain samples.
3D Face Reconstruction from A Single Image Assisted by 2D Face Images in the Wild
3D face reconstruction from a single 2D image is a challenging problem with broad applications.
Ranked #7 on
Face Alignment
on AFLW2000-3D
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search
It prunes the architecture search space with a partial order assumption to automatically search for the architectures with the best speed and accuracy trade-off.
Dynamic Feature Fusion for Semantic Edge Detection
In this work, we propose a novel dynamic feature fusion strategy that assigns different fusion weights for different input images and locations adaptively.
Multi-Prototype Networks for Unconstrained Set-based Face Recognition
In this paper, we study the challenging unconstrained set-based face recognition problem where each subject face is instantiated by a set of media (images and videos) instead of a single image.
Deep Reasoning with Multi-Scale Context for Salient Object Detection
However, the saliency inference module that performs saliency prediction from the fused features receives much less attention on its architecture design and typically adopts only a few fully convolutional layers.
Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing
Face anti-spoofing (a. k. a presentation attack detection) has drawn growing attention due to the high-security demand in face authentication systems.
Ranked #2 on
Face Anti-Spoofing
on MSU-MFSD
Better Guider Predicts Future Better: Difference Guided Generative Adversarial Networks
In this paper, we propose a novel GAN based on inter-frame difference to circumvent the difficulties.
Similarity R-C3D for Few-shot Temporal Activity Detection
In this paper, we present a conceptually simple and general yet novel framework for few-shot temporal activity detection which detects the start and end time of the few-shot input activities in an untrimmed video.
Few-shot Object Detection via Feature Reweighting
The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples.
Ranked #18 on
Few-Shot Object Detection
on MS-COCO (30-shot)
A^2-Nets: Double Attention Networks
Learning to capture long-range relations is fundamental to image/video recognition.