Search Results for author:
Found 308 papers, 126 papers with code
ConvBERT: Improving BERT with Span-based Dynamic Convolution
The novel convolution heads, together with the rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context learning.
MetaFormer Is Actually What You Need for Vision
Based on this observation, we hypothesize that the general architecture of the Transformers, instead of the specific token mixer module, is more essential to the model's performance.
Ranked #9 on
Semantic Segmentation
on DensePASS
Dual Path Networks
In this work, we present a simple, highly efficient and modularized Dual Path Network (DPN) for image classification which presents a new topology of connection paths internally.
VOLO: Vision Outlooker for Visual Recognition
Though recently the prevailing vision transformers (ViTs) have shown great potential of self-attention based models in ImageNet classification, their performance is still inferior to that of the latest SOTA CNNs if no extra data are provided.
Ranked #1 on
Image Classification
on VizWiz-Classification
MetaFormer Baselines for Vision
By simply applying depthwise separable convolutions as token mixer in the bottom stages and vanilla self-attention in the top stages, the resulting model CAFormer sets a new record on ImageNet-1K: it achieves an accuracy of 85. 5% at 224x224 resolution, under normal supervised training without external data or distillation.
Ranked #2 on
Domain Generalization
on ImageNet-C
(using extra training data)
MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model
Existing animation works typically employ the frame-warping technique to animate the reference image towards the target motion.
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
To this end, we scale up the dataset by designing a data engine to collect and automatically annotate large-scale unlabeled data (~62M), which significantly enlarges the data coverage and thus is able to reduce the generalization error.
Ranked #2 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
Understanding Humans in Crowded Scenes: Deep Nested Adversarial Learning and A New Benchmark for Multi-Human Parsing
Despite the noticeable progress in perceptual tasks like detection, instance segmentation and human parsing, computers still perform unsatisfactorily on visually understanding humans in crowded scenes, such as group behavior analysis, person re-identification and autonomous driving, etc.
Ranked #1 on
Multi-Human Parsing
on PASCAL-Part
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet
To overcome such limitations, we propose a new Tokens-To-Token Vision Transformer (T2T-ViT), which incorporates 1) a layer-wise Tokens-to-Token (T2T) transformation to progressively structurize the image to tokens by recursively aggregating neighboring Tokens into one Token (Tokens-to-Token), such that local structure represented by surrounding tokens can be modeled and tokens length can be reduced; 2) an efficient backbone with a deep-narrow structure for vision transformer motivated by CNN architecture design after empirical study.
Ranked #400 on
Image Classification
on ImageNet
Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution
Similarly, the output feature maps of a convolution layer can also be seen as a mixture of information at different frequencies.
Ranked #147 on
Action Classification
on Kinetics-400
Revisiting Knowledge Distillation via Label Smoothing Regularization
Without any extra computation cost, Tf-KD achieves up to 0. 65\% improvement on ImageNet over well-established baseline models, which is superior to label smoothing regularization.
Rethinking Bottleneck Structure for Efficient Mobile Network Design
In this paper, we rethink the necessity of such design changes and find it may bring risks of information loss and gradient confusion.
Coordinate Attention for Efficient Mobile Network Design
Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e. g., the Squeeze-and-Excitation attention) for lifting model performance, but they generally neglect the positional information, which is important for generating spatially selective attention maps.
Decoupling Representation and Classifier for Long-Tailed Recognition
The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem.
Ranked #3 on
Long-tail learning with class descriptors
on CUB-LT
PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer
In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image.
PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively.
Multiple-Human Parsing in the Wild
To address the multi-human parsing problem, we introduce a new multi-human parsing (MHP) dataset and a novel multi-human parsing model named MH-Parser.
Ranked #3 on
Multi-Human Parsing
on MHP v1.0
A Simple Pooling-Based Design for Real-Time Salient Object Detection
We further design a feature aggregation module (FAM) to make the coarse-level semantic information well fused with the fine-level features from the top-down pathway.
Ranked #1 on
RGB Salient Object Detection
on SOD
Few-shot Object Detection via Feature Reweighting
The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples.
Ranked #21 on
Few-Shot Object Detection
on MS-COCO (30-shot)
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
Our experiments show that BuboGPT achieves impressive multi-modality understanding and visual grounding abilities during the interaction with human.
3D Face Reconstruction from A Single Image Assisted by 2D Face Images in the Wild
3D face reconstruction from a single 2D image is a challenging problem with broad applications.
Ranked #7 on
Face Alignment
on AFLW2000-3D
Understanding The Robustness in Vision Transformers
Our study is motivated by the intriguing properties of the emerging visual grouping in Vision Transformers, which indicates that self-attention may promote robustness through improved mid-level representations.
Ranked #4 on
Domain Generalization
on ImageNet-R
(using extra training data)
Deep Long-Tailed Learning: A Survey
Deep long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing deep models from a large number of images that follow a long-tailed class distribution.
All Tokens Matter: Token Labeling for Training Better Vision Transformers
In this paper, we present token labeling -- a new training objective for training high-performance vision transformers (ViTs).
Ranked #3 on
Efficient ViTs
on ImageNet-1K (With LV-ViT-S)
Distilling Object Detectors with Fine-grained Feature Imitation
To address the challenge of distilling knowledge in detection model, we propose a fine-grained feature imitation method exploiting the cross-location discrepancy of feature response.
Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) aims to leverage the knowledge learned from a labeled source dataset to solve similar tasks in a new unlabeled domain.
Ranked #1 on
Source-Free Domain Adaptation
on VisDA-2017
DINE: Domain Adaptation from Single and Multiple Black-box Predictors
To ease the burden of labeling, unsupervised domain adaptation (UDA) aims to transfer knowledge in previous and related labeled datasets (sources) to a new unlabeled dataset (target).
Magic-Me: Identity-Specific Video Customized Diffusion
To achieve this, we propose three novel components that are essential for high-quality identity preservation and stable video generation: 1) a noise initialization method with 3D Gaussian Noise Prior for better inter-frame stability; 2) an ID module based on extended Textual Inversion trained with the cropped identity to disentangle the ID information from the background 3) Face VCD and Tiled VCD modules to reinforce faces and upscale the video to higher resolution while preserving the identity's features.
Strip Pooling: Rethinking Spatial Pooling for Scene Parsing
Spatial pooling has been proven highly effective in capturing long-range contextual information for pixel-wise prediction tasks, such as scene parsing.
Ranked #32 on
Semantic Segmentation
on Cityscapes test
No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data
Motivated by the above findings, we propose a novel and simple algorithm called Classifier Calibration with Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated gaussian mixture model.
Look Across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition
Benchmarking our model on one of the most popular unconstrained face recognition datasets IJB-C additionally verifies the promising generalizability of AIM in recognizing faces in the wild.
Ranked #1 on
Age-Invariant Face Recognition
on MORPH Album2
PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation
To address this problem, we present PoseAug, a new auto-augmentation framework that learns to augment the available training poses towards a greater diversity and thus improve generalization of the trained 2D-to-3D pose estimator.
Ranked #1 on
Monocular 3D Human Pose Estimation
on Human3.6M
(Use Video Sequence metric)
Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax
Solving long-tail large vocabulary object detection with deep learning based models is a challenging and demanding task, which is however under-explored. In this work, we provide the first systematic analysis on the underperformance of state-of-the-art models in front of long-tail distribution.
Graph-Based Global Reasoning Networks
In this work, we propose a new approach for reasoning globally in which a set of features are globally aggregated over the coordinate space and then projected to an interaction space where relational reasoning can be efficiently computed.
Correlation Alignment for Unsupervised Domain Adaptation
In contrast to subspace manifold methods, it aligns the original feature distributions of the source and target domains, rather than the bases of lower-dimensional subspaces.
Ranked #8 on
Domain Adaptation
on Office-Caltech
Domain Adaptation with Auxiliary Target Domain-Oriented Classifier
ATDOC alleviates the classifier bias by introducing an auxiliary classifier for target data only, to improve the quality of pseudo labels.
Direct Multi-view Multi-person 3D Pose Estimation
Instead of estimating 3D joint locations from costly volumetric representation or reconstructing the per-person 3D pose from multiple detected 2D poses as in previous methods, MvP directly regresses the multi-person 3D poses in a clean and efficient way, without relying on intermediate tasks.
Ranked #3 on
3D Multi-Person Pose Estimation
on Panoptic
(using extra training data)
PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision
Existing self-supervised 3D human pose estimation schemes have largely relied on weak supervisions like consistency loss to guide the learning, which, inevitably, leads to inferior results in real-world scenarios with unseen poses.
Ranked #37 on
3D Human Pose Estimation
on MPI-INF-3DHP
PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment
In this paper, we tackle the challenging few-shot segmentation problem from a metric learning perspective and present PANet, a novel prototype alignment network to better utilize the information of the support set.
Ranked #70 on
Few-Shot Semantic Segmentation
on COCO-20i (5-shot)
AvatarGen: a 3D Generative Model for Animatable Human Avatars
Unsupervised generation of clothed virtual humans with various appearance and animatable poses is important for creating 3D human avatars and other AR/VR applications.
AvatarGen: A 3D Generative Model for Animatable Human Avatars
Specifically, we decompose the generative 3D human synthesis into pose-guided mapping and canonical representation with predefined human pose and shape, such that the canonical representation can be explicitly driven to different poses and shapes with the guidance of a 3D parametric human model SMPL.
Voxel Transformer for 3D Object Detection
We present Voxel Transformer (VoTr), a novel and effective voxel-based Transformer backbone for 3D object detection from point clouds.
Ranked #3 on
3D Object Detection
on waymo vehicle
(L1 mAP metric)
Dataset Quantization
Extensive experiments demonstrate that DQ is able to generate condensed small datasets for training unseen network architectures with state-of-the-art compression ratios for lossless model training.
Central Similarity Quantization for Efficient Image and Video Retrieval
In this work, we propose a new \emph{global} similarity metric, termed as \emph{central similarity}, with which the hash codes of similar data pairs are encouraged to approach a common center and those for dissimilar pairs to converge to different centers, to improve hash learning efficiency and retrieval accuracy.
PPDM: Parallel Point Detection and Matching for Real-time Human-Object Interaction Detection
Human and object points are the center of the detection boxes, and the interaction point is the midpoint of the human and object points.
Ranked #25 on
Human-Object Interaction Detection
on V-COCO
Dynamic Feature Fusion for Semantic Edge Detection
In this work, we propose a novel dynamic feature fusion strategy that assigns different fusion weights for different input images and locations adaptively.
Shunted Self-Attention via Multi-Scale Token Aggregation
This novel merging scheme enables the self-attention to learn relationships between objects with different sizes and simultaneously reduces the token numbers and the computational cost.
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition
By realizing the importance of the positional information carried by 2D feature representations, unlike recent MLP-like models that encode the spatial information along the flattened spatial dimensions, Vision Permutator separately encodes the feature representations along the height and width dimensions with linear projections.
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning
With the proposed SSF, our model obtains 2. 46% (90. 72% vs. 88. 54%) and 11. 48% (73. 10% vs. 65. 57%) performance improvement on FGVC and VTAB-1k in terms of Top-1 accuracy compared to the full fine-tuning but only fine-tuning about 0. 3M parameters.
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search
It prunes the architecture search space with a partial order assumption to automatically search for the architectures with the best speed and accuracy trade-off.
Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition
Existing long-tailed recognition methods, aiming to train class-balanced models from long-tailed data, generally assume the models would be evaluated on the uniform test class distribution.
Ranked #7 on
Long-tail Learning
on iNaturalist 2018
DeepViT: Towards Deeper Vision Transformer
In this paper, we show that, unlike convolution neural networks (CNNs)that can be improved by stacking more convolutional layers, the performance of ViTs saturate fast when scaled to be deeper.
Ranked #423 on
Image Classification
on ImageNet
PnP-DETR: Towards Efficient Visual Analysis with Transformers
Recently, DETR pioneered the solution of vision tasks with transformers, it directly translates the image feature map into the object detection result.
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition
This paper does not attempt to design a state-of-the-art method for visual recognition but investigates a more efficient way to make use of convolutions to encode spatial features.
Single-Stage Multi-Person Pose Machines
Based on SPR, we develop the SPM model that can directly predict structured poses for multiple persons in a single stage, and thus offer a more compact pipeline and attractive efficiency advantage over two-stage methods.
Ranked #3 on
Keypoint Detection
on MPII Multi-Person
Natural Language Object Retrieval
In this paper, we address the task of natural language object retrieval, to localize a target object within a given image based on a natural language query of the object.
Ranked #12 on
Referring Expression Comprehension
on Talk2Car
Tensor Robust Principal Component Analysis with A New Tensor Nuclear Norm
Equipped with the new tensor nuclear norm, we then solve the TRPCA problem by solving a convex program and provide the theoretical guarantee for the exact recovery.
DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks
The performance of deep neural networks is well-known to be sensitive to the setting of their hyperparameters.
Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer
Furthermore, we propose a new labeling transfer strategy, which separates the target data into two splits based on the confidence of predictions (labeling information), and then employ semi-supervised learning to improve the accuracy of less-confident predictions in the target domain.
Refiner: Refining Self-attention for Vision Transformers
Vision Transformers (ViTs) have shown competitive accuracy in image classification tasks compared with CNNs.
Ranked #172 on
Image Classification
on ImageNet
Classification Calibration for Long-tail Instance Segmentation
In this report, we investigate the performance drop phenomenon of state-of-the-art two-stage instance segmentation models when processing extreme long-tail training data based on the LVIS [5] dataset, and find a major cause is the inaccurate classification of object proposals.
The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation
Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset, and unveil that a major cause is the inaccurate classification of object proposals.
Body Meshes as Points
In this work, we present a single-stage model, Body Meshes as Points (BMP), to simplify the pipeline and lift both efficiency and performance.
Ranked #9 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
3D Human Shape Estimation
3D Multi-Person Pose Estimation
+1
Generative Partition Networks for Multi-Person Pose Estimation
This paper proposes a new Generative Partition Network (GPN) to address the challenging multi-person pose estimation problem.
Ranked #1 on
Multi-Person Pose Estimation
on WAF
(AP metric)
Deep Joint Rain Detection and Removal from a Single Image
Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output.
Revisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring
In this paper, based on the CLIP model, we revisit temporal modeling in the context of image-to-video knowledge transferring, which is the key point for extending image-text pretrained models to the video domain.
Ranked #7 on
Video Retrieval
on MSR-VTT-1kA
(using extra training data)
Recovering the Unbiased Scene Graphs from the Biased Ones
Given input images, scene graph generation (SGG) aims to produce comprehensive, graphical representations describing visual relationships among salient objects.
Ranked #2 on
Unbiased Scene Graph Generation
on Visual Genome
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
Empirical results show that state-of-the-art models have an outstanding ability to capture biases contained in the dataset with high accuracy on EASY set.
Ranked #1 on
Logical Reasoning Question Answering
on ReClor
Harnessing Diffusion Models for Visual Perception with Meta Prompts
Our key insight is to introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception.
Ranked #2 on
Semantic Segmentation
on Cityscapes test
(using extra training data)
Efficient Sharpness-aware Minimization for Improved Training of Neural Networks
Recently, the relation between the sharpness of the loss landscape and the generalization error has been established by Foret et al. (2020), in which the Sharpness Aware Minimizer (SAM) was proposed to mitigate the degradation of the generalization.
Expanding Small-Scale Datasets with Guided Imagination
Specifically, GIF conducts data imagination by optimizing the latent features of the seed data in the semantically meaningful space of the prior model, resulting in the creation of photo-realistic images with new content.
Contrastive Masked Autoencoders are Stronger Vision Learners
The momentum encoder, fed with the full images, enhances the feature discriminability via contrastive learning with its online counterpart.
ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos
Detecting human-object interactions (HOI) is an important step toward a comprehensive visual understanding of machines.
Ranked #3 on
Human-Object Interaction Anticipation
on VidHOI
Query-efficient Meta Attack to Deep Neural Networks
Black-box attack methods aim to infer suitable attack patterns to targeted DNN models by only using output feedback of the models and the corresponding input queries.
The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up
TADPOLE's unique results suggest that current prediction algorithms provide sufficient accuracy to exploit biomarkers related to clinical diagnosis and ventricle volume, for cohort refinement in clinical trials for Alzheimer's disease.
A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation
On one hand, negative transfer results in misclassification of target samples to the classes only present in the source domain.
Ranked #2 on
Partial Domain Adaptation
on ImageNet-Caltech
Divide to Adapt: Mitigating Confirmation Bias for Domain Adaptation of Black-Box Predictors
Domain Adaptation of Black-box Predictors (DABP) aims to learn a model on an unlabeled target domain supervised by a black-box predictor trained on a source domain.
Clover: Towards A Unified Video-Language Alignment and Fusion Model
We then introduce \textbf{Clover}\textemdash a Correlated Video-Language pre-training method\textemdash towards a universal Video-Language model for solving multiple video understanding tasks with neither performance nor efficiency compromise.
Ranked #1 on
Video Question Answering
on LSMDC-FiB
Continual Learning via Bit-Level Information Preserving
Continual learning tackles the setting of learning different tasks sequentially.
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model
Due to the limited scale and quality of video-text training corpus, most vision-language foundation models employ image-text datasets for pretraining and primarily focus on modeling visually semantic representations while disregarding temporal semantic representations and correlations.
Ranked #1 on
TGIF-Frame
on TGIF-QA
(using extra training data)
Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning
Specifically, we experimentally show that directly encouraging CIL Learner at the initial phase to output similar representations as the model jointly trained on all classes can greatly boost the CIL performance.
TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision
We conduct extensive experiments to analyze each of our proposed components and show the efficacy of our framework in generating high-fidelity 3D textured and text-relevant shapes.
Learning Detection with Diverse Proposals
To predict a set of diverse and informative proposals with enriched representations, this paper introduces a differentiable Determinantal Point Process (DPP) layer that is able to augment the object detection architectures.
A^2-Nets: Double Attention Networks
Learning to capture long-range relations is fundamental to image/video recognition.
Deep Future Gaze: Gaze Anticipation on Egocentric Videos Using Adversarial Networks
Through competition with discriminator, the generator progressively improves quality of the future frames and thus anticipates future gaze better.
Improving Generalization in Reinforcement Learning with Mixture Regularization
Deep reinforcement learning (RL) agents trained in a limited set of environments tend to suffer overfitting and fail to generalize to unseen testing environments.
Few-shot Classification via Adaptive Attention
To be specific, we devise a simple and efficient meta-reweighting strategy to adapt the sample representations and generate soft attention to refine the representation such that the relevant features from the query and support samples can be extracted for a better few-shot classification.
Understanding and Resolving Performance Degradation in Graph Convolutional Networks
In this work, we study performance degradation of GCNs by experimentally examining how stacking only TRANs or PROPs works.
MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration
Large Language Models (LLMs) have marked a significant advancement in the field of natural language processing, demonstrating exceptional capabilities in reasoning, tool usage, and memory.
STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation
Then, a better network called Enhanced-DCNN is learned with supervision from the predicted segmentation masks of simple images based on the Initial-DCNN as well as the image-level annotations.
Sharpness-Aware Training for Free
Intuitively, SAF achieves this by avoiding sudden drops in the loss in the sharp local minima throughout the trajectory of the updates of the weights.
MagicMix: Semantic Mixing with Diffusion Models
Unlike style transfer, where an image is stylized according to the reference style without changing the image content, semantic blending mixes two different concepts in a semantic manner to synthesize a novel concept while preserving the spatial layout and geometry.
Return of Frustratingly Easy Domain Adaptation
Unlike human learning, machine learning often fails to handle changes between training (source) and test (target) input distributions.
Ranked #4 on
Domain Adaptation
on Synth Digits-to-SVHN
Exact Low Tubal Rank Tensor Recovery from Gaussian Measurements
Specifically, we show that by solving a TNN minimization problem, the underlying tensor of size $n_1\times n_2\times n_3$ with tubal rank $r$ can be exactly recovered when the given number of Gaussian measurements is $O(r(n_1+n_2-r)n_3)$.
Video Recognition in Portrait Mode
While existing datasets mainly comprise landscape mode videos, our paper seeks to introduce portrait mode videos to the research community and highlight the unique challenges associated with this video format.
Generalizing Few-Shot NAS with Gradient Matching
Efficient performance estimation of architectures drawn from large search spaces is essential to Neural Architecture Search.
AggMask: Exploring locally aggregated learning of mask representations for instance segmentation
Recently proposed one-stage instance segmentation models (\emph{e. g.}, SOLO) learn to directly predict location-specific object mask with fully-convolutional networks.
CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection
By comparing \textit{non-robust} (normally trained) and \textit{robustified} (adversarially trained) models, we observe that adversarial training (AT) robustifies CNNs by aligning the channel-wise activations of adversarial data with those of their natural counterparts.
Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning
In this paper, we investigate whether applying contrastive learning to fine-tuning would bring further benefits, and analytically find that optimizing the contrastive loss benefits both discriminative representation learning and model optimization during fine-tuning.
SODAR: Segmenting Objects by DynamicallyAggregating Neighboring Mask Representations
Unlike the original per grid cell object masks, SODAR is implicitly supervised to learn mask representations that encode geometric structure of nearby objects and complement adjacent representations with context.
Hierarchical Neural Architecture Search via Operator Clustering
Recently, the efficiency of automatic neural architecture design has been significantly improved by gradient-based search methods such as DARTS.
LV-BERT: Exploiting Layer Variety for BERT
In this paper, beyond this stereotyped layer pattern, we aim to improve pre-trained models by exploiting layer variety from two aspects: the layer type set and the layer order.
Visual Relationship Detection with Visual-Linguistic Knowledge from Multimodal Representations
Visual relationship detection aims to reason over relationships among salient objects in images, which has drawn increasing attention over the past few years.
Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawing
It enables learning high-quality Laplacian representations that faithfully approximate the ground truth.
A Simple Loss Function for Improving the Convergence and Accuracy of Visual Question Answering Models
On the other hand, very little focus has been put on the models' loss function, arguably one of the most important aspects of training deep learning models.
ManiCLIP: Multi-Attribute Face Manipulation from Text
In this paper we present a novel multi-attribute face manipulation method based on textual descriptions.
PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction
We therefore propose a novel Position-Velocity Recurrent Encoder-Decoder (PVRED) for human motion prediction, which makes full use of pose velocities and temporal positional information.
Adversarial Complementary Learning for Weakly Supervised Object Localization
With such an adversarial learning, the two parallel-classifiers are forced to leverage complementary object regions for classification and can finally generate integral object localization together.
Ranked #2 on
Weakly-Supervised Object Localization
on ILSVRC 2016
On Robustness of Neural Ordinary Differential Equations
We then provide an insightful understanding of this phenomenon by exploiting a certain desirable property of the flow of a continuous-time ODE, namely that integral curves are non-intersecting.
Global Knowledge Calibration for Fast Open-Vocabulary Segmentation
Recent advancements in pre-trained vision-language models, such as CLIP, have enabled the segmentation of arbitrary concepts solely from textual inputs, a process commonly referred to as open-vocabulary semantic segmentation (OVS).
Knowledge Distillation
Open Vocabulary Semantic Segmentation
+4
A Good Practice Towards Top Performance of Face Recognition: Transferred Deep Feature Fusion
In this paper, we propose a unified learning framework named Transferred Deep Feature Fusion (TDFF) targeting at the new IARPA Janus Benchmark A (IJB-A) face recognition dataset released by NIST face challenge.
Egocentric Spatial Memory
Egocentric spatial memory (ESM) defines a memory system with encoding, storing, recognizing and recalling the spatial information about the environment from an egocentric perspective.
Variational Prototype Replays for Continual Learning
In each classification task, our method learns a set of variational prototypes with their means and variances, where embedding of the samples from the same class can be represented in a prototypical distribution and class-representative prototypes are separated apart.
Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing
Face anti-spoofing (a. k. a presentation attack detection) has drawn growing attention due to the high-security demand in face authentication systems.
Ranked #2 on
Face Anti-Spoofing
on MSU-MFSD
Adaptive ROI Generation for Video Object Segmentation Using Reinforcement Learning
In this paper, we aim to tackle the task of semi-supervised video object segmentation across a sequence of frames where only the ground-truth segmentation of the first frame is provided.
AutoSpace: Neural Architecture Search with Less Human Interference
Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction.
Deep Learning with S-shaped Rectified Linear Activation Units
Rectified linear activation units are important components for state-of-the-art deep convolutional networks.
Video-based Person Re-identification with Accumulative Motion Context
Video based person re-identification plays a central role in realistic security and video surveillance.
Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts
In particular, we propose a novel joint-training framework to train plain CNN by leveraging the gradients of the ResNet counterpart.
Towards Understanding Why Lookahead Generalizes Better Than SGD and Beyond
Specifically, we prove that lookahead using SGD as its inner-loop optimizer can better balance the optimization error and generalization error to achieve smaller excess risk error than vanilla SGD on (strongly) convex problems and nonconvex problems with Polyak-{\L}ojasiewicz condition which has been observed/proved in neural networks.
RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
However, similar to existing preprocessing-based methods, the randomized process will degrade the prediction accuracy.
Class Prototype-based Cleaner for Label Noise Learning
Semi-supervised learning based methods are current SOTA solutions to the noisy-label learning problem, which rely on learning an unsupervised label cleaner first to divide the training samples into a labeled set for clean data and an unlabeled set for noise data.
Ranked #3 on
Image Classification
on Clothing1M
(using extra training data)
Understanding Generalization and Optimization Performance of Deep CNNs
Besides, we prove that for an arbitrary gradient descent algorithm, the computed approximate stationary point by minimizing empirical risk is also an approximate stationary point to the population risk.
Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach
We investigate a principle way to progressively mine discriminative object regions using classification networks to address the weakly-supervised semantic segmentation problems.
Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation
It can produce dense and reliable object localization maps and effectively benefit both weakly- and semi- supervised semantic segmentation.
Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization
In this work, we prove that under certain suitable assumptions, we can recover both the low-rank and the sparse components exactly by simply solving a convex program whose objective is a weighted combination of the tensor nuclear norm and the $\ell_1$-norm, i. e., $\min_{{\mathcal{L}},\ {\mathcal{E}}} \ \|{{\mathcal{L}}}\|_*+\lambda\|{{\mathcal{E}}}\|_1, \ \text{s. t.}
Subspace Clustering by Block Diagonal Representation
Second, we observe that many existing methods approximate the block diagonal representation matrix by using different structure priors, e. g., sparsity and low-rankness, which are indirect.
WSNet: Compact and Efficient Networks Through Weight Sampling
We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks.
Learning Markov Clustering Networks for Scene Text Detection
MCN predicts instance-level bounding boxes by firstly converting an image into a Stochastic Flow Graph (SFG) and then performing Markov Clustering on this graph.
Learning Pixel-wise Labeling from the Internet without Human Interaction
Collecting sufficient annotated data is very expensive in many applications, especially for pixel-level prediction tasks such as semantic segmentation.
Transferable Semi-supervised Semantic Segmentation
The performance of deep learning based semantic segmentation models heavily depends on sufficient data with careful annotations.
Zigzag Learning for Weakly Supervised Object Detection
Unlike them, we propose a zigzag learning strategy to simultaneously discover reliable object instances and prevent the model from overfitting initial seeds.
Left-Right Comparative Recurrent Model for Stereo Matching
Left-right consistency check is an effective way to enhance the disparity estimation by referring to the information from the opposite view.
Multi-View Image Generation from a Single-View
This paper addresses a challenging problem -- how to generate multi-view cloth images from only a single view input.
Stochastic Primal-Dual Proximal ExtraGradient Descent for Compositely Regularized Optimization
This optimization model abstracts a number of important applications in artificial intelligence and machine learning, such as fused Lasso, fused logistic regression, and a class of graph-guided regularized minimization.
Cross-domain Human Parsing via Adversarial Feature and Label Adaptation
Our proposed model explicitly learns a feature compensation network, which is specialized for mitigating the cross-domain differences.
Weaving Multi-scale Context for Single Shot Detector
For 320x320 input of batch size = 8, WeaveNet reaches 79. 5% mAP on PASCAL VOC 2007 test in 101 fps with only 4 fps extra cost, and further improves to 79. 7% mAP with more iterations.
Nonconvex Sparse Spectral Clustering by Alternating Direction Method of Multipliers and Its Convergence Analysis
Experimental analysis on several real data sets verifies the effectiveness of our method.
Personalized and Occupational-aware Age Progression by Generative Adversarial Networks
Second, we propose a new occupational-aware adversarial face aging network, which learns human aging process under different occupations.
HashGAN:Attention-aware Deep Adversarial Hashing for Cross Modal Retrieval
The proposed new adversarial network, HashGAN, consists of three building blocks: 1) the feature learning module to obtain feature representations, 2) the generative attention module to generate an attention mask, which is used to obtain the attended (foreground) and the unattended (background) feature representations, 3) the discriminative hash coding module to learn hash functions that preserve the similarities between different modalities.
Integrated Face Analytics Networks through Cross-Dataset Hybrid Training
Specifically, iFAN achieves an overall F-score of 91. 15% on the Helen dataset for face parsing, a normalized mean error of 5. 81% on the MTFL dataset for facial landmark localization and an accuracy of 45. 73% on the BNU dataset for emotion recognition with a single model.
Predicting Scene Parsing and Motion Dynamics in the Future
The ability of predicting the future is important for intelligent systems, e. g. autonomous vehicles and robots to plan early and make decisions accordingly.
Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms
As most deep learning algorithms are stochastic (e. g., Stochastic Gradient Descent, Dropout, and Bayes-by-backprop), we revisit the robustness arguments of Xu & Mannor, and introduce a new approach, ensemble robustness, that concerns the robustness of a population of hypotheses.
Deep Sparse Subspace Clustering
In this paper, we present a deep extension of Sparse Subspace Clustering, termed Deep Sparse Subspace Clustering (DSSC).
Discriminative Similarity for Clustering and Semi-Supervised Learning
By generalization analysis via Rademacher complexity, the generalization error bound for the kernel classifier learned from hypothetical labeling is expressed as the sum of pairwise similarity between the data from different classes, parameterized by the weights of the kernel classifier.
On the Suboptimality of Proximal Gradient Descent for $\ell^{0}$ Sparse Approximation
We study the proximal gradient descent (PGD) method for $\ell^{0}$ sparse approximation problem as well as its accelerated optimization with randomized algorithms in this paper.
Self-explanatory Deep Salient Object Detection
Through visualizing the differences, we can interpret the capability of different deep neural networks based saliency detection models and demonstrate that our proposed model indeed uses more reasonable structure for salient object detection.
Training Group Orthogonal Neural Networks with Privileged Information
Learning rich and diverse representations is critical for the performance of deep convolutional neural networks (CNNs).
Learning with Rethinking: Recurrently Improving Convolutional Neural Networks through Feedback
However, most of the existing CNN models only learn features through a feedforward structure and no feedback information from top to bottom layers is exploited to enable the networks to refine themselves.
FoveaNet: Perspective-aware Urban Scene Parsing
Thus, they suffer from heterogeneous object scales caused by perspective projection of cameras on actual scenes and inevitably encounter parsing failures on distant objects as well as other boundary and recognition errors.
The Landscape of Deep Learning Algorithms
For an $l$-layer linear neural network, we prove its empirical risk uniformly converges to its population risk at the rate of $\mathcal{O}(r^{2l}\sqrt{d\log(l)}/\sqrt{n})$ with training sample size of $n$, the total weight dimension of $d$ and the magnitude bound $r$ of weight of each layer.
Accelerated Randomized Mirror Descent Algorithms For Composite Non-strongly Convex Optimization
Avoiding this devise, we propose an accelerated randomized mirror descent method for solving this problem without the strongly convex assumption.
Neural Person Search Machines
We investigate the problem of person search in the wild in this work.
Ranked #4 on
Person Re-Identification
on CUHK-SYSU
Perceptual Generative Adversarial Networks for Small Object Detection
In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to "super-resolved" ones, achieving similar characteristics as large objects and thus more discriminative for detection.
Video-based Person Re-identification with Accumulative Motion Context
Video based person re-identification plays a central role in realistic security and video surveillance.
A Unified Framework for Stochastic Matrix Factorization via Variance Reduction
We propose a unified framework to speed up the existing stochastic matrix factorization (SMF) algorithms via variance reduction.
Diversified Visual Attention Networks for Fine-Grained Object Classification
Fine-grained object classification is a challenging task due to the subtle inter-class difference and large intra-class variation.
IAN: The Individual Aggregation Network for Person Search
In addition, to relieve the negative effect caused by varying visual appearances of the same individual, IAN introduces a novel center loss that can increase the intra-class compactness of feature representations.
Deep Self-Taught Learning for Weakly Supervised Object Localization
To overcome this issue, we propose a deep self-taught learning approach, which makes the detector learn the object-level features reliable for acquiring tight positive samples and afterwards re-train itself based on them.
End-to-End Comparative Attention Networks for Person Re-identification
The CAN model is able to learn which parts of images are relevant for discerning persons and automatically integrates information from different parts to determine whether a pair of images belongs to the same person.
On Fundamental Limits of Robust Learning
We consider the problems of robust PAC learning from distributed and streaming data, which may contain malicious errors and outliers, and analyze their fundamental complexity questions.
Interpretable Structure-Evolving LSTM
Instead of learning LSTM models over the pre-fixed structures, we propose to further learn the intermediate interpretable multi-level graph structures in a progressive and stochastic way from data during the LSTM network optimization.
Tree-Structured Reinforcement Learning for Sequential Object Localization
Therefore, Tree-RL can better cover different objects with various scales which is quite appealing in the context of object proposal.
Efficient Hyperparameter Optimization of Deep Learning Algorithms Using Deterministic RBF Surrogates
Those methods adopt probabilistic surrogate models like Gaussian processes to approximate and minimize the validation error function of hyperparameter values.
Outlier Robust Online Learning
We consider the problem of learning from noisy data in practical settings where the size of data is too large to store on a single machine.
Robust LSTM-Autoencoders for Face De-Occlusion in the Wild
The first one, named multi-scale spatial LSTM encoder, reads facial patches of various scales sequentially to output a latent representation, and occlusion-robustness is achieved owing to the fact that the influence of occlusion is only upon some of the patches.
Video Scene Parsing with Predictive Feature Learning
In this way, the network can effectively learn to capture video dynamics and temporal context, which are critical clues for video scene parsing, without requiring extra manual annotations.
Multi-Path Feedback Recurrent Neural Network for Scene Parsing
In this paper, we consider the scene parsing problem and propose a novel Multi-Path Feedback recurrent neural network (MPF-RNN) for parsing scene images.
Deep Recurrent Regression for Facial Landmark Detection
We propose a novel end-to-end deep architecture for face landmark detection, based on a deep convolutional and deconvolutional network followed by carefully designed recurrent network structures.
Multi-stage Object Detection with Group Recursive Learning
Most of existing detection pipelines treat object proposals independently and predict bounding box locations and classification scores over them separately.
Hyperparameter Transfer Learning through Surrogate Alignment for Efficient Deep Neural Network Training
Recently, several optimization methods have been successfully applied to the hyperparameter optimization of deep neural networks (DNNs).
Scale-aware Pixel-wise Object Proposal Networks
In particular, in order to improve the localization accuracy, a fully convolutional network is employed which predicts locations of object proposals for each pixel.
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks
In this paper, we propose a layer-wise discriminative learning method to enhance the discriminative capability of a deep network by allowing its layers to work collaboratively for classification.
Training Skinny Deep Neural Networks with Iterative Hard Thresholding Methods
In this paper, we propose an iterative hard thresholding (IHT) approach to train Skinny Deep Neural Networks (SDNNs).
Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution
To address this essentially ill-posed problem, we introduce a Deep Edge Guided REcurrent rEsidual~(DEGREE) network to progressively recover the high-frequency details.
Scale-aware Fast R-CNN for Pedestrian Detection
Taking pedestrian detection as an example, we illustrate how we can leverage this philosophy to develop a Scale-Aware Fast R-CNN (SAF R-CNN) framework.
Ranked #23 on
Pedestrian Detection
on Caltech
A Focused Dynamic Attention Model for Visual Question Answering
Solving VQA problems requires techniques from both computer vision for understanding the visual contents of a presented image or video, as well as the ones from natural language processing for understanding semantics of the question and generating the answers.
Attentive Contexts for Object Detection
We provide preliminary answers to these questions through developing a novel Attention to Context Convolution Neural Network (AC-CNN) based object detection model.
Semantic Object Parsing with Graph LSTM
By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data.
Auxiliary Image Regularization for Deep CNNs with Noisy Labels
Precisely-labeled data sets with sufficient amount of samples are very important for training deep convolutional neural networks (CNNs).
Learning with $\ell^{0}$-Graph: $\ell^{0}$-Induced Sparse Subspace Clustering
Sparse subspace clustering methods, such as Sparse Subspace Clustering (SSC) \cite{ElhamifarV13} and $\ell^{1}$-graph \cite{YanW09, ChengYYFH10}, are effective in partitioning the data that lie in a union of subspaces.
Semantic Object Parsing with Local-Global Long Short-Term Memory
The long chains of sequential computation by stacked LG-LSTM layers also enable each pixel to sense a much larger region for inference benefiting from the memorization of previous dependencies in all positions along all dimensions.
Sense Beyond Expressions: Cuteness
With the development of Internet culture, cuteness has become a popular concept.
Modality-dependent Cross-media Retrieval
Specifically, by jointly optimizing the correlation between images and text and the linear regression from one modal space (image or text) to the semantic space, two couples of mappings are learned to project images and text from their original feature spaces into two common latent subspaces (one for I2T and the other for T2I).
Distributed Robust Learning
We propose a framework for distributed robust statistical learning on {\em big contaminated data}.
Correlation Adaptive Subspace Segmentation by Trace Lasso
In this work, we argue that both sparsity and the grouping effect are important for subspace segmentation.
TS2C: Tight Box Mining with Surrounding Segmentation Context for Weakly Supervised Object Detection
This work provides a simple approach to discover tight object bounding boxes with only image-level supervision, called Tight box mining with Surrounding Segmentation Context (TS2C).
Object Relation Detection Based on One-shot Learning
Detecting the relations among objects, such as "cat on sofa" and "person ride horse", is a crucial task in image understanding, and beneficial to bridging the semantic gap between images and natural language.
Multi-Fiber Networks for Video Recognition
In this paper, we aim to reduce the computational cost of spatio-temporal deep neural networks, making them run as fast as their 2D counterparts while preserving state-of-the-art accuracy on video recognition benchmarks.
Ranked #36 on
Action Recognition
on UCF101
(using extra training data)
$A^2$-Nets: Double Attention Networks
Learning to capture long-range relations is fundamental to image/video recognition.
Ranked #35 on
Action Recognition
on UCF101
New Insight into Hybrid Stochastic Gradient Descent: Beyond With-Replacement Sampling and Convexity
In this paper, we affirmatively answer this open question by showing that under WoRS and for both convex and non-convex problems, it is still possible for HSGD (with constant step-size) to match full gradient descent in rate of convergence, while maintaining comparable sample-size-independent incremental first-order oracle complexity to stochastic gradient descent.
Efficient Stochastic Gradient Hard Thresholding
To address these deficiencies, we propose an efficient hybrid stochastic gradient hard thresholding (HSG-HT) method that can be provably shown to have sample-size-independent gradient evaluation and hard thresholding complexity bounds.
Dual-Agent GANs for Photorealistic and Identity Preserving Profile Face Synthesis
In particular, we employ an off-the-shelf 3D face model as a simulator to generate profile face images with varying poses.
Ranked #1 on
Face Verification
on IJB-A
Multimodal Learning and Reasoning for Visual Question Answering
In this work we introduce a modular neural network model that learns a multimodal and multifaceted representation of the image and the question.
Robust Logistic Regression and Classification
We consider logistic regression with arbitrary outliers in the covariate matrix.
Online Robust PCA via Stochastic Optimization
Robust PCA methods are typically based on batch optimization and have to load all the samples into memory.
Online PCA for Contaminated Data
We consider the online Principal Component Analysis (PCA) for contaminated samples (containing outliers) which are revealed sequentially to the Principal Components (PCs) estimator.
MoNet: Deep Motion Exploitation for Video Object Segmentation
In this paper, we propose a novel MoNet model to deeply exploit motion cues for boosting video object segmentation performance from two aspects, i. e., frame representation learning and segmentation refinement.
Deep Adversarial Subspace Clustering
To solve this issue, we propose a novel deep adversarial subspace clustering (DASC) model, which learns more favorable sample representations by deep learning for subspace clustering, and more importantly introduces adversarial learning to supervise sample representation learning and subspace clustering.
Ranked #2 on
Image Clustering
on coil-40
