Search Results for author:
Found 719 papers, 326 papers with code
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models
To address this issue, we propose \emph{Img2Prompt}, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
Domain Generalization via Conditional Invariant Representation
With the conditional invariant representation, the invariance of the joint distribution $\mathbb{P}(h(X), Y)$ can be guaranteed if the class prior $\mathbb{P}(Y)$ does not change across training and test domains.
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation
In this paper, we show the surprisingly good capabilities of plain vision transformers for pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model called ViTPose.
Ranked #1 on
Pose Estimation
on COCO test-dev
ViTPose++: Vision Transformer for Generic Body Pose Estimation
In this paper, we show the surprisingly good properties of plain vision transformers for body pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model dubbed ViTPose.
Ranked #1 on
Animal Pose Estimation
on AP-10K
(using extra training data)
A Comprehensive Survey of Dataset Distillation
Dataset distillation, a dataset reduction method, addresses this problem by synthesizing a small typical dataset from substantial data and has attracted much attention from the deep learning community.
Positive-Unlabeled Compression on the Cloud
In practice, only a small portion of the original training set is required as positive examples and more useful training examples can be obtained from the massive unlabeled data on the cloud through a PU classifier with an attention based multi-scale feature extractor.
Bridging Composite and Real: Towards End-to-end Deep Image Matting
Furthermore, we provide a benchmark containing 2, 000 high-resolution real-world animal images and 10, 000 portrait images along with their manually labeled alpha mattes to serve as a test bed for evaluating matting model's generalization ability on real-world images.
Ranked #2 on
Image Matting
on AM-2K
GMFlow: Learning Optical Flow via Global Matching
Learning-based optical flow estimation has been dominated with the pipeline of cost volume with convolutions for flow regression, which is inherently limited to local correlations and thus is hard to address the long-standing challenge of large displacements.
Ranked #8 on
Optical Flow Estimation
on Spring
Unifying Flow, Stereo and Depth Estimation
We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images.
Ranked #1 on
Optical Flow Estimation
on Sintel-clean
VanillaNet: the Power of Minimalism in Deep Learning
In this study, we introduce VanillaNet, a neural network architecture that embraces elegance in design.
Towards Open Vocabulary Learning: A Survey
To our knowledge, this is the first comprehensive literature review of open vocabulary learning.
SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model
In this study, we leverage SAM and existing RS object detection datasets to develop an efficient pipeline for generating a large-scale RS segmentation dataset, dubbed SAMRS.
HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpainting
Given a generated failed image due to malformed hands, we utilize ControlNet modules to re-inject such correct hand information.
ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond
Vision transformers have shown great potential in various computer vision tasks owing to their strong capability to model long-range dependency using the self-attention mechanism.
Ranked #2 on
Image Classification
on ImageNet ReaL
Deep Ordinal Regression Network for Monocular Depth Estimation
These methods model depth estimation as a regression problem and train the regression networks by minimizing mean squared error, which suffers from slow convergence and unsatisfactory local solutions.
Ranked #13 on
Depth Estimation
on NYU-Depth V2
Deep Modular Co-Attention Networks for Visual Question Answering
In this paper, we propose a deep Modular Co-Attention Network (MCAN) that consists of Modular Co-Attention (MCA) layers cascaded in depth.
Ranked #5 on
Question Answering
on SQA3D
An Empirical Study of Remote Sensing Pretraining
To this end, we train different networks from scratch with the help of the largest RS scene recognition dataset up to now -- MillionAID, to obtain a series of RS pretrained backbones, including both convolutional neural networks (CNN) and vision transformers such as Swin and ViTAE, which have shown promising performance on computer vision tasks.
Ranked #1 on
Aerial Scene Classification
on UCM (80% as trainset)
Aerial Scene Classification
Building change detection for remote sensing images
+5
Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model
Large-scale vision foundation models have made significant progress in visual tasks on natural images, with vision transformers being the primary choice due to their good scalability and representation ability.
Ranked #1 on
Aerial Scene Classification
on AID (50% as trainset)
Deep Learning for Camera Calibration and Beyond: A Survey
In this paper, we provide a comprehensive survey of learning-based camera calibration techniques, by analyzing their strengths and limitations.
Deep Automatic Natural Image Matting
To address the problem, a novel end-to-end matting network is proposed, which can predict a generalized trimap for any image of the above types as a unified semantic representation.
Ranked #2 on
Image Matting
on AIM-500
SFNet: Faster, Accurate, and Domain Agnostic Semantic Segmentation via Semantic Flow
In this paper, we focus on exploring effective methods for faster, accurate, and domain agnostic semantic segmentation.
Recurrent Feature Reasoning for Image Inpainting
To capture information from distant places in the feature map for RFR, we further develop KCA and incorporate it in RFR.
HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens
To achieve an extremely fast NAS while preserving the high accuracy, we propose to identify the vital blocks and make them the priority in the architecture search.
SCOP: Scientific Control for Reliable Neural Network Pruning
To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output.
Manifold Regularized Dynamic Network Pruning
Then, the manifold relationship between instances and the pruned sub-networks will be aligned in the training procedure.
Privacy-Preserving Portrait Matting
We systematically evaluate both trimap-free and trimap-based matting methods on P3M-10k and find that existing matting methods show different generalization capabilities when following the Privacy-Preserving Training (PPT) setting, i. e., training on face-blurred images and testing on arbitrary images.
Ranked #3 on
Image Matting
on P3M-10k
Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction
For the application of dense image prediction, the validity of CEN is tested by four different scenarios: multimodal fusion, cycle multimodal fusion, multitask learning, and multimodal multitask learning.
Ranked #7 on
Semantic Segmentation
on LLRGBD-synthetic
ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning
In particular, we propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously, which is called ST-P3.
Ranked #7 on
Bird's-Eye View Semantic Segmentation
on nuScenes
(IoU ped - 224x480 - Vis filter. - 100x100 at 0.5 metric)
Learning to Generalize Provably in Learning to Optimize
While the optimizer generalization has been recently studied, the optimizee generalization (or learning to generalize) has not been rigorously studied in the L2O context, which is the aim of this paper.
Perceptual Adversarial Networks for Image-to-Image Transformation
The proposed PAN consists of two feed-forward convolutional neural networks (CNNs), the image transformation network T and the discriminative network D. Through combining the generative adversarial loss and the proposed perceptual adversarial loss, these two networks can be trained alternately to solve image-to-image transformation tasks.
ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias
Nevertheless, vision transformers treat an image as 1D sequence of visual tokens, lacking an intrinsic inductive bias (IB) in modeling local visual structures and dealing with scale variance.
Ranked #2 on
Video Object Segmentation
on DAVIS 2017
DehazeNet: An End-to-End System for Single Image Haze Removal
The key to achieve haze removal is to estimate a medium transmission map for an input hazy image.
Ranked #7 on
Image Dehazing
on RS-Haze
BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning
We perform extensive experiments on over ten datasets and the proposed method achieves significant improvements on different data scarcity applications without any bells and whistles, including the tasks of long-tailed recognition, compositional zero-shot learning, domain generalization, and contrastive learning.
Ranked #17 on
Long-tail Learning
on iNaturalist 2018
BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning
Specifically, when applying the proposed module, it employs a two-stream pipeline during training, i. e., either with or without a BatchFormerV2 module, where the batchformer stream can be removed for testing.
Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping
Consequently, we introduce Trajectory Consistency Distillation (TCD), which encompasses trajectory consistency function and strategic stochastic sampling.
Generating Holistic 3D Human Motion from Speech
This work addresses the problem of generating 3D holistic body motions from human speech.
Ranked #2 on
Gesture Generation
on BEAT2
A Survey on Knowledge Distillation of Large Language Models
In the era of Large Language Models (LLMs), Knowledge Distillation (KD) emerges as a pivotal methodology for transferring advanced capabilities from leading proprietary LLMs, such as GPT-4, to their open-source counterparts like LLaMA and Mistral.
DUT: Learning Video Stabilization by Simply Watching Unstable Videos
In this paper, we attempt to tackle the video stabilization problem in a deep unsupervised learning manner, which borrows the divide-and-conquer idea from traditional stabilizers while leveraging the representation power of DNNs to handle the challenges in real-world scenarios.
DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformer
However, these methods built upon detection transformer framework might achieve sub-optimal training efficiency and performance due to coarse positional query modeling. In addition, the point label form exploited in previous works implies the reading order of humans, which impedes the detection robustness from our observation.
Ranked #3 on
Scene Text Detection
on SCUT-CTW1500
DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting
In this paper, we present DeepSolo, a simple DETR-like baseline that lets a single Decoder with Explicit Points Solo for text detection and recognition simultaneously.
Ranked #1 on
Text Spotting
on Total-Text
(using extra training data)
DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Multilingual Text Spotting
In this paper, we present DeepSolo++, a simple DETR-like baseline that lets a single decoder with explicit points solo for text detection, recognition, and script identification simultaneously.
Ranked #1 on
Text Spotting
on Inverse-Text
Referring Image Matting
Different from conventional image matting, which either requires user-defined scribbles/trimap to extract a specific foreground object or directly extracts all the foreground objects in the image indiscriminately, we introduce a new task named Referring Image Matting (RIM) in this paper, which aims to extract the meticulous alpha matte of the specific object that best matches the given natural language description, thus enabling a more natural and simpler instruction for image matting.
Ranked #1 on
Referring Image Matting (RefMatte-RW100)
on RefMatte
TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers
Detection Transformer (DETR) and Deformable DETR have been proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance as previous complex hand-crafted detectors.
Ranked #4 on
Video Object Detection
on ImageNet VID
(using extra training data)
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT
Recently, ChatGPT has attracted great attention, as it can generate fluent and high-quality responses to human inquiries.
Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics
Unsupervised monocular depth and ego-motion estimation has drawn extensive research attention in recent years.
Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering
For fine-grained image and question representations, a `co-attention' mechanism is developed by using a deep neural network architecture to jointly learn the attentions for both the image and the question, which can allow us to reduce the irrelevant features effectively and obtain more discriminative features for image and question representations.
Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
For multi-modal feature fusion, here we develop a Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and effectively combine multi-modal features, which results in superior performance for VQA compared with other bilinear pooling approaches.
Tensor Canonical Correlation Analysis for Multi-view Dimension Reduction
As a consequence, the high order correlation information contained in the different views is explored and thus a more reliable common subspace shared by all features can be obtained.
Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval
In addition, we harness a self-supervised semantic network to discover high-level semantic information in the form of multi-label annotations.
Stroke Controllable Fast Style Transfer with Adaptive Receptive Fields
In this paper, we present a stroke controllable style transfer network that can achieve continuous and spatial stroke size control.
Progressive LiDAR Adaptation for Road Detection
To this end, LiDAR sensor data can be incorporated to improve the visual image-based road detection, because LiDAR data is less susceptible to visual noises.
Deep Image Matting: A Comprehensive Survey
Image matting refers to extracting precise alpha matte from natural images, and it plays a critical role in various downstream applications, such as image editing.
TriDet: Temporal Action Detection with Relative Boundary Modeling
In this paper, we present a one-stage framework TriDet for temporal action detection.
Ranked #2 on
Temporal Action Localization
on EPIC-KITCHENS-100
Temporal Action Localization with Enhanced Instant Discriminability
Temporal action detection (TAD) aims to detect all action boundaries and their corresponding categories in an untrimmed video.
Ranked #1 on
Temporal Action Localization
on MultiTHUMOS
VSA: Learning Varied-Size Window Attention in Vision Transformers
Attention within windows has been widely explored in vision transformers to balance the performance, computation complexity, and memory footprint.
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
Thus, we propose RobustART, the first comprehensive Robustness investigation benchmark on ImageNet regarding ARchitecture design (49 human-designed off-the-shelf architectures and 1200+ networks from neural architecture search) and Training techniques (10+ techniques, e. g., data augmentation) towards diverse noises (adversarial, natural, and system noises).
Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation
Although there has been a progress in matching the marginal distributions between two domains, the classifier favors the source domain features and makes incorrect predictions on the target domain due to category-agnostic feature alignment.
Ranked #24 on
Image-to-Image Translation
on SYNTHIA-to-Cityscapes
Contrastive Boundary Learning for Point Cloud Segmentation
Point cloud segmentation is fundamental in understanding 3D environments.
Ranked #17 on
Semantic Segmentation
on S3DIS
Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation
Supervised depth estimation has achieved high accuracy due to the advanced deep network architectures.
Ranked #68 on
Monocular Depth Estimation
on KITTI Eigen split
One for All: Towards Training One Graph Model for All Classification Tasks
For in-context learning on graphs, OFA introduces a novel graph prompting paradigm that appends prompting substructures to the input graph, which enables it to address varied tasks without fine-tuning.
SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data
As the main discriminative information of a fine-grained image usually resides in subtle regions, methods along this line are prone to heavy label noise in fine-grained recognition.
Ranked #31 on
Fine-Grained Image Classification
on CUB-200-2011
AP-10K: A Benchmark for Animal Pose Estimation in the Wild
The experimental results provide sound empirical evidence on the superiority of learning from diverse animals species in terms of both accuracy and generalization ability.
APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking
Based on APT-36K, we benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking.
Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentation
In terms of the AMG mode, Hi-SAM segments text stroke foreground masks initially, then samples foreground points for hierarchical text mask generation and achieves layout analysis in passing.
Ranked #1 on
Hierarchical Text Segmentation
on HierText
Vision Transformer with Quadrangle Attention
Window-based attention has become a popular choice in vision transformers due to its superior performance, lower computational complexity, and less memory footprint.
Pruning Self-attentions into Convolutional Layers in Single Path
Relying on the single-path space, we introduce learnable binary gates to encode the operation choices in MSA layers.
Ranked #18 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
Graph Pooling for Graph Neural Networks: Progress, Challenges, and Opportunities
Graph neural networks have emerged as a leading architecture for many graph-level tasks, such as graph classification and graph generation.
Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs
Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data.
Ranked #1 on
Face Sketch Synthesis
on CUFS
(FID metric)
TGRNet: A Table Graph Reconstruction Network for Table Structure Recognition
A table arranging data in rows and columns is a very effective data structure, which has been widely used in business and scientific research.
Exploring Sequence Feature Alignment for Domain Adaptive Detection Transformers
In DQFA, a novel domain query is used to aggregate and align global context from the token sequence of both domains.
A Comprehensive Survey and Taxonomy on Single Image Dehazing Based on Deep Learning
With the development of convolutional neural networks, hundreds of deep learning based dehazing methods have been proposed.
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining
However, transferring the pretrained models to downstream tasks may encounter task discrepancy due to their formulation of pretraining as image classification or object discrimination tasks.
Ranked #1 on
Semantic Segmentation
on SpaceNet 1
(using extra training data)
Aerial Scene Classification
Building change detection for remote sensing images
+12
SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection
We observe that the prevailing set abstraction design for down-sampling points may maintain too much unimportant background information that can affect feature learning for detecting objects.
Event-based Simultaneous Localization and Mapping: A Comprehensive Survey
This paper presents a timely and comprehensive review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks.
Fashionformer: A simple, Effective and Unified Baseline for Human Fashion Segmentation and Recognition
This focus on joint human fashion segmentation and attribute recognition.
Diff-Font: Diffusion Model for Robust One-Shot Font Generation
Specifically, a large stroke-wise dataset is constructed, and a stroke-wise diffusion model is proposed to preserve the structure and the completion of each generated character.
Deepfake Generation and Detection: A Benchmark and Survey
Deepfake is a technology dedicated to creating highly realistic facial images and videos under specific conditions, which has significant application potential in fields such as entertainment, movie production, digital human creation, to name a few.
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
To this end, we augment BARTScore by incorporating the human-like error analysis strategies, namely BARTScore++, where the final score consists of both the evaluations of major errors and minor errors.
Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models
To further improve the performance of LLMs on MT quality assessment, we investigate several prompting designs, and propose a new prompting method called \textbf{\texttt{Error Analysis Prompting}} (EAPrompt) by combining Chain-of-Thoughts (Wei et al., 2022) and Error Analysis (Lu et al., 2023).
Self-Augmented Unpaired Image Dehazing via Density and Depth Decomposition
With estimated scene depth, our method is capable of re-rendering hazy images with different thicknesses which further benefits the training of the dehazing network.
Heterogeneous Federated Learning: State-of-the-art and Research Challenges
Therefore, a systematic survey on this topic about the research challenges and state-of-the-art is essential.
Visual Compositional Learning for Human-Object Interaction Detection
The integration of decomposition and composition enables VCL to share object and verb features among different HOI samples and images, and to generate new interaction samples and new types of HOI, and thus largely alleviates the long-tail distribution problem and benefits low-shot or zero-shot HOI detection.
Ranked #3 on
Affordance Recognition
on HICO-DET(Unknown Concepts)
Detecting Human-Object Interaction via Fabricated Compositional Learning
With the proposed object fabricator, we are able to generate large-scale HOI samples for rare and unseen categories to alleviate the open long-tailed issues in HOI detection.
Ranked #4 on
Affordance Recognition
on HICO-DET
Affordance Transfer Learning for Human-Object Interaction Detection
The proposed method can thus be used to 1) improve the performance of HOI detection, especially for the HOIs with unseen objects; and 2) infer the affordances of novel objects.
Ranked #2 on
Affordance Recognition
on HICO-DET(Unknown Concepts)
Discovering Human-Object Interaction Concepts via Self-Compositional Learning
Therefore, the proposed method enables the learning on both known and unknown HOI concepts.
Affordance Recognition
Human-Object Interaction Concept Discovery
+1
Towards Data-Efficient Detection Transformers
Besides, we introduce a simple yet effective label augmentation method to provide richer supervision and improve data efficiency.
Rethinking Portrait Matting with Privacy Preserving
P3M-10k consists of 10, 421 high resolution face-blurred portrait images along with high-quality alpha mattes, which enables us to systematically evaluate both trimap-free and trimap-based matting methods and obtain some useful findings about model generalization ability under the privacy preserving training (PPT) setting.
Ranked #1 on
Image Matting
on P3M-10k
JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenes
A naive way is to accomplish them independently in a sequential or parallel manner, but there are many drawbacks, i. e., 1) the depth and VO results suffer from the inherent scale ambiguity issue; 2) the BEV layout is directly predicted from the front-view image without using any depth-related information, although the depth map contains useful geometry clues for inferring scene layouts.
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Ranked #3 on
Few-Shot Class-Incremental Learning
on CUB-200-2011
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Towards Making the Most of ChatGPT for Machine Translation
We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information can further improve ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community.
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants
Beyond the normal case, long-tail class incremental learning and few-shot class incremental learning are also proposed to consider the data imbalance and data scarcity, respectively, which are common in real-world implementations and further exacerbate the well-known problem of catastrophic forgetting.
Evolutionary Generative Adversarial Networks
In this paper, we propose a novel GAN framework called evolutionary generative adversarial networks (E-GAN) for stable GAN training and improved generative performance.
Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mapping
Unsupervised domain mapping aims to learn a function to translate domain X to Y by a function GXY in the absence of paired examples.
BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering
Graph convolutional networks are used to obtain a relation-aware representation of nodes for entity graphs built from documents with multi-level features.
Progressive One-shot Human Parsing
In this paper, we devise a novel Progressive One-shot Parsing network (POPNet) to address two critical challenges , i. e., testing bias and small sizes.
End-to-end One-shot Human Parsing
To address three main challenges in OSHP, i. e., small sizes, testing bias, and similar parts, we devise an End-to-end One-shot human Parsing Network (EOP-Net).
I3CL:Intra- and Inter-Instance Collaborative Learning for Arbitrary-shaped Scene Text Detection
Existing methods for arbitrary-shaped text detection in natural scenes face two critical issues, i. e., 1) fracture detections at the gaps in a text instance; and 2) inaccurate detections of arbitrary-shaped text instances with diverse background context.
Ranked #5 on
Scene Text Detection
on SCUT-CTW1500
Domain-Specific Risk Minimization for Out-of-Distribution Generalization
Our bound motivates two strategies to reduce the gap: the first one is ensembling multiple classifiers to enrich the hypothesis space, then we propose effective gap estimation methods for guiding the selection of a better hypothesis for the target.
Adaptive Context-Aware Multi-Modal Network for Depth Completion
Furthermore, considering the mutli-modality of input data, we exploit the graph propagation on the two modalities respectively to extract multi-modal representations.
A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning
Visual reinforcement learning (RL), which makes decisions directly from high-dimensional visual inputs, has demonstrated significant potential in various domains.
Grapy-ML: Graph Pyramid Mutual Learning for Cross-dataset Human Parsing
In this paper, we propose a novel GRAph PYramid Mutual Learning (Grapy-ML) method to address the cross-dataset human parsing problem, where the annotations are at different granularities.
DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training
In this work, we propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL, which employs personalized sparse masks to customize sparse local models on the edge.
LIIR: Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning
A great challenge in cooperative decentralized multi-agent reinforcement learning (MARL) is generating diversified behaviors for each individual agent when receiving only a team reward.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
GPS-Net: Graph Property Sensing Network for Scene Graph Generation
There are three key properties of scene graph that have been underexplored in recent works: namely, the edge direction information, the difference in priority between nodes, and the long-tailed distribution of relationships.
Ranked #5 on
Scene Graph Generation
on Visual Genome
Part-dependent Label Noise: Towards Instance-dependent Label Noise
Learning with the \textit{instance-dependent} label noise is challenging, because it is hard to model such real-world noise.
Domain Generalization via Entropy Regularization
To arrive at this, some methods introduce a domain discriminator through adversarial learning to match the feature distributions in multiple source domains.
Ranked #43 on
Domain Generalization
on PACS
Online Multiple Object Tracking with Cross-Task Synergy
On the other hand, such reliable embeddings can boost identity-awareness through memory aggregation, hence strengthen attention modules and suppress drifts.
One-Shot Affordance Detection
To empower robots with this ability in unseen scenarios, we consider the challenging one-shot affordance detection problem in this paper, i. e., given a support image that depicts the action purpose, all objects in a scene with the common affordance should be detected.
One-Shot Object Affordance Detection in the Wild
To empower robots with this ability in unseen scenarios, we first study the challenging one-shot affordance detection problem in this paper, i. e., given a support image that depicts the action purpose, all objects in a scene with the common affordance should be detected.
Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification
Third, we introduce a Compound Ranking (CR) loss that makes use of textual descriptions for other images of the same identity to provide extra supervision, thereby effectively reducing the intra-class variance in textual features.
Ranked #1 on
Image Retrieval
on ICFG-PEDES
PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation
The Depth-aware Video Panoptic Segmentation (DVPS) is a new challenging vision problem that aims to predict panoptic segmentation and depth in a video simultaneously.
Image Captions are Natural Prompts for Text-to-Image Models
With the rapid development of Artificial Intelligence Generated Content (AIGC), it has become common practice in many learning tasks to train or fine-tune large models on synthetic data due to the data-scarcity and privacy leakage problems.
Dynamic Focus-aware Positional Queries for Semantic Segmentation
In this paper, we propose a simple yet effective query design for semantic segmentation termed Dynamic Focus-aware Positional Queries (DFPQ), which dynamically generates positional queries conditioned on the cross-attention scores from the preceding decoder block and the positional encodings for the corresponding image features, simultaneously.
Ranked #21 on
Semantic Segmentation
on ADE20K
Distilling Knowledge from Graph Convolutional Networks
To enable the knowledge transfer from the teacher GCN to the student, we propose a local structure preserving module that explicitly accounts for the topological semantics of the teacher.
SynFace: Face Recognition with Synthetic Data
We then analyze the underlying causes behind the performance gap, e. g., the poor intra-class variations and the domain gap between synthetic and real face images.
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
It is both crucial and natural to extend this research direction to the video domain for video question answering (VideoQA).
Ranked #29 on
Video Question Answering
on ActivityNet-QA
Visual Question Answering (VQA)
Zero-Shot Video Question Answer
Region-wise Generative Adversarial ImageInpainting for Large Missing Areas
To address these problems, we propose a generic inpainting framework capable of handling with incomplete images on both continuous and discontinuous large missing areas, in an adversarial manner.
Nighttime Dehazing with a Synthetic Benchmark
To address this issue, we propose a novel synthetic method called 3R to simulate nighttime hazy images from daytime clear images, which first reconstructs the scene geometry, then simulates the light rays and object reflectance, and finally renders the haze effects.
Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation
To the best of our knowledge, we are the first to solve the PPS problem via \textit{a unified and end-to-end transformer model.
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.
Sensitivity-Aware Visual Parameter-Efficient Fine-Tuning
Visual Parameter-Efficient Fine-Tuning (PEFT) has become a powerful alternative for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only tunes a small number of parameters while freezing the vast majority ones to ease storage burden and optimization difficulty.
Exploring Plain ViT Reconstruction for Multi-class Unsupervised Anomaly Detection
Following this spirit, this paper explores plain ViT architecture for MUAD.
Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark
Moreover, current metrics such as AU-ROC have nearly reached saturation on simple datasets, which prevents a comprehensive evaluation of different methods.
Variational Distillation for Multi-View Learning
Information Bottleneck (IB) based multi-view learning provides an information theoretic principle for seeking shared information contained in heterogeneous data descriptions.
Heatmap Regression via Randomized Rounding
Previous methods to overcome the sub-pixel localization problem usually rely on high-resolution heatmaps.
Improved Techniques for Learning to Dehaze and Beyond: A Collective Study
Here we explore two related but important tasks based on the recently released REalistic Single Image DEhazing (RESIDE) benchmark dataset: (i) single image dehazing as a low-level image restoration problem; and (ii) high-level visual understanding (e. g., object detection) of hazy images.
Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss
In this paper, a novel 3D convolutional network, called two-stage-age-network (TSAN), is proposed to estimate brain age from T1-weighted MRI data.
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends
Deep supervised learning algorithms typically require a large volume of labeled data to achieve satisfactory performance.
Learning with Biased Complementary Labels
We therefore reason that the transition probabilities will be different.
Where and What? Examining Interpretable Disentangled Representations
We thus impose a perturbation on a certain dimension of the latent code, and expect to identify the perturbation along this dimension from the generated images so that the encoding of simple variations can be enforced.
WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking
In this work, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers.
Unified Discrete Diffusion for Simultaneous Vision-Language Generation
The recently developed discrete diffusion models perform extraordinarily well in the text-to-image task, showing significant promise for handling the multi-modality signals.
SPAGAN: Shortest Path Graph Attention Network
SPAGAN therefore allows for a more informative and intact exploration of the graph structure and further {a} more effective aggregation of information from distant neighbors into the center node, as compared to node-based GCN methods.
Learn From Model Beyond Fine-Tuning: A Survey
LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks.
Semantic Structure-based Unsupervised Deep Hashing
Hashing is becoming increasingly popular for approximate nearest neighbor searching in massive databases due to its storage and search efficiency.
Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrieval
In contrast to previous works, we focus on the reproducibility of the approaches and the examination of the elements that lead to improved performance by pretrained and nonpretrained models in retrieving images and text.
Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis
To this end, we propose a knowledge graph augmented network KGAN, which aims to effectively incorporate external knowledge with explicitly syntactic and contextual information.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Robust Unlearnable Examples: Protecting Data Against Adversarial Learning
To address this concern, methods are proposed to make data unlearnable for deep learning models by adding a type of error-minimizing noise.
Vega-MT: The JD Explore Academy Translation System for WMT22
As for model sizes, we scale the Transformer-Big up to the extremely large model that owns nearly 4. 7 Billion parameters, to fully enhance the model capacity for our Vega-MT.
Ranked #1 on
Machine Translation
on WMT 2022 English-Russian
DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification
Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions.
Recurrent Glimpse-based Decoder for Detection with Transformer
Then, a glimpse-based decoder is introduced to provide refined detection results based on both the glimpse features and the attention modeling outputs of the previous stage.
Ranked #1 on
Object Detection
on MS COCO
(GFlops metric)
Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?
Modern deep neural networks for classification usually jointly learn a backbone for representation and a linear classifier to output the logit of each class.
Ranked #26 on
Long-tail Learning
on CIFAR-10-LT (ρ=100)
End2End Occluded Face Recognition by Masking Corrupted Features
However, the state-of-the-art general face recognition models do not generalize well to occluded face images, which are exactly the common cases in real-world scenarios.
Learning Affordance Grounding from Exocentric Images
To empower an agent with such ability, this paper proposes a task of affordance grounding from exocentric view, i. e., given exocentric human-object interaction and egocentric object images, learning the affordance knowledge of the object and transferring it to the egocentric image using only the affordance label as supervision.
Grounded Affordance from Exocentric View
Due to the diversity of interactive affordance, the uniqueness of different individuals leads to diverse interactions, which makes it difficult to establish an explicit link between object parts and affordance labels.
GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation
However, recent projection-based methods for point cloud semantic segmentation usually utilize a vanilla late fusion strategy for the predictions of different views, failing to explore the complementary information from a geometric perspective during the representation learning.
Ranked #1 on
Robust 3D Semantic Segmentation
on nuScenes-C
ReAct: Temporal Action Detection with Relational Queries
Moreover, we propose two losses to facilitate and stabilize the training of action classification.
Ranked #15 on
Temporal Action Localization
on THUMOS’14
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach
One of the popular solutions is Sharpness-Aware Minimization (SAM), which smooths the loss landscape via minimizing the maximized change of training loss when adding a perturbation to the weight.
Segmentation-Aware Image Denoising without Knowing True Segmentation
Several recent works discussed application-driven image restoration neural networks, which are capable of not only removing noise in images but also preserving their semantic-aware details, making them suitable for various high-level computer vision tasks as the pre-processing step.
FAMED-Net: A Fast and Accurate Multi-scale End-to-end Dehazing Network
Single image dehazing is a critical image pre-processing step for subsequent high-level computer vision tasks.
Learn, Imagine and Create: Text-to-Image Generation from Prior Knowledge
Given a text description, we immediately imagine an overall visual impression using this prior and, based on this, we draw a picture by progressively adding more and more details.
Artificial Neural Variability for Deep Learning: On Overfitting, Noise Memorization, and Catastrophic Forgetting
Thus it motivates us to design a similar mechanism named {\it artificial neural variability} (ANV), which helps artificial neural networks learn some advantages from ``natural'' neural networks.
An Underwater Image Enhancement Benchmark Dataset and Beyond
In this paper, we construct an Underwater Image Enhancement Benchmark (UIEB) including 950 real-world underwater images, 890 of which have the corresponding reference images.
Ranked #5 on
Underwater Image Restoration
on LSUI
(using extra training data)
RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning?
In this paper, we make the first attempt to demonstrate the importance of both regions in cropping from a complete perspective and propose a simple yet effective pretext task called Region Contrastive Learning (RegionCL).
Video Frame Interpolation without Temporal Priors
However, in complicated real-world situations, the temporal priors of videos, i. e. frames per second (FPS) and frame exposure time, may vary from different camera sensors.
One-pass Multi-task Networks with Cross-task Guided Attention for Brain Tumor Segmentation
The model cascade (MC) strategy significantly alleviates the class imbalance issue via running a set of individual deep models for coarse-to-fine segmentation.
Ranked #1 on
Brain Tumor Segmentation
on BRATS-2015
Exposure Trajectory Recovery from Motion Blur
However, the motion information contained in a blurry image has yet to be fully explored and accurately formulated because: (i) the ground truth of dynamic motion is difficult to obtain; (ii) the temporal ordering is destroyed during the exposure; and (iii) the motion estimation from a blurry image is highly ill-posed.
Topology-aware Generalization of Decentralized SGD
Our theory indicates that the generalizability of D-SGD is positively correlated with the spectral gap, and can explain why consensus control in initial training phase can ensure better generalization.
GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstruction
Demystifying complex human-ground interactions is essential for accurate and realistic 3D human motion reconstruction from RGB videos, as it ensures consistency between the humans and the ground plane.
Pretrained Language Models for Dialogue Generation with Multiple Input Sources
In this work, we study dialogue models with multiple input sources adapted from the pretrained language model GPT2.
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm
Motivated by biological evolution, this paper explains the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) and derives that both have consistent mathematical formulation.
Make Landscape Flatter in Differentially Private Federated Learning
Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with better stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance.
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy
To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise.
Searching for Low-Bit Weights in Quantized Neural Networks
Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators.
Glance and Gaze: Inferring Action-aware Points for One-Stage Human-Object Interaction Detection
In this paper, we propose a novel one-stage method, namely Glance and Gaze Network (GGNet), which adaptively models a set of actionaware points (ActPoints) via glance and gaze steps.
Ranked #17 on
Human-Object Interaction Detection
on V-COCO
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
Instead, we propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG), which relieves the issue of direct model aggregation.
CLAMP: Prompt-based Contrastive Learning for Connecting Language and Animal Pose
Motivated by the progress of visual-language research, we propose that pre-trained language models (e. g., CLIP) can facilitate animal pose estimation by providing rich prior knowledge for describing animal keypoints in text.
Upcycling Models under Domain and Category Shift
We examine the superiority of our GLC on multiple benchmarks with different category shift scenarios, including partial-set, open-set, and open-partial-set DA.
Ranked #2 on
Universal Domain Adaptation
on VisDA2017
GLC++: Source-Free Universal Domain Adaptation through Global-Local Clustering and Contrastive Affinity Learning
GLC++ enhances the novel category clustering accuracy of GLC by 4. 3% in open-set scenarios on Office-Home.
MirrorGAN: Learning Text-to-image Generation by Redescription
Generating an image from a given text description has two goals: visual realism and semantic consistency.
Ranked #8 on
Text-to-Image Generation
on CUB
(Inception score metric)
Quantum circuit architecture search for variational quantum algorithms
Variational quantum algorithms (VQAs) are expected to be a path to quantum advantages on noisy intermediate-scale quantum devices.
Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing
Essentially, DGCPN addresses the inaccurate similarity problem by exploring and exploiting the data's intrinsic relationships in a graph.
Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization
However, since the view outside the boundary is not available during warping, the resulting holes around the boundary of the stabilized frame must be discarded (i. e., cropping) to maintain visual consistency, and thus does leads to a tradeoff between stability and cropping ratio.
BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation
Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data, which is a much more practical setting due to the data privacy, security, and transmission issues.
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy
Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections.
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
In federated learning (FL), a cluster of local clients are chaired under the coordination of the global server and cooperatively train one model with privacy protection.
Towards High Performance Human Keypoint Detection
Human keypoint detection from a single image is very challenging due to occlusion, blur, illumination and scale variance.
Ranked #5 on
Pose Estimation
on COCO test-dev
Deep Multimodal Neural Architecture Search
Most existing works focus on a single task and design neural architectures manually, which are highly task-specific and hard to generalize to different tasks.
Ranked #19 on
Visual Question Answering (VQA)
on VQA v2 test-std
Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer
We propose adversarial gated networks (Gated GAN) to transfer multiple styles in a single model.
Improving Video Instance Segmentation via Temporal Pyramid Routing
To incorporate both temporal and scale information, we propose a Temporal Pyramid Routing (TPR) strategy to conditionally align and conduct pixel-level aggregation from a feature pyramid pair of two adjacent frames.
FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis
However, designing a unified BA method that can be applied to various MIA systems is challenging due to the diversity of imaging modalities (e. g., X-Ray, CT, and MRI) and analysis tasks (e. g., classification, detection, and segmentation).
CktGNN: Circuit Graph Neural Network for Electronic Design Automation
The electronic design automation of analog circuits has been a longstanding challenge in the integrated circuit field due to the huge design space and complex design trade-offs among circuit specifications.
Continual Learning on Graphs: Challenges, Solutions, and Opportunities
To bridge the gap, we provide a comprehensive review of existing continual graph learning (CGL) algorithms by elucidating the different task settings and categorizing the existing methods based on their characteristics.
Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding
Visual grounding aims to localize an object in an image referred to by a textual query phrase.
Ranked #9 on
Phrase Grounding
on Flickr30k Entities Test
On Positive-Unlabeled Classification in GAN
In contrast, it is more reasonable to treat the generated data as unlabeled, which could be positive or negative according to their quality.
Commutative Lie Group VAE for Disentanglement Learning
Instead, we propose to encode the data variations with groups, a structure not only can equivariantly represent variations, but can also be adaptively optimized to preserve the properties of data variations.
DSP: Dual Soft-Paste for Unsupervised Domain Adaptive Semantic Segmentation
In addition, feature-level alignment is carried out by aligning the feature maps of the source and target images from student network using a weighted maximum mean discrepancy loss.
Ranked #18 on
Synthetic-to-Real Translation
on SYNTHIA-to-Cityscapes
Merging Experts into One: Improving Computational Efficiency of Mixture of Experts
Although a sparse Mixture of Experts (MoE) can reduce the cost by activating a small subset of parameters (e. g., one expert) for each input, its computation escalates significantly if increasing the number of activated experts, limiting its practical utility.
Centered Weight Normalization in Accelerating Training of Deep Neural Networks
Training deep neural networks is difficult for the pathological curvature problem.
Quantum Geometric Machine Learning for Quantum Circuits and Control
Our results demonstrate how geometric control techniques can be used to both (a) verify the extent to which geometrically synthesised quantum circuits lie along geodesic, and thus time-optimal, routes and (b) synthesise those circuits.
Symbiotic Adversarial Learning for Attribute-based Person Search
The current state-of-the-art methods either focus on learning better cross-modal embeddings by mining only seen data, or they explicitly use generative adversarial networks (GANs) to synthesize unseen features.
Auto Learning Attention
Attention modules have been demonstrated effective in strengthening the representation ability of a neural network via reweighting spatial or channel features or stacking both operations sequentially.
A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis
Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task, which focuses on detecting the sentiment polarity towards the aspect in a sentence.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+4
Benchmarking Single Image Dehazing and Beyond
We present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE).
Boundary-assisted Region Proposal Networks for Nucleus Segmentation
Accordingly, in this paper, we devise a Boundary-assisted Region Proposal Network (BRP-Net) that achieves robust instance-level nucleus segmentation.
Polysemy Deciphering Network for Robust Human-Object Interaction Detection
To address this issue, in this paper, we propose a novel Polysemy Deciphering Network (PD-Net) that decodes the visual polysemy of verbs for HOI detection in three distinct ways.
Ranked #19 on
Human-Object Interaction Detection
on V-COCO
Polysemy Deciphering Network for Human-Object Interaction Detection
First, PD-Net augments human pose and spatial features for HOI detection using language priors, enabling the verb classifiers to receive language hints that reduce the intra-class variation of the same verb.
A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation
To alleviate the above data issues, we propose a data manipulation method, which is model-agnostic to be packed with any persona-based dialogue generation model to improve its performance.
Toward Real-world Single Image Deraining: A New Benchmark and Beyond
To address these issues, we establish a new high-quality dataset named RealRain-1k, consisting of $1, 120$ high-resolution paired clean and rainy images with low- and high-density rain streaks, respectively.
