Search Results for author:
Found 577 papers, 254 papers with code
CvT: Introducing Convolutions to Vision Transformers
We present in this paper a new architecture, named Convolutional vision Transformer (CvT), that improves Vision Transformer (ViT) in performance and efficiency by introducing convolutions into ViT to yield the best of both designs.
Ranked #3 on
Image Classification
on Flowers-102
(using extra training data)
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
To effectively fuse language and vision modalities, we conceptually divide a closed-set detector into three phases and propose a tight fusion solution, which includes a feature enhancer, a language-guided query selection, and a cross-modality decoder for cross-modality fusion.
Ranked #1 on
Zero-Shot Object Detection
on MSCOCO
Dynamic Head: Unifying Object Detection Heads with Attentions
In this paper, we present a novel dynamic head framework to unify object detection heads with attentions.
Ranked #3 on
Object Detection
on COCO 2017 val
(AP75 metric)
MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
In this paper, we design a benchmark task and provide the associated datasets for recognizing face images and link them to corresponding entity keys in a knowledge base.
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Compared to other models on the leaderboard, DINO significantly reduces its model size and pre-training data size while achieving better results.
Ranked #1 on
Real-Time Object Detection
on COCO 2017 val
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
We introduce Grounded SAM, which uses Grounding DINO as an open-set object detector to combine with the segment anything model (SAM).
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
HigherHRNet even surpasses all top-down methods on CrowdPose test (67. 6% AP), suggesting its robustness in crowded scene.
Ranked #2 on
Pose Estimation
on UAV-Human
Lite-HRNet: A Lightweight High-Resolution Network
We introduce a lightweight unit, conditional channel weighting, to replace costly pointwise (1x1) convolutions in shuffle blocks.
Ranked #33 on
Pose Estimation
on COCO test-dev
(using extra training data)
VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results
Results of 33 object detection algorithms are presented.
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation
In this paper we present Mask DINO, a unified object detection and segmentation framework.
Ranked #1 on
Panoptic Segmentation
on COCO test-dev
Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases
However current research rarely studies the impact of different amounts of instruction data on model performance, especially in the real-world use cases.
WantWords: An Open-source Online Reverse Dictionary System
A reverse dictionary takes descriptions of words as input and outputs words semantically matching the input descriptions.
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Top-down visual attention mechanisms have been used extensively in image captioning and visual question answering (VQA) to enable deeper image understanding through fine-grained analysis and even multiple steps of reasoning.
Ranked #29 on
Visual Question Answering (VQA)
on VQA v2 test-std
Large-Scale Intelligent Microservices
Deploying Machine Learning (ML) algorithms within databases is a challenge due to the varied computational footprints of modern ML algorithms and the myriad of database technologies each with its own restrictive syntax.
Osprey: Pixel Understanding with Visual Instruction Tuning
In this paper, we propose Osprey, a mask-text instruction tuning approach, to extend MLLMs by incorporating fine-grained mask regions into language instruction, aiming at achieving pixel-wise visual understanding.
AIM 2020 Challenge on Efficient Super-Resolution: Methods and Results
This paper reviews the AIM 2020 challenge on efficient single image super-resolution with focus on the proposed solutions and results.
Gradient Centralization: A New Optimization Technique for Deep Neural Networks
It has been shown that using the first and second order statistics (e. g., mean and variance) to perform Z-score standardization on network activations or weight vectors, such as batch normalization (BN) and weight standardization (WS), can improve the training performance.
Tag2Text: Guiding Vision-Language Model via Image Tagging
This paper presents Tag2Text, a vision language pre-training (VLP) framework, which introduces image tagging into vision-language models to guide the learning of visual-linguistic features.
Recognize Anything: A Strong Image Tagging Model
We are releasing the RAM at \url{https://recognize-anything. github. io/} to foster the advancements of large models in computer vision.
Open-Set Image Tagging with Multi-Grained Text Supervision
Specifically, for predefined commonly used tag categories, RAM++ showcases 10. 2 mAP and 15. 4 mAP enhancements over CLIP on OpenImages and ImageNet.
GAN Prior Embedded Network for Blind Face Restoration in the Wild
The proposed GAN prior embedded network (GPEN) is easy-to-implement, and it can generate visually photo-realistic results.
Ranked #1 on
Blind Face Restoration
on CelebA-HQ
Mutual Consistency Learning for Semi-supervised Medical Image Segmentation
In this paper, we propose a novel mutual consistency network (MC-Net+) to effectively exploit the unlabeled data for semi-supervised medical image segmentation.
DN-DETR: Accelerate DETR Training by Introducing Query DeNoising
Our method is universal and can be easily plugged into any DETR-like methods by adding dozens of lines of code to achieve a remarkable improvement.
Grounded Language-Image Pre-training
The unification brings two benefits: 1) it allows GLIP to learn from both detection and grounding data to improve both tasks and bootstrap a good grounding model; 2) GLIP can leverage massive image-text pairs by generating grounding boxes in a self-training fashion, making the learned representation semantic-rich.
Ranked #1 on
2D Object Detection
on RF100
Semantic-SAM: Segment and Recognize Anything at Any Granularity
In this paper, we introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity.
Visual In-Context Prompting
In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergy
Recognizing the complementary strengths and weaknesses of both text and visual prompts, we introduce T-Rex2 that synergizes both prompts within a single model through contrastive learning.
DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR
We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR.
Ranked #11 on
2D Object Detection
on SARDet-100K
detrex: Benchmarking Detection Transformers
To address this issue, we develop a unified, highly modular, and lightweight codebase called detrex, which supports a majority of the mainstream DETR-based instance recognition algorithms, covering various fundamental tasks, including object detection, segmentation, and pose estimation.
Are Transformers Effective for Time Series Forecasting?
Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task.
Ranked #1 on
Time Series Forecasting
on ETTh1 (96) Univariate
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Discriminative model learning for image denoising has been recently attracting considerable attentions due to its favorable denoising performance.
A Simple Framework for Open-Vocabulary Segmentation and Detection
We present OpenSeeD, a simple Open-vocabulary Segmentation and Detection framework that jointly learns from different segmentation and detection datasets.
Ranked #2 on
Instance Segmentation
on ADE20K val
(using extra training data)
Unified Vision-Language Pre-Training for Image Captioning and VQA
The model is unified in that (1) it can be fine-tuned for either vision-language generation (e. g., image captioning) or understanding (e. g., visual question answering) tasks, and (2) it uses a shared multi-layer transformer network for both encoding and decoding, which differs from many existing methods where the encoder and decoder are implemented using separate models.
Ranked #1 on
Image Captioning
on Flickr30k Captions test
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks.
Ranked #1 on
Image Retrieval
on MS COCO
(Recall@10 metric)
Accelerating Dataset Distillation via Model Augmentation
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.
VinVL: Revisiting Visual Representations in Vision-Language Models
In our experiments we feed the visual features generated by the new object detection model into a Transformer-based VL fusion model \oscar \cite{li2020oscar}, and utilize an improved approach \short\ to pre-train the VL model and fine-tune it on a wide range of downstream VL tasks.
Ranked #2 on
Image-text matching
on CommercialAdsDataset
Blind Face Restoration via Deep Multi-scale Component Dictionaries
Next, with the degraded input, we match and select the most similar component features from their corresponding dictionaries and transfer the high-quality details to the input via the proposed dictionary feature transfer (DFT) block.
Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels
In this paper, we propose a principled formulation and framework by extending bicubic degradation based deep SISR with the help of plug-and-play framework to handle LR images with arbitrary blur kernels.
Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization
Diffusion models have demonstrated impressive performance in various image generation, editing, enhancement and translation tasks.
Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time
The small CNN works on the down-sampled version of the input image to predict content-dependent weights to fuse the multiple basis 3D LUTs into an image-adaptive one, which is employed to transform the color and tone of source images efficiently.
Ranked #5 on
Image Enhancement
on MIT-Adobe 5k
(SSIM on proRGB metric)
A Strong and Reproducible Object Detector with Only Public Datasets
This work presents Focal-Stable-DINO, a strong and reproducible object detection model which achieves 64. 6 AP on COCO val2017 and 64. 8 AP on COCO test-dev using only 700M parameters without any test time augmentation.
Ranked #5 on
Object Detection
on COCO minival
(using extra training data)
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
LLaVA-Plus is a general-purpose multimodal assistant that expands the capabilities of large multimodal models.
Ranked #1 on
LMM real-life tasks
on Leaderboard
Plug-and-Play Image Restoration with Deep Denoiser Prior
Recent works on plug-and-play image restoration have shown that a denoiser can implicitly serve as the image prior for model-based methods to solve many inverse problems.
Learning Deep CNN Denoiser Prior for Image Restoration
Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e. g., deblurring).
Ranked #1 on
Color Image Denoising
on BSD68 sigma5
One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer
It is challenging to perform this task with a single network due to resolution issues, i. e., the face and hands are usually located in extremely small regions.
Ranked #3 on
3D Human Pose Estimation
on UBody
Toward Convolutional Blind Denoising of Real Photographs
While deep convolutional neural networks (CNNs) have achieved impressive success in image denoising with additive white Gaussian noise (AWGN), their performance remains limited on real-world noisy photographs.
Ranked #4 on
Denoising
on Darmstadt Noise Dataset
ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation
In addition to the unprecedented ability in imaginary creation, large text-to-image models are expected to take customized concepts in image generation.
Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions
Though deep learning-based object detection methods have achieved promising results on the conventional datasets, it is still challenging to locate objects from the low-quality images captured in adverse weather conditions.
Improving Nighttime Driving-Scene Segmentation via Dual Image-adaptive Learnable Filters
With DIAL-Filters, we design both unsupervised and supervised frameworks for nighttime driving-scene segmentation, which can be trained in an end-to-end manner.
FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising
Due to the fast inference and good performance, discriminative learning methods have been widely studied in image denoising.
Ranked #1 on
Grayscale Image Denoising
on BSD68 sigma75
Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset
In this paper, we present Motion-X, a large-scale 3D expressive whole-body motion dataset.
Learning a Single Convolutional Super-Resolution Network for Multiple Degradations
Recent years have witnessed the unprecedented success of deep convolutional neural networks (CNNs) in single image super-resolution (SISR).
High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network
Existing image-to-image translation (I2IT) methods are either constrained to low-resolution images or long inference time due to their heavy computational burden on the convolution of high-resolution feature maps.
Ranked #1 on
Photo Retouching
on MIT-Adobe 5k (1080p)
Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding
This paper presents a new Vision Transformer (ViT) architecture Multi-Scale Vision Longformer, which significantly enhances the ViT of \cite{dosovitskiy2020image} for encoding high-resolution images using two techniques.
Ranked #45 on
Instance Segmentation
on COCO minival
Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution
In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention.
Query2Label: A Simple Transformer Way to Multi-Label Classification
The use of Transformer is rooted in the need of extracting local discriminative features adaptively for different labels, which is a strongly desired property due to the existence of multiple objects in one image.
Ranked #1 on
Multi-Label Classification
on PASCAL VOC 2012
Image Scene Graph Generation (SGG) Benchmark
There is a surge of interest in image scene graph generation (object, attribute and relationship detection) due to the need of building fine-grained image understanding models that go beyond object detection.
Progressive Semantic-Aware Style Transformation for Blind Face Restoration
Compared with previous networks, the proposed PSFR-GAN makes full use of the semantic (parsing maps) and pixel (LQ images) space information from different scales of input pairs.
Ranked #4 on
Blind Face Restoration
on CelebA-Test
Improving the Stability of Diffusion Models for Content Consistent Super-Resolution
To improve the stability of diffusion prior-based SR, we propose to employ the diffusion models to refine image structures, while employing the generative adversarial training to enhance image fine details.
MicroNet: Improving Image Recognition with Extremely Low FLOPs
This paper aims at addressing the problem of substantial performance degradation at extremely low computational cost (e. g. 5M FLOPs on ImageNet classification).
Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection
To demonstrate the superiority and generality of the proposed method, we evaluate the proposed method on five crack datasets and compare it with state-of-the-art crack detection, edge detection, semantic segmentation methods.
OTAvatar: One-shot Talking Face Avatar with Controllable Tri-plane Rendering
In this paper, we propose One-shot Talking face Avatar (OTAvatar), which constructs face avatars by a generalized controllable tri-plane rendering solution so that each personalized avatar can be constructed from only one portrait as the reference.
Object-driven Text-to-Image Synthesis via Adversarial Training
In this paper, we propose Object-driven Attentive Generative Adversarial Newtorks (Obj-GANs) that allow object-centered text-to-image synthesis for complex scenes.
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution
First, we train a degradation-aware prompt extractor, which can generate accurate soft and hard semantic prompts even under strong degradation.
Semi-supervised Left Atrium Segmentation with Mutual Consistency Training
Such mutual consistency encourages the two decoders to have consistent and low-entropy predictions and enables the model to gradually capture generalized features from these unlabeled challenging regions.
Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization
The proposed methods are highly modular, readily plugged into existing deep CNNs.
Ranked #1 on
Image Classification
on iNaturalist
(Top 3 Error metric)
Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution
In this paper, we demonstrate that it is possible to train a GAN-based SISR model which can stably generate perceptually realistic details while inhibiting visual artifacts.
HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation
While such a plug-and-play approach is appealing, the inevitable and uncertain conflicts between the original images produced from the frozen SD branch and the given condition incur significant challenges for the learnable branch, which essentially conducts image feature editing for condition enforcement.
PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency
HRP requires that more attention should be paid to human regions, while GLC requires that a group of portrait photos should be retouched to a consistent tone.
LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models
To address this issue, we have created GVC data that allows for the combination of grounding and chat capabilities.
UniPose: Detecting Any Keypoints
This work proposes a unified framework called UniPose to detect keypoints of any articulated (e. g., human and animal), rigid, and soft objects via visual or textual prompts for fine-grained vision understanding and manipulation.
Ranked #1 on
2D Human Pose Estimation
on Human-Art
(using extra training data)
Real-world Noisy Image Denoising: A New Benchmark
In order to promote the study on this problem while implementing the concurrent real-world image denoising datasets, we construct a new benchmark dataset which contains comprehensive real-world noisy images of different natural scenes.
Unsupervised Pre-training for Person Re-identification
In this paper, we present a large scale unlabeled person re-identification (Re-ID) dataset "LUPerson" and make the first attempt of performing unsupervised pre-training for improving the generalization ability of the learned person Re-ID feature representation.
Ranked #1 on
Person Re-Identification
on Market-1501
(using extra training data)
Large-Scale Pre-training for Person Re-identification with Noisy Labels
Since theses ID labels automatically derived from tracklets inevitably contain noises, we develop a large-scale Pre-training framework utilizing Noisy Labels (PNL), which consists of three learning modules: supervised Re-ID learning, prototype-based contrastive learning, and label-guided contrastive learning.
Ranked #7 on
Person Re-Identification
on CUHK03
Variational Denoising Network: Toward Blind Noise Modeling and Removal
On one hand, as other data-driven deep learning methods, our method, namely variational denoising network (VDN), can perform denoising efficiently due to its explicit form of posterior expression.
Ranked #10 on
Image Denoising
on DND
Deep Variational Network Toward Blind Image Restoration
In this proposed model, a pixel-wise non-i. i. d.
Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes
Humans have long been recorded in a variety of forms since antiquity.
Efficient Long-Range Attention Network for Image Super-resolution
A highly efficient long-range attention block (ELAB) is then built by simply cascading two shift-conv with a GMSA module, which is further accelerated by using a shared attention mechanism.
Ranked #11 on
Image Super-Resolution
on Manga109 - 4x upscaling
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds
VoxSeT is built upon a voxel-based set attention (VSA) module, which reduces the self-attention in each voxel by two cross-attentions and models features in a hidden space induced by a group of latent codes.
A PID Controller Approach for Stochastic Optimization of Deep Networks
We first reveal the intrinsic connections between SGD-Momentum and PID based controller, then present the optimization algorithm which exploits the past, current, and change of gradients to update the network parameters.
Box-supervised Instance Segmentation with Level Set Evolution
A simple mask supervised SOLOv2 model is adapted to predict the instance-aware mask map as the level set for each instance.
Detection Transformer with Stable Matching
We point out that the unstable matching in DETR is caused by a multi-optimization path problem, which is highlighted by the one-to-one matching design in DETR.
Lite DETR : An Interleaved Multi-Scale Encoder for Efficient DETR
Recent DEtection TRansformer-based (DETR) models have obtained remarkable performance.
Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation
Specifically, we approximate the joint distribution with two different factorized forms, which can be formulated as a denoiser mapping the noisy image to the clean one and a generator mapping the clean image to the noisy one.
Ranked #2 on
Noise Estimation
on SIDD
Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networks
Deep learning's performance has been extensively recognized recently.
A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution
The semantics of the text are firstly extracted by a text recognition module as text prior information.
Suppress and Balance: A Simple Gated Network for Salient Object Detection
With the help of multilevel gate units, the valuable context information from the encoder can be optimally transmitted to the decoder.
Ranked #15 on
Dichotomous Image Segmentation
on DIS-TE4
Towards Diverse Binary Segmentation via A Simple yet General Gated Network
They ignore two key problems when the encoder exchanges information with the decoder: one is the lack of interference control mechanism between them, the other is without considering the disparity of the contributions from different encoder levels.
Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalization
In this work, we, for the first time to our best knowledge, propose to perform Exact Feature Distribution Matching (EFDM) by exactly matching the empirical Cumulative Distribution Functions (eCDFs) of image features, which could be implemented by applying the Exact Histogram Matching (EHM) in the image feature space.
Learning Spatial-Temporal Regularized Correlation Filters for Visual Tracking
Compared with SRDCF, STRCF with hand-crafted features provides a 5 times speedup and achieves a gain of 5. 4% and 3. 6% AUC score on OTB-2015 and Temple-Color, respectively.
Ranked #9 on
Visual Object Tracking
on VOT2017/18
Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimation
This paper presents a novel end-to-end framework with Explicit box Detection for multi-person Pose estimation, called ED-Pose, where it unifies the contextual learning between human-level (global) and keypoint-level (local) information.
Ranked #2 on
2D Human Pose Estimation
on Human-Art
A Dual Weighting Label Assignment Scheme for Object Detection
Existing LA methods mostly focus on the design of pos weighting function, while the neg weight is directly derived from the pos weight.
Text Prior Guided Scene Text Image Super-resolution
Our experiments on the benchmark TextZoom dataset show that TPGSR can not only effectively improve the visual quality of scene text images, but also significantly improve the text recognition accuracy over existing STISR methods.
Deep Convolutional Dictionary Learning for Image Denoising
Inspired by the great success of deep neural networks (DNNs), many unfolding methods have been proposed to integrate traditional image modeling techniques, such as dictionary learning (DicL) and sparse coding, into DNNs for image restoration.
Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolution
Specifically, a tiny regression network is employed to predict the degradation parameters of the input image, while several convolutional experts with the same topology are jointly optimized to specify the network parameters via a non-linear mixture of experts.
UniVS: Unified and Universal Video Segmentation with Prompts as Queries
Despite the recent advances in unified image segmentation (IS), developing a unified video segmentation (VS) model remains a challenge.
Ranked #2 on
Video Semantic Segmentation
on VSPW
(using extra training data)
Referring Expression Segmentation
Referring Video Object Segmentation
+6
A Probabilistic Quality Representation Approach to Deep Blind Image Quality Prediction
Recognizing this, we propose a new representation of perceptual image quality, called probabilistic quality representation (PQR), to describe the image subjective score distribution, whereby a more robust loss function can be employed to train a deep BIQA model.
Learning Dual Memory Dictionaries for Blind Face Restoration
Generally, it is a challenging and intractable task to improve the photo-realistic performance of blind restoration and adaptively handle the generic and specific restoration scenarios with a single unified model.
Grid Anchor based Image Cropping: A New Benchmark and An Efficient Model
The employed evaluation metrics such as intersection-over-union cannot reliably reflect the real performance of a cropping model, either.
Reliable and Efficient Image Cropping: A Grid Anchor based Approach
Consequently, a grid anchor based cropping benchmark is constructed, where all crops of each image are annotated and more reliable evaluation metrics are defined.
Real-World Video Super-Resolution: A Benchmark Dataset and a Decomposition Based Learning Scheme
Existing VSR methods are mostly trained and evaluated on synthetic datasets, where the LR videos are uniformly downsampled from their high-resolution (HR) counterparts by some simple operators (e. g., bicubic downsampling).
Human Guided Ground-truth Generation for Realistic Image Super-resolution
A human guided GT image dataset with both positive and negative samples is then constructed, and a loss function is proposed to train the Real-ISR models.
Multi-channel Reverse Dictionary Model
A reverse dictionary takes the description of a target word as input and outputs the target word together with other words that match the description.
MP-Former: Mask-Piloted Transformer for Image Segmentation
We present a mask-piloted Transformer which improves masked-attention in Mask2Former for image segmentation.
From Face to Natural Image: Learning Real Degradation for Blind Image Super-Resolution
Notably, LQ face images, which may have the same degradation process as natural images, can be robustly restored with photo-realistic textures by exploiting their strong structural priors.
Deep Adaptive Inference Networks for Single Image Super-Resolution
Nonetheless, content and resource adaptive model is more preferred, and it is encouraging to apply simpler and efficient networks to the easier regions with less details and the scenarios with restricted efficiency constraints.
Dense Learning based Semi-Supervised Object Detection
Semi-supervised object detection (SSOD) aims to facilitate the training and deployment of object detectors with the help of a large amount of unlabeled data.
HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset
Secondly, we conduct more sophisticated alignment and temporal fusion in the feature space of the coarse HDR video to produce better reconstruction.
MosAIc: Finding Artistic Connections across Culture with Conditional Image Retrieval
We introduce MosAIc, an interactive web app that allows users to find pairs of semantically related artworks that span different cultures, media, and millennia.
Cultural Vocal Bursts Intensity Prediction
Image Retrieval
+2
CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise
We demonstrate the effectiveness of the proposed algorithm on both of the label noise detection task and the image classification on noisy data task on several large-scale datasets.
Ranked #2 on
Image Classification
on Food-101N
(using extra training data)
Blind Image Super-resolution with Elaborate Degradation Modeling on Noise and Kernel
To address the above issues, this paper proposes a model-based blind SISR method under the probabilistic framework, which elaborately models image degradation from the perspectives of noise and blur kernel.
Chasing Sparsity in Vision Transformers: An End-to-End Exploration
For example, our sparsified DeiT-Small at (5%, 50%) sparsity for (data, architecture), improves 0. 28% top-1 accuracy, and meanwhile enjoys 49. 32% FLOPs and 4. 40% running time savings.
Ranked #20 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks
To this end, we propose to learn implicit surface reconstruction by sign-agnostic optimization of convolutional occupancy networks, to simultaneously achieve advanced scalability to large-scale scenes, generality to novel shapes, and applicability to raw scans in a unified framework.
Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice
By using MCSD as a measure of domain distance, we develop a new domain adaptation bound for multi-class UDA; its data-dependent, probably approximately correct bound is also developed that naturally suggests adversarial learning objectives to align conditional feature distributions across source and target domains.
Generative Action Description Prompts for Skeleton-based Action Recognition
More specifically, we employ a pre-trained large-scale language model as the knowledge engine to automatically generate text descriptions for body parts movements of actions, and propose a multi-modal training scheme by utilizing the text encoder to generate feature vectors for different body parts and supervise the skeleton encoder for action representation learning.
Ranked #5 on
Skeleton Based Action Recognition
on N-UCLA
AutoLoc: Weakly-supervised Temporal Action Localization
In this paper, we first develop a novel weakly-supervised TAL framework called AutoLoc to directly predict the temporal boundary of each action instance.
Weakly-supervised Temporal Action Localization
Weakly Supervised Temporal Action Localization
A Benchmark for Edge-Preserving Image Smoothing
This benchmark includes an image dataset with groundtruth image smoothing results as well as baseline algorithms that can generate competitive edge-preserving smoothing results for a wide range of image contents.
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption
Due to this aligned representation learning, even pre-trained on the same downstream task dataset, TAP already boosts the absolute accuracy on the TextVQA dataset by +5. 4%, compared with a non-TAP baseline.
A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection
In this work, we design a single stream network to directly use the depth map to guide early fusion and middle fusion between RGB and depth, which saves the feature encoder of the depth stream and achieves a lightweight and real-time model.
Ranked #15 on
Thermal Image Segmentation
on RGB-T-Glass-Segmentation
Class-Balanced Pixel-Level Self-Labeling for Domain Adaptive Semantic Segmentation
One popular solution to this challenging task is self-training, which selects high-scoring predictions on target samples as pseudo labels for training.
Ranked #9 on
Image-to-Image Translation
on SYNTHIA-to-Cityscapes
Learning Domain Adaptive Object Detection with Probabilistic Teacher
In addition, we conduct anchor adaptation in parallel with localization adaptation, since anchor can be regarded as a learnable parameter.
Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport
Weakly-supervised image segmentation has recently attracted increasing research attentions, aiming to avoid the expensive pixel-wise labeling.
MomentDiff: Generative Video Moment Retrieval from Random to Real
Video moment retrieval pursues an efficient and generalized solution to identify the specific temporal segments within an untrimmed video that correspond to a given language description.
Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks
In this work, aiming at a general and comprehensive way for neural network acceleration, we develop a Wavelet-like Auto-Encoder (WAE) that decomposes the original input image into two low-resolution channels (sub-images) and incorporate the WAE into the classification neural networks for joint training.
Neural Interactive Keypoint Detection
Click-Pose explores how user feedback can cooperate with a neural keypoint detector to correct the predicted keypoints in an interactive way for a faster and more effective annotation process.
A Benchmark for Chinese-English Scene Text Image Super-resolution
Scene Text Image Super-resolution (STISR) aims to recover high-resolution (HR) scene text images with visually pleasant and readable text content from the given low-resolution (LR) input.
Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy
While single task image restoration (IR) has achieved significant successes, it remains a challenging issue to train a single model which can tackle multiple IR tasks.
Sharpness-Aware Gradient Matching for Domain Generalization
In this paper, we present two conditions to ensure that the model could converge to a flat minimum with a small loss, and present an algorithm, named Sharpness-Aware Gradient Matching (SAGM), to meet the two conditions for improving model generalization capability.
MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks
In contrast, though human engineers have the incredible ability to understand tasks and reason about solutions, their experience and knowledge are often sparse and difficult to utilize by quantitative approaches.
When Unsupervised Domain Adaptation Meets Tensor Representations
Domain adaption (DA) allows machine learning methods trained on data sampled from one distribution to be applied to data sampled from another.
Label-efficient Segmentation via Affinity Propagation
In this work, we formulate the affinity modeling as an affinity propagation process, and propose a local and a global pairwise affinity terms to generate accurate soft pseudo labels.
MSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection from Point Cloud Sequences
Current top-performing multi-frame detectors mostly follow a Detect-and-Fuse framework, which extracts features from each frame of the sequence and fuses them to detect the objects in the current frame.
Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution
To ensure the content consistency among adjacent frames, we exploit the temporal dynamics in LR videos to guide the diffusion process by optimizing the latent sampling path with a motion-guided loss, ensuring that the generated HR video maintains a coherent and continuous visual flow.
PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation
In this paper, we introduce a novel prompt-driven adapter into SAM, namely Prompt Adapter Segment Anything Model (PA-SAM), aiming to enhance the segmentation mask quality of the original SAM.
DQ-DETR: Dual Query Detection Transformer for Phrase Extraction and Grounding
As phrase extraction can be regarded as a $1$D text segmentation problem, we formulate PEG as a dual detection problem and propose a novel DQ-DETR model, which introduces dual queries to probe different features from image and text for object prediction and phrase mask prediction.
Ranked #7 on
Referring Expression Comprehension
on RefCOCO
Spatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition
For 3D video-based tasks such as action recognition, however, directly applying spatiotemporal transformers on video data will bring heavy computation and memory burdens due to the largely increased number of patches and the quadratic complexity of self-attention computation.
Ranked #9 on
Action Recognition
on Diving-48
Joint HDR Denoising and Fusion: A Real-World Mobile HDR Image Dataset
In this work, we develop, for the first time to our best knowledge, an HDR image dataset by using mobile phone cameras, namely Mobile-HDR dataset.
Neural Architecture Search With Representation Mutual Information
Building upon RMI, we further propose a new search algorithm termed RMI-NAS, facilitating with a theorem to guarantee the global optimal of the searched architecture.
WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection
We study on weakly-supervised object detection (WSOD) which plays a vital role in relieving human involvement from object-level annotations.
Conditional Directed Graph Convolution for 3D Human Pose Estimation
Based on this representation, we further propose a spatial-temporal conditional directed graph convolution to leverage varying non-local dependence for different poses by conditioning the graph topology on input poses.
Ranked #15 on
3D Human Pose Estimation
on MPI-INF-3DHP
A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift
Denoising and demosaicking are two essential steps to reconstruct a clean full-color image from the raw data.
One Shot Learning as Instruction Data Prospector for Large Language Models
Nuggets assesses the potential of individual instruction examples to act as effective one shot examples, thereby identifying those that can significantly enhance diverse task performance.
Learning Symmetry Consistent Deep CNNs for Face Completion
As for missing pixels on both of half-faces, we present a generative reconstruction subnet together with a perceptual symmetry loss to enforce symmetry consistency of recovered structures.
Ranked #1 on
Facial Inpainting
on VggFace2
One-to-Few Label Assignment for End-to-End Dense Detection
The positive and negative weights of these soft anchors are dynamically adjusted during training so that they can contribute more to ``representation learning'' in the early training stage, and contribute more to ``duplicated prediction removal'' in the later stage.
DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training
In this way, we can reduce the GPU memory consumption of contrastive loss computation from $\bigO(B^2)$ to $\bigO(\frac{B^2}{N})$, where $B$ and $N$ are the batch size and the number of GPUs used for training.
Attention Diversification for Domain Generalization
Under this guidance, a novel Attention Diversification framework is proposed, in which Intra-Model and Inter-Model Attention Diversification Regularization are collaborated to reassign appropriate attention to diverse task-related features.
Simultaneous Fidelity and Regularization Learning for Image Restoration
For blind deconvolution, as estimation error of blur kernel is usually introduced, the subsequent non-blind deconvolution process does not restore the latent image well.
Joint Denoising and Demosaicking with Green Channel Prior for Real-world Burst Images
Considering the fact that the green channel has twice the sampling rate and better quality than the red and blue channels in CFA raw data, we propose to use this green channel prior (GCP) to build a GCP-Net for the JDD-B task.
Boosting Weakly Supervised Object Detection with Progressive Knowledge Transfer
In this paper, we propose an effective knowledge transfer framework to boost the weakly supervised object detection accuracy with the help of an external fully-annotated source dataset, whose categories may not overlap with the target domain.
Boosting Human-Object Interaction Detection with Text-to-Image Diffusion Model
Extensive experiments demonstrate that DiffHOI significantly outperforms the state-of-the-art in regular detection (i. e., 41. 50 mAP) and zero-shot detection.
Ranked #2 on
Zero-Shot Human-Object Interaction Detection
on HICO-DET
(using extra training data)
Human-Object Interaction Detection
Zero-Shot Human-Object Interaction Detection
SEED: Self-supervised Distillation For Visual Representation
This paper is concerned with self-supervised learning for small models.
Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation
To further explore temporal correlation among video frames, we aggregate a temporal fusion module to infer instance masks from each frame to its adjacent frames, which helps our framework to handle challenging videos such as motion blur, partial occlusion and unusual object-to-camera poses.
Ranked #24 on
Video Instance Segmentation
on YouTube-VIS 2021
ScatterFormer: Efficient Voxel Transformer with Scattered Linear Attention
Window-based transformers have demonstrated strong ability in large-scale point cloud understanding by capturing context-aware representations with affordable attention computation in a more localized manner.
SIM: Semantic-aware Instance Mask Generation for Box-Supervised Instance Segmentation
Weakly supervised instance segmentation using only bounding box annotations has recently attracted much research attention.
TMP: Temporal Motion Propagation for Online Video Super-Resolution
Online video super-resolution (online-VSR) highly relies on an effective alignment module to aggregate temporal information, while the strict latency requirement makes accurate and efficient alignment very challenging.
PMFSNet: Polarized Multi-scale Feature Self-attention Network For Lightweight Medical Image Segmentation
In this work, we propose PMFSNet, a novel medical imaging segmentation model that effectively balances global and local feature processing while avoiding the computational redundancy typical in larger models.
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning
In this study, we present a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems.
TIGEr: Text-to-Image Grounding for Image Caption Evaluation
This paper presents a new metric called TIGEr for the automatic evaluation of image captioning systems.
Unsupervised Adaptation Learning for Hyperspectral Imagery Super-Resolution
Following this idea, we develop a two-stage SR network that leverages two consecutive modules: a fusion module and an adaptation module, to recover the latent HSI in a coarse-to-fine scheme.
Glocal Energy-based Learning for Few-Shot Open-Set Recognition
Few-shot open-set recognition (FSOR) is a challenging task of great practical value.
CORE: Cooperative Reconstruction for Multi-Agent Perception
This paper presents CORE, a conceptually simple, effective and communication-efficient model for multi-agent cooperative perception.
LipsFormer: Introducing Lipschitz Continuity to Vision Transformers
In contrast to previous practical tricks that address training instability by learning rate warmup, layer normalization, attention formulation, and weight initialization, we show that Lipschitz continuity is a more essential property to ensure training stability.
Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction
This paper focuses on the task of 4D shape reconstruction from a sequence of point clouds.
SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images
The existing deep learning fusion methods mainly concentrate on the convolutional neural networks, and few attempts are made with transformer.
Hashing-based Non-Maximum Suppression for Crowded Object Detection
Considering the intersection-over-union (IoU) as the metric, we propose a simple yet effective hashing algorithm, named IoUHash, which guarantees that the boxes within the same cell are close enough by a lower IoU bound.
A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment
While Multimodal Large Language Models (MLLMs) have experienced significant advancement on visual understanding and reasoning, their potentials to serve as powerful, flexible, interpretable, and text-driven models for Image Quality Assessment (IQA) remains largely unexplored.
Isomer: Isomerous Transformer for Zero-shot Video Object Segmentation
Recent leading zero-shot video object segmentation (ZVOS) works devote to integrating appearance and motion information by elaborately designing feature fusion modules and identically applying them in multiple feature stages.
Masked Surfel Prediction for Self-Supervised Point Cloud Learning
In this work, we make the first attempt, to the best of our knowledge, to consider the local geometry information explicitly into the masked auto-encoding, and propose a novel Masked Surfel Prediction (MaskSurf) method.
Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges
Then, the learnt knowledge from edge clients will be aggregated by centralized parameter server, where the knowledge will be selectively and attentively distilled from spatial- and temporal-dimension with carefully designed mechanisms.
Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domains
In this work, we present an idea to remove non-causal factors from common features by multi-view adversarial training on source domains, because we observe that such insignificant non-causal factors may still be significant in other latent spaces (views) due to the multi-mode structure of data.
Directional Deep Embedding and Appearance Learning for Fast Video Object Segmentation
We propose a directional deep embedding and appearance learning (DDEAL) method, which is free of the online fine-tuning process, for fast VOS.
Unfolded Deep Kernel Estimation for Blind Image Super-resolution
Nonetheless, the existing deep unfolding methods cannot explicitly solve the data term of the unfolding objective function, limiting their capability in blur kernel estimation.
Generating Diverse and Accurate Visual Captions by Comparative Adversarial Learning
By contrasting with human-written captions and image-mismatched captions, the caption generator effectively exploits the inherent characteristics of human languages, and generates more discriminative captions.
Virtual Fully-Connected Layer: Training a Large-Scale Face Recognition Dataset With Limited Computational Resources
This is because the classification paradigm needs to train a fully connected layer as the category classifier, and its parameters will be in the hundreds of millions if the training dataset contains millions of identities.
Beyond a Video Frame Interpolator: A Space Decoupled Learning Approach to Continuous Image Transition
Most of the existing deep learning based VFI methods adopt off-the-shelf optical flow algorithms to estimate the bidirectional flows and interpolate the missing frames accordingly.
Adaptive Network Combination for Single-Image Reflection Removal: A Domain Generalization Perspective
Meanwhile, diverse testing sets are also provided with different types of reflection and scenes.
Revisiting Prototypical Network for Cross Domain Few-Shot Learning
Prototypical Network is a popular few-shot solver that aims at establishing a feature metric generalizable to novel few-shot classification (FSC) tasks using deep neural networks.
HyperLLaVA: Dynamic Visual and Language Expert Tuning for Multimodal Large Language Models
Recent advancements indicate that scaling up Multimodal Large Language Models (MLLMs) effectively enhances performance on downstream multimodal tasks.
Ranked #72 on
Visual Question Answering
on MM-Vet
Probability Weighted Compact Feature for Domain Adaptive Retrieval
Most of the existing image retrieval methods only focus on single-domain retrieval, which assumes that the distributions of retrieval databases and queries are similar.
Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation
To make the proposed A$^2$LP useful for UDA, we propose empirical schemes to generate such virtual instances.
Variational Attention: Propagating Domain-Specific Knowledge for Multi-Domain Learning in Crowd Counting
In crowd counting, due to the problem of laborious labelling, it is perceived intractability of collecting a new large-scale dataset which has plentiful images with large diversity in density, scene, etc.
Benchmark Dataset and Effective Inter-Frame Alignment for Real-World Video Super-Resolution
On the other hand, alignment algorithms in existing VSR methods perform poorly for real-world videos, leading to unsatisfactory results.
IVAC-P2L: Leveraging Irregular Repetition Priors for Improving Video Action Counting
Our research addresses the shortfall by introducing a novel approach to VAC, called Irregular Video Action Counting (IVAC).
Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models
In this paper, we introduce a versatile adaptation approach that can effectively work under all three settings.
DR-Unet104 for Multimodal MRI brain tumor segmentation
We verified the effect of introducing the regularisation of dropout with small rate (e. g. 0. 2) on the architecture, and found a dropout of 0. 2 improved the overall performance compared to no dropout, or a dropout of 0. 5.
FPR: False Positive Rectification for Weakly Supervised Semantic Segmentation
Without introducing any external supervision and human priors, the proposed FPR effectively suppresses wrong activations from the background objects.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Multi-adversarial Faster-RCNN for Unrestricted Object Detection
Conventional object detection methods essentially suppose that the training and testing data are collected from a restricted target domain with expensive labeling cost.
Semi-Supervised Domain Generalization with Evolving Intermediate Domain
From this perspective, we introduce a novel paradigm of DG, termed as Semi-Supervised Domain Generalization (SSDG), to explore how the labeled and unlabeled source domains can interact, and establish two settings, including the close-set and open-set SSDG.
A Survey on Leveraging Pre-trained Generative Adversarial Networks for Image Editing and Restoration
Generative adversarial networks (GANs) have drawn enormous attention due to the simple yet effective training mechanism and superior image generation quality.
Inferring and Leveraging Parts from Object Shape for Improving Semantic Image Synthesis
Despite the progress in semantic image synthesis, it remains a challenging problem to generate photo-realistic parts from input semantic map.
Parameter Exchange for Robust Dynamic Domain Generalization
The parameters of dynamic networks can be decoupled into a static and a dynamic component, which are designed to learn domain-invariant and domain-specific features, respectively.
Self-Supervised Video Desmoking for Laparoscopic Surgery
On the other hand, in order to enhance the desmoking performance, we further feed the valuable information from PS frame into models, where a masking strategy and a regularization term are presented to avoid trivial solutions.
Towards Efficient Data Free Black-Box Adversarial Attack
The proposed method can efficiently imitate the target model through a small number of queries and achieve high attack success rate.
Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle
In this paper, we try to solve this problem by introducing a principled and unified framework based on Information Bottleneck (IB) theory, which further guides us to an automatic pruning approach.
Remove Cosine Window from Correlation Filter-based Visual Trackers: When and How
Therefore, we in this paper investigate the feasibility to remove cosine window from CF trackers with spatial regularization.
MDQE: Mining Discriminative Query Embeddings to Segment Occluded Instances on Challenging Videos
The proposed MDQE is the first VIS method with per-clip input that achieves state-of-the-art results on challenging videos and competitive performance on simple videos.
Ranked #13 on
Video Instance Segmentation
on YouTube-VIS 2021
Optimization-Free Test-Time Adaptation for Cross-Person Activity Recognition
However, the high computational cost of optimization-based TTA algorithms makes it intractable to run on resource-constrained edge devices.
Toward Accurate and Temporally Consistent Video Restoration from Raw Data
Extensive experiments demonstrate the leading VJDD performance of our method in term of restoration accuracy, perceptual quality and temporal consistency.
Solving Rubik's Cube with a Robot Hand
We demonstrate that models trained only in simulation can be used to solve a manipulation problem of unprecedented complexity on a real robot.
