Search Results for author:
Found 190 papers, 143 papers with code
Fixing Localization Errors to Improve Image Classification
This way, in an effort to fix localization errors, our loss provides an extra supervisory signal that helps the model to better discriminate between similar classes.
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
To address this gap in the geospatial domain, we present GEOBench-VLM, a comprehensive benchmark specifically designed to evaluate VLMs on geospatial tasks, including scene understanding, object counting, localization, fine-grained categorization, and temporal analysis.
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
In pursuit of culturally diverse global multimodal models, our proposed All Languages Matter Benchmark (ALM-bench) represents the largest and most comprehensive effort to date for evaluating LMMs across 100 languages.
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos
To enable fine-grained grounding, we curate a multimodal dataset featuring detailed visually-grounded conversations using a semiautomatic annotation pipeline, resulting in a diverse set of 38k video-QA triplets along with 83k objects and 671k masks.
ROAD-Waymo: Action Awareness at Scale for Autonomous Driving
Autonomous Vehicle (AV) perception systems require more than simply seeing, via e. g., object detection or scene segmentation.
COSNet: A Novel Semantic Segmentation Network using Enhanced Boundaries in Cluttered Scenes
Automated waste recycling aims to efficiently separate the recyclable objects from the waste by employing vision-based systems.
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark
In this work, we develop a comprehensive LMM evaluation benchmark for the Arabic language to represent a large population of over 400 million speakers.
How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?
Moreover, they heavily suffer from catastrophic forgetting and concept neglect on old personalized concepts when continually learning a series of new concepts.
Frontiers in Intelligent Colonoscopy
Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer.
A New Perspective to Boost Performance Fairness for Medical Federated Learning
However, existing fair FL methods ignore the specific characteristics of medical FL applications, i. e., domain shift among the datasets from different hospitals.
AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment
Further, this domain desires fine-grained feature learning due to the subtle nature of the downstream tasks (e. g, nutrient deficiency detection, livestock breed classification).
Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking
To address this limitation, we introduce open-vocabulary 3D tracking, which extends the scope of 3D tracking to include objects beyond predefined categories.
CDChat: A Large Multimodal Model for Remote Sensing Change Description
In order to achieve this, we introduce a change description instruction dataset that can be utilized to finetune an LMM and provide better change descriptions for RS images.
Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region
Accurate weather and climate modeling is critical for both scientific advancement and safeguarding communities against environmental risks.
CONDA: Condensed Deep Association Learning for Co-Salient Object Detection
They directly rely on raw associations which are not reliable in complex scenarios, and their image feature optimization approach is not explicit for inter-image association modeling.
BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learning
Inspired by the latest developments in learnable prompts, this work introduces a method to embed a backdoor into the medical foundation model during the prompt learning phase.
Underwater Object Detection Enhancement via Channel Stabilization
The best-performing backbone method incorporates our channel stabilization and augmentation techniques.
GroupMamba: Parameter-Efficient and Accurate Group Visual State Space Model
Recent advancements in state-space models (SSMs) have showcased effective performance in modeling long-range dependencies with subquadratic complexity.
FANet: Feature Amplification Network for Semantic Segmentation in Cluttered Background
Existing deep learning approaches leave out the semantic cues that are crucial in semantic segmentation present in complex scenarios including cluttered backgrounds and translucent objects, etc.
CPT: Consistent Proxy Tuning for Black-box Optimization
This consistency benefits Proxy-tuning and enhances model performance.
CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation
Our method achieves an absolute increase of 3. 9% in performance compared to existing methods for zero-shot learning multi-label classification tasks.
Open-Vocabulary Temporal Action Localization using Multimodal Guidance
Open-Vocabulary Temporal Action Localization (OVTAL) enables a model to recognize any desired action category in videos without the need to explicitly curate training data for all categories.
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs
Despite their impressive capabilities, current Video-LMMs have not been evaluated for anomaly detection tasks, which is critical to their deployment in practical scenarios e. g., towards identifying deepfakes, manipulated video content, traffic accidents and crimes.
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding
Building on the advances of language models, Large Multimodal Models (LMMs) have contributed significant improvements in video understanding.
Ranked #1 on
VCGBench-Diverse
on VideoInstruct
Towards Evaluating the Robustness of Visual State Space Models
To gain a deeper understanding of VSSMs' adversarial robustness, we conduct a frequency-based analysis of adversarial attacks, evaluating their performance against low-frequency and high-frequency perturbations.
On Evaluating Adversarial Robustness of Volumetric Medical Segmentation Models
We extend this investigation across four volumetric segmentation datasets, evaluating robustness under both white box and black box adversarial attacks.
Efficient 3D-Aware Facial Image Editing via Attribute-Specific Prompt Learning
The LAE harnesses a pre-trained vision-language model to find text-guided attribute-specific editing direction in the latent space of any pre-trained 3D-aware GAN.
Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation
To this end, we propose a fast yet accurate open-vocabulary 3D instance segmentation approach, named Open-YOLO 3D, that effectively leverages only 2D object detection from multi-view RGB images for open-vocabulary 3D instance segmentation.
3D Instance Segmentation
3D Open-Vocabulary Instance Segmentation
+4
Dual Hyperspectral Mamba for Efficient Spectral Compressive Imaging
Experiments verify the benefits of our DHM for HSI reconstruction.
Multi-modal Generation via Cross-Modal In-Context Learning
Moreover, they often result in misaligned image generation for prompt sequences featuring multiple objects.
Adapting Large Multimodal Models to Distribution Shifts: The Role of In-Context Learning
Recent studies indicate that large multimodal models (LMMs) potentially act as general-purpose assistants and are highly robust against different distributions.
How Good is my Video LMM? Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMs
Recent advancements in Large Language Models (LLMs) have led to the development of Video Large Multi-modal Models (Video-LMMs) that can handle a wide range of video understanding tasks.
Visual-Augmented Dynamic Semantic Prototype for Generative Zero-Shot Learning
Generative Zero-shot learning (ZSL) learns a generator to synthesize visual samples for unseen classes, which is an effective way to advance ZSL.
Cross-Modal Self-Training: Aligning Images and Pointclouds to Learn Classification without Labels
Thereby we demonstrate that 2D vision language models such as CLIP can be used to complement 3D representation learning to improve classification performance without the need for expensive class annotations.
Progressive Semantic-Guided Vision Transformer for Zero-Shot Learning
ZSLViT mainly considers two properties in the whole network: i) discover the semantic-related visual representations explicitly, and ii) discard the semantic-unrelated visual information.
Dynamic Pre-training: Towards Efficient and Scalable All-in-One Image Restoration
All-in-one image restoration tackles different types of degradations with a unified model instead of having task-specific, non-generic models for each degradation.
Language Guided Domain Generalized Medical Image Segmentation
Incorporating text features alongside visual features is a potential solution to enhance the model's understanding of the data, as it goes beyond pixel-level information to provide valuable context.
Efficient Video Object Segmentation via Modulated Cross-Attention Memory
Recently, transformer-based approaches have shown promising results for semi-supervised video object segmentation.
ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detection
Deep learning has shown remarkable success in remote sensing change detection (CD), aiming to identify semantic change regions between co-registered satellite image pairs acquired at distinct time stamps.
Composed Video Retrieval via Enriched Context and Discriminative Embeddings
Composed video retrieval (CoVR) is a challenging problem in computer vision which has recently highlighted the integration of modification text with visual queries for more sophisticated video search in large databases.
VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding
In contrast, this paper introduces a Video Understanding and Reasoning Framework (VURF) based on the reasoning power of LLMs.
Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learning
Self-supervised representation learning has been highly promising for histopathology image analysis with numerous approaches leveraging their patient-slide-patch hierarchy to learn better representations.
AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation
Our approach is motivated by the observation that different degradation types impact the image content on different frequency subbands, thereby requiring different treatments for each restoration task.
Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery
Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks by pre-training on large amount of unlabelled data.
ObjectCompose: Evaluating Resilience of Vision-Based Models on Object-to-Background Compositional Changes
We produce various versions of standard vision datasets (ImageNet, COCO), incorporating either diverse and realistic backgrounds into the images or introducing color, texture, and adversarial changes in the background.
MedContext: Learning Contextual Cues for Efficient Volumetric Medical Segmentation
The proposed approach induces contextual knowledge in the network by learning to reconstruct the missing organ or parts of an organ in the output segmentation space.
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT
"Bigger the better" has been the predominant trend in recent Large Language Models (LLMs) development.
PALO: A Polyglot Large Multimodal Model for 5B People
PALO offers visual reasoning capabilities in 10 major languages, including English, Chinese, Hindi, Spanish, French, Arabic, Bengali, Russian, Urdu, and Japanese, that span a total of ~5B people (65% of the world population).
BiMediX: Bilingual Medical Mixture of Experts LLM
In this paper, we introduce BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic.
VideoGrounding-DINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding
Our contributions include a novel spatio-temporal video grounding model surpassing state-of-the-art results in closed-set evaluations on multiple datasets and demonstrating superior performance in open-vocabulary scenarios.
Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding
Our contributions include a novel spatio-temporal video grounding model, surpassing state-of-the-art results in closed-set evaluations on multiple datasets and demonstrating superior performance in open-vocabulary scenarios.
VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering
By feeding the retrieved image and question to the VQA model, one can find the images inconsistent with relative caption when the answer by VQA is inconsistent with the answer in the QA pair.
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM
To this end, we propose a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability.
How Well Does GPT-4V(ision) Adapt to Distribution Shifts? A Preliminary Investigation
We further investigate its adaptability to controlled data perturbations and examine the efficacy of in-context learning as a tool to enhance its adaptation.
Spatiotemporal Event Graphs for Dynamic Scene Understanding
In this thesis, we present a series of frameworks for dynamic scene understanding starting from road event detection from an autonomous driving perspective to complex video activity detection, followed by continual learning approaches for the life-long learning of the models.
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
Furthermore, the lack of domain-specific multimodal instruction following data as well as strong backbone models for RS make it hard for the models to align their behavior with user queries.
Hardware Resilience Properties of Text-Guided Image Classifiers
This paper presents a novel method to enhance the reliability of image classification models during deployment in the face of transient hardware errors.
PG-Video-LLaVA: Pixel Grounding Large Video-Language Models
Extending image-based Large Multimodal Models (LMMs) to videos is challenging due to the inherent complexity of video data.
Enhancing Novel Object Detection via Cooperative Foundational Models
We present a novel approach to transform existing closed-set detectors into open-set detectors.
Ranked #1 on
Novel Object Detection
on LVIS v1.0 val
GLaMM: Pixel Grounding Large Multimodal Model
In this work, we present Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly intertwined with corresponding object segmentation masks.
Cal-DETR: Calibrated Detection Transformer
Third, we develop a logit mixing approach that acts as a regularizer with detection-specific losses and is also complementary to the uncertainty-guided logit modulation technique to further improve the calibration performance.
Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization
The promising zero-shot generalization of vision-language models such as CLIP has led to their adoption using prompt learning for numerous downstream tasks.
A Hybrid Graph Network for Complex Activity Detection in Video
Attention is then applied to this graph to obtain an overall representation of the local dynamic scene.
Videoprompter: an ensemble of foundational models for zero-shot video understanding
Consistent improvements across multiple benchmarks and with various VLMs demonstrate the effectiveness of our proposed framework.
Ranked #4 on
Video-Text Retrieval
on Test-of-Time
LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts
Diffusion-based generative models have significantly advanced text-to-image generation but encounter challenges when processing lengthy and intricate text prompts describing complex scenes with multiple objects.
Sentence-level Prompts Benefit Composed Image Retrieval
Composed image retrieval (CIR) is the task of retrieving specific images by using a query that involves both a reference image and a relative caption.
Ranked #2 on
Image Retrieval
on Fashion IQ
(using extra training data)
3D Indoor Instance Segmentation in an Open-World
We argue that such a closed-world assumption is restrictive and explore for the first time 3D indoor instance segmentation in an open-world setting, where the model is allowed to distinguish a set of known classes as well as identify an unknown object as unknown and then later incrementally learning the semantic category of the unknown when the corresponding category labels are available.
Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation
We decompose the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance, respectively.
Unsupervised Landmark Discovery Using Consistency Guided Bottleneck
In the current work, we introduce a consistency-guided bottleneck in an image reconstruction-based pipeline that leverages landmark consistency, a measure of compatibility score with the pseudo-ground truth to generate adaptive heatmaps.
Towards Realistic Zero-Shot Classification via Self Structural Semantic Alignment
To address this challenge, we propose the Self Structural Semantic Alignment (S^3A) framework, which extracts the structural semantic information from unlabeled data while simultaneously self-learning.
Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning
In this paper, we focus on a particular setting of learning adaptive prompts on the fly for each test sample from an unseen new domain, which is known as test-time prompt tuning (TPT).
Towards Instance-adaptive Inference for Federated Learning
However, the performance of the global model is often hampered by non-i. i. d.
How Good is Google Bard's Visual Understanding? An Empirical Study on Open Challenges
Google's Bard has emerged as a formidable competitor to OpenAI's ChatGPT in the field of conversational AI.
Foundational Models Defining a New Era in Vision: A Survey and Outlook
Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world.
Frequency Domain Adversarial Training for Robust Volumetric Medical Segmentation
While recent advances in deep learning have improved the performance of volumetric medical image segmentation models, these models cannot be deployed for real-world applications immediately due to their vulnerability to adversarial attacks.
Self-regulating Prompts: Foundational Model Adaptation without Forgetting
To the best of our knowledge, this is the first regularization framework for prompt learning that avoids overfitting by jointly attending to pre-trained model features, the training trajectory during prompting, and the textual diversity.
Ranked #2 on
Prompt Engineering
on ImageNet V2
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition
Video transformer designs are based on self-attention that can model global context at a high computational cost.
Ranked #2 on
Action Recognition
on Diving-48
PromptIR: Prompting for All-in-One Blind Image Restoration
We present a prompt-based learning approach, PromptIR, for All-In-One image restoration that can effectively restore images from various types and levels of degradation.
Learnable Weight Initialization for Volumetric Medical Image Segmentation
Hybrid volumetric medical image segmentation models, combining the advantages of local convolution and global attention, have recently received considerable attention.
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks.
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
Conversation agents fueled by Large Language Models (LLMs) are providing a new way to interact with visual data.
Ranked #5 on
Question Answering
on NExT-QA (Open-ended VideoQA)
VCGBench-Diverse
Video-based Generative Performance Benchmarking (Consistency)
+6
Concept Drift and Long-Tailed Distribution in Fine-Grained Visual Categorization: Benchmark and Method
In the existing FGVC datasets used in computer vision, it is generally assumed that each collected instance has fixed characteristics and the distribution of different categories is relatively balanced.
Modulate Your Spectrum in Self-Supervised Learning
In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse.
Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
Then, we use two types of pre-defined tokens to mine co-saliency and background information via our proposed contrast-induced pixel-to-token correlation and co-saliency token-to-token correlation modules.
Ranked #1 on
Co-Salient Object Detection
on CoSal2015
Remote Sensing Change Detection With Transformers Trained from Scratch
Current transformer-based change detection (CD) approaches either employ a pre-trained model trained on large-scale image classification ImageNet dataset or rely on first pre-training on another CD dataset and then fine-tuning on the target benchmark.
Gated Multi-Resolution Transfer Network for Burst Restoration and Enhancement
Burst image processing is becoming increasingly popular in recent years.
Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting
Through this prompting scheme, we can achieve state-of-the-art zero-shot performance on Kinetics-600, HMDB51 and UCF101 while remaining competitive in the supervised setting.
Burstormer: Burst Image Restoration and Enhancement Transformer
Unlike existing methods, the proposed alignment module not only aligns burst features but also exchanges feature information and maintains focused communication with the reference frame through the proposed reference-based feature enrichment mechanism, which facilitates handling complex motions.
Generative Multiplane Neural Radiance for 3D-Aware Image Generation
We present a method to efficiently generate 3D-aware high-resolution images that are view-consistent across multiple target views.
Video Instance Segmentation in an Open-World
Open-world formulation relaxes the close-world static-learning assumption as follows: (a) first, it distinguishes a set of known categories as well as labels an unknown object as `unknown' and then (b) it incrementally learns the class of an unknown as and when the corresponding semantic labels become available.
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications
Using our proposed efficient additive attention, we build a series of models called "SwiftFormer" which achieves state-of-the-art performance in terms of both accuracy and mobile inference speed.
Bridging Precision and Confidence: A Train-Time Loss for Calibrating Object Detection
Since the original formulation of our loss depends on the counts of true positives and false positives in a minibatch, we develop a differentiable proxy of our loss that can be used during training with other application-specific loss functions.
Boosting Adversarial Transferability using Dynamic Cues
Our temporal prompts are the result of a learnable transformation that allows optimizing for temporal gradients during an adversarial attack to fool the motion dynamics.
Guidance Through Surrogate: Towards a Generic Diagnostic Attack
Recently, different adversarial training defenses are proposed that not only maintain a high clean accuracy but also show significant robustness against popular and well studied adversarial attacks such as PGD.
PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discovery
The GNCD setting aims to categorize unlabeled training data coming from known and novel classes by leveraging the information of partially labeled known classes.
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation
Owing to the success of transformer models, recent works study their applicability in 3D medical segmentation tasks.
Fine-tuned CLIP Models are Efficient Video Learners
Since training on a similar scale for videos is infeasible, recent approaches focus on the effective transfer of image-based CLIP to the video domain.
An Investigation into Whitening Loss for Self-supervised Learning
A desirable objective in self-supervised learning (SSL) is to avoid feature collapse.
CLIP model is an Efficient Continual Learner
In this work, we show that a frozen CLIP (Contrastive Language-Image Pretraining) model offers astounding continual learning performance without any fine-tuning (zero-shot evaluation).
MaPLe: Multi-modal Prompt Learning
Pre-trained vision-language (V-L) models such as CLIP have shown excellent generalization ability to downstream tasks.
Ranked #2 on
Prompt Engineering
on ImageNet-A
ROAD-R: The Autonomous Driving Dataset with Logical Requirements
Neural networks have proven to be very powerful at computer vision tasks.
Situation Awareness for Automated Surgical Check-listing in AI-Assisted Operating Room
Nowadays, there are more surgical procedures that are being performed using minimally invasive surgery (MIS).
Transformers in Remote Sensing: A Survey
Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade.
AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility
While being greatly benefiting to the tracking research, existing benchmarks do not pose the same difficulty as before with recent trackers achieving higher performance mainly due to (i) the introduction of more sophisticated transformers-based methods and (ii) the lack of diverse scenarios with adverse visibility such as, severe weather conditions, camouflage and imaging effects.
3D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field.
D3Former: Debiased Dual Distilled Transformer for Incremental Learning
In contrast to a recent ViT based CIL approach, our $\textrm{D}^3\textrm{Former}$ does not dynamically expand its architecture when new tasks are learned and remains suitable for a large number of incremental tasks.
Self-Distilled Vision Transformer for Domain Generalization
Similar to CNNs, ViTs also struggle in out-of-distribution scenarios and the main culprit is overfitting to source domains.
Adversarial Pixel Restoration as a Pretext Task for Transferable Perturbations
Our training approach is based on a min-max scheme which reduces overfitting via an adversarial objective and thus optimizes for a more generalizable surrogate model.
Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detection
Two popular forms of weak-supervision used in open-vocabulary detection (OVD) include pretrained CLIP model and image-level supervision.
Ranked #1 on
Open Vocabulary Object Detection
on OpenImages-v4
OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learning
In the open-world SSL problem, the objective is to recognize samples of known classes, and simultaneously detect and cluster samples belonging to novel classes present in unlabeled data.
Ranked #1 on
Open-World Semi-Supervised Learning
on CIFAR-10
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications
Our EdgeNeXt model with 1. 3M parameters achieves 71. 2% top-1 accuracy on ImageNet-1K, outperforming MobileViT with an absolute gain of 2. 2% with 28% reduction in FLOPs.
Ranked #29 on
Semantic Segmentation
on PASCAL VOC 2012 test
NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results
The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29. 00dB on DIV2K validation set.
Self-Supervised Video Object Segmentation via Cutout Prediction and Tagging
We propose a novel self-supervised Video Object Segmentation (VOS) approach that strives to achieve better object-background discriminability for accurate object segmentation.
Learning Enriched Features for Fast Image Restoration and Enhancement
In the former case, spatial details are preserved but the contextual information cannot be precisely encoded.
Energy-based Latent Aligner for Incremental Learning
Deep learning models tend to forget their earlier knowledge while incrementally learning new tasks.
Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer
When using the ResNet50 backbone, our MS-STS achieves a mask AP of 50. 1 %, outperforming the best reported results in literature by 2. 7 % and by 4. 8 % at higher overlap threshold of AP_75, while being comparable in model size and speed on Youtube-VIS 2019 val.
Transformers in Medical Imaging: A Survey
Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators.
Vision in adverse weather: Augmentation using CycleGANs with various object detectors for robust perception in autonomous racing
In an autonomous driving system, perception - identification of features and objects from the environment - is crucial.
A Novel Incremental Learning Driven Instance Segmentation Framework to Recognize Highly Cluttered Instances of the Contraband Items
Screening cluttered and occluded contraband items from baggage X-ray scans is a cumbersome task even for the expert security staff.
Spatio-temporal Relation Modeling for Few-shot Action Recognition
Experiments are performed on four few-shot action recognition benchmarks: Kinetics, SSv2, HMDB51 and UCF101.
Ranked #1 on
Few Shot Action Recognition
on UCF101
(using extra training data)
DoodleFormer: Creative Sketch Drawing with Transformers
Creative sketch image generation is a challenging vision problem, where the task is to generate diverse, yet realistic creative sketches possessing the unseen composition of the visual-world objects.
OW-DETR: Open-world Detection Transformer
In the case of incremental object detection, OW-DETR outperforms the state-of-the-art for all settings on PASCAL VOC.
Self-supervised Video Transformer
To the best of our knowledge, the proposed approach is the first to alleviate the dependency on negative samples or dedicated memory banks in Self-supervised Video Transformer (SVT).
Class-agnostic Object Detection with Multi-modal Transformer
This has been a long-standing question in computer vision.
Restormer: Efficient Transformer for High-Resolution Image Restoration
Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks.
Ranked #1 on
Grayscale Image Denoising
on Urban100 sigma15
International Workshop on Continual Semi-Supervised Learning: Introduction, Benchmarks and Baselines
The aim of this paper is to formalize a new continual semi-supervised learning (CSSL) paradigm, proposed to the attention of the machine learning community via the IJCAI 2021 International Workshop on Continual Semi-Supervised Learning (CSSL-IJCAI), with the aim of raising field awareness about this problem and mobilizing its effort in this direction.
Burst Image Restoration and Enhancement
Our central idea is to create a set of pseudo-burst features that combine complementary information from all the input burst frames to seamlessly exchange information.
Ranked #4 on
Burst Image Super-Resolution
on BurstSR
Tensor Pooling Driven Instance Segmentation Framework for Baggage Threat Recognition
Furthermore, to the best of our knowledge, this is the first contour instance segmentation framework that leverages multi-scale information to recognize cluttered and concealed contraband data from the colored and grayscale security X-ray imagery.
Discriminative Region-based Multi-Label Zero-Shot Learning
We note that the best existing multi-label ZSL method takes a shared approach towards attending to region features with a common set of attention maps for all the classes.
Ranked #2 on
Multi-label zero-shot learning
on Open Images V4
Learning Discriminative Representations for Multi-Label Image Recognition
Multi-label recognition is a fundamental, and yet is a challenging task in computer vision.
Unsupervised Anomaly Instance Segmentation for Baggage Threat Recognition
Identifying potential threats concealed within the baggage is of prime concern for the security staff.
On Improving Adversarial Transferability of Vision Transformers
(ii) Token Refinement: We then propose to refine the tokens to further enhance the discriminative capacity at each block of ViT.
Intriguing Properties of Vision Transformers
We show and analyze the following intriguing properties of ViT: (a) Transformers are highly robust to severe occlusions, perturbations and domain shifts, e. g., retain as high as 60% top-1 accuracy on ImageNet even after randomly occluding 80% of the image content.