Search Results for author:
Found 59 papers, 48 papers with code
Count- and Similarity-aware R-CNN for Pedestrian Detection
We further introduce a count-and-similarity branch within the two-stage detection framework, which predicts pedestrian count as well as proposal similarity.
Ranked #17 on
Human Instance Segmentation
on OCHuman
Tracking Meets Large Multimodal Models for Driving Scenario Understanding
We introduce a novel approach for embedding this tracking information into LMMs to enhance their spatiotemporal understanding of driving scenarios.
DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding
Our model achieves a +7. 49% gain in final answer accuracy, along with a 3. 62% improvement in reasoning score over the previous best open-source model.
CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation
Unlike prior methods, our framework minimizes ER usage, with KD preventing forgetting and supporting the IC module in compiling past class statistics to balance learning of rare classes during incremental updates.
Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts
Our goal is to establish AI as a reliable partner in preserving cultural heritage, ensuring that technological advancements contribute meaningfully to historical discovery.
AIN: The Arabic INclusive Large Multimodal Model
While Arabic LLMs have seen notable progress, Arabic LMMs remain largely unexplored, often narrowly focusing on a few specific aspects of the language and visual understanding.
LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
The benchmark presents a diverse set of challenges with eight different categories ranging from complex visual perception to scientific reasoning with over 4k reasoning steps in total, enabling robust evaluation of LLMs' abilities to perform accurate and interpretable visual reasoning across multiple steps.
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
In pursuit of culturally diverse global multimodal models, our proposed All Languages Matter Benchmark (ALM-bench) represents the largest and most comprehensive effort to date for evaluating LMMs across 100 languages.
AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning
In this work, we propose an approach to construct instruction-tuning data that harnesses vision-only data for the agriculture domain.
DB-SAM: Delving into High Quality Universal Medical Image Segmentation
On 21 3D medical image segmentation tasks, our proposed DB-SAM achieves an absolute gain of 8. 8%, compared to a recent medical SAM adapter in the literature.
AgriCLIP: Adapting CLIP for Agriculture and Livestock via Domain-Specialized Cross-Model Alignment
Further, this domain desires fine-grained feature learning due to the subtle nature of the downstream tasks (e. g, nutrient deficiency detection, livestock breed classification).
Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking
To address this limitation, we introduce open-vocabulary 3D tracking, which extends the scope of 3D tracking to include objects beyond predefined categories.
CDChat: A Large Multimodal Model for Remote Sensing Change Description
In order to achieve this, we introduce a change description instruction dataset that can be utilized to finetune an LMM and provide better change descriptions for RS images.
BAPLe: Backdoor Attacks on Medical Foundational Models using Prompt Learning
Inspired by the latest developments in learnable prompts, this work introduces a method to embed a backdoor into the medical foundation model during the prompt learning phase.
BOrg: A Brain Organoid-Based Mitosis Dataset for Automatic Analysis of Brain Diseases
Quantifying cellular processes like mitosis in these organoids offers insights into neurodevelopmental disorders, but the manual analysis is time-consuming, and existing datasets lack specific details for brain organoid studies.
Efficient 3D-Aware Facial Image Editing via Attribute-Specific Prompt Learning
The LAE harnesses a pre-trained vision-language model to find text-guided attribute-specific editing direction in the latent space of any pre-trained 3D-aware GAN.
Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation
To this end, we propose a fast yet accurate open-vocabulary 3D instance segmentation approach, named Open-YOLO 3D, that effectively leverages only 2D object detection from multi-view RGB images for open-vocabulary 3D instance segmentation.
3D Instance Segmentation
3D Open-Vocabulary Instance Segmentation
+4
Multi-modal Generation via Cross-Modal In-Context Learning
Moreover, they often result in misaligned image generation for prompt sequences featuring multiple objects.
Composed Video Retrieval via Enriched Context and Discriminative Embeddings
Composed video retrieval (CoVR) is a challenging problem in computer vision which has recently highlighted the integration of modification text with visual queries for more sophisticated video search in large databases.
Semi-supervised Open-World Object Detection
We demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations.
BiMediX: Bilingual Medical Mixture of Experts LLM
In this paper, we introduce BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic.
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM
To this end, we propose a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability.
SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentation
Specifically, we propose scale-aware attention (SA2) module designed to capture inherent variations in scales and shapes of microscopic regions, such as cells, for accurate segmentation.
3D Indoor Instance Segmentation in an Open-World
We argue that such a closed-world assumption is restrictive and explore for the first time 3D indoor instance segmentation in an open-world setting, where the model is allowed to distinguish a set of known classes as well as identify an unknown object as unknown and then later incrementally learning the semantic category of the unknown when the corresponding category labels are available.
Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation
We decompose the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance, respectively.
A Spatial-Temporal Deformable Attention based Framework for Breast Lesion Detection in Videos
Existing video-based breast lesion detection approaches typically perform temporal feature aggregation of deep backbone features based on the self-attention operation.
Foundational Models Defining a New Era in Vision: A Survey and Outlook
Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world.
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks.
Modulate Your Spectrum in Self-Supervised Learning
In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse.
Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
Then, we use two types of pre-defined tokens to mine co-saliency and background information via our proposed contrast-induced pixel-to-token correlation and co-saliency token-to-token correlation modules.
Ranked #1 on
Co-Salient Object Detection
on CoSal2015
Remote Sensing Change Detection With Transformers Trained from Scratch
Current transformer-based change detection (CD) approaches either employ a pre-trained model trained on large-scale image classification ImageNet dataset or rely on first pre-training on another CD dataset and then fine-tuning on the target benchmark.
Cross-modulated Few-shot Image Generation for Colorectal Tissue Classification
In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues.
Generative Multiplane Neural Radiance for 3D-Aware Image Generation
We present a method to efficiently generate 3D-aware high-resolution images that are view-consistent across multiple target views.
Video Instance Segmentation in an Open-World
Open-world formulation relaxes the close-world static-learning assumption as follows: (a) first, it distinguishes a set of known categories as well as labels an unknown object as `unknown' and then (b) it incrementally learns the class of an unknown as and when the corresponding semantic labels become available.
LEAPS: End-to-End One-Step Person Search With Learnable Proposals
Given a set of sparse and learnable proposals, LEAPS employs a dynamic person search head to directly perform person detection and corresponding re-id feature generation without non-maximum suppression post-processing.
3D Mitochondria Instance Segmentation with Spatio-Temporal Transformers
Accurate 3D mitochondria instance segmentation in electron microscopy (EM) is a challenging problem and serves as a prerequisite to empirically analyze their distributions and morphology.
PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search
Our PS-ARM achieves state-of-the-art performance on both datasets.
An Investigation into Whitening Loss for Self-supervised Learning
A desirable objective in self-supervised learning (SSL) is to avoid feature collapse.
CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection
In this work, we propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework, which takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene at multiple levels to predict a set of object bounding-boxes along with their corresponding semantic labels.
Transformers in Remote Sensing: A Survey
Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade.
Multi-scale Feature Aggregation for Crowd Counting
The SkipAgg module directly propagates features with small receptive fields to features with much larger receptive fields.
3D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field.
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications
Our EdgeNeXt model with 1. 3M parameters achieves 71. 2% top-1 accuracy on ImageNet-1K, outperforming MobileViT with an absolute gain of 2. 2% with 28% reduction in FLOPs.
Ranked #29 on
Semantic Segmentation
on PASCAL VOC 2012 test
PSTR: End-to-End One-Step Person Search With Transformers
We propose a novel one-step transformer-based person search framework, PSTR, that jointly performs person detection and re-identification (re-id) in a single architecture.
Energy-based Latent Aligner for Incremental Learning
Deep learning models tend to forget their earlier knowledge while incrementally learning new tasks.
Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer
When using the ResNet50 backbone, our MS-STS achieves a mask AP of 50. 1 %, outperforming the best reported results in literature by 2. 7 % and by 4. 8 % at higher overlap threshold of AP_75, while being comparable in model size and speed on Youtube-VIS 2019 val.
Spatio-temporal Relation Modeling for Few-shot Action Recognition
Experiments are performed on four few-shot action recognition benchmarks: Kinetics, SSv2, HMDB51 and UCF101.
Ranked #1 on
Few Shot Action Recognition
on UCF101
(using extra training data)
DoodleFormer: Creative Sketch Drawing with Transformers
Creative sketch image generation is a challenging vision problem, where the task is to generate diverse, yet realistic creative sketches possessing the unseen composition of the visual-world objects.
Class-agnostic Object Detection with Multi-modal Transformer
This has been a long-standing question in computer vision.
Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains
The need to address the scarcity of task-specific annotated data has resulted in concerted efforts in recent years for specific settings such as zero-shot learning (ZSL) and domain generalization (DG), to separately address the issues of semantic shift and domain shift, respectively.
Handwriting Transformers
We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns.
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation
In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3. 0% (mask AP) under similar settings, while operating at comparable speed on a Titan Xp.
Ranked #11 on
Real-time Instance Segmentation
on MSCOCO
PSC-Net: Learning Part Spatial Co-occurrence for Occluded Pedestrian Detection
On the heavy occluded (\textbf{HO}) set of CityPerosns test set, our PSC-Net obtains an absolute gain of 4. 0\% in terms of log-average miss rate over the state-of-the-art with same backbone, input scale and without using additional VBB supervision.
Deep Contextual Attention for Human-Object Interaction Detection
Our approach outperforms the state-of-the-art on all datasets.
Mask-Guided Attention Network for Occluded Pedestrian Detection
Our approach obtains an absolute gain of 9. 5% in log-average miss rate, compared to the best reported results on the heavily occluded (HO) pedestrian set of CityPersons test set.
Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification
To the best of our knowledge, we are the first to investigate Binary Patterns encoded CNNs and different deep network fusion architectures for texture recognition and remote sensing scene classification.
Ranked #13 on
Aerial Scene Classification
on AID (20% as trainset)
Scale Coding Bag of Deep Features for Human Attribute and Action Recognition
Most approaches to human attribute and action recognition in still images are based on image representation in which multi-scale local features are pooled across scale into a single, scale-invariant encoding.
