Search Results for author:
Found 70 papers, 54 papers with code
Fixing Localization Errors to Improve Image Classification
This way, in an effort to fix localization errors, our loss provides an extra supervisory signal that helps the model to better discriminate between similar classes.
Count- and Similarity-aware R-CNN for Pedestrian Detection
We further introduce a count-and-similarity branch within the two-stage detection framework, which predicts pedestrian count as well as proposal similarity.
Ranked #17 on
Human Instance Segmentation
on OCHuman
Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions
In this work, we introduce MedAgentSim, an open-source simulated clinical environment with doctor, patient, and measurement agents designed to evaluate and enhance LLM performance in dynamic diagnostic settings.
Tracking Meets Large Multimodal Models for Driving Scenario Understanding
We introduce a novel approach for embedding this tracking information into LMMs to enhance their spatiotemporal understanding of driving scenarios.
DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understanding
Our model achieves a +7. 49% gain in final answer accuracy, along with a 3. 62% improvement in reasoning score over the previous best open-source model.
LLM Post-Training: A Deep Dive into Reasoning Large Language Models
Large Language Models (LLMs) have transformed the natural language processing landscape and brought to life diverse applications.
CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation
Unlike prior methods, our framework minimizes ER usage, with KD preventing forgetting and supporting the IC module in compiling past class statistics to balance learning of rare classes during incremental updates.
Time Travel: A Comprehensive Benchmark to Evaluate LMMs on Historical and Cultural Artifacts
Our goal is to establish AI as a reliable partner in preserving cultural heritage, ensuring that technological advancements contribute meaningfully to historical discovery.
AIN: The Arabic INclusive Large Multimodal Model
While Arabic LLMs have seen notable progress, Arabic LMMs remain largely unexplored, often narrowly focusing on a few specific aspects of the language and visual understanding.
LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs
The benchmark presents a diverse set of challenges with eight different categories ranging from complex visual perception to scientific reasoning with over 4k reasoning steps in total, enabling robust evaluation of LLMs' abilities to perform accurate and interpretable visual reasoning across multiple steps.
BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities
This paper introduces BiMediX2, a bilingual (Arabic-English) Bio-Medical EXpert Large Multimodal Model (LMM) with a unified architecture that integrates text and visual modalities, enabling advanced image understanding and medical applications.
CLIP meets DINO for Tuning Zero-Shot Classifier using Unlabeled Image Collections
This three-step process allows us to harness the best of visual and textual foundation models, resulting in a powerful and efficient approach that surpasses state-of-the-art label-free classification methods.
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages
In pursuit of culturally diverse global multimodal models, our proposed All Languages Matter Benchmark (ALM-bench) represents the largest and most comprehensive effort to date for evaluating LMMs across 100 languages.
Improving 3D Medical Image Segmentation at Boundary Regions using Local Self-attention and Global Volume Mixing
In this work, we propose a novel hierarchical encoder-decoder-based framework that strives to explicitly capture the local and global dependencies for volumetric 3D medical image segmentation.
AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning
In this work, we propose an approach to construct instruction-tuning data that harnesses vision-only data for the agriculture domain.
Open3DTrack: Towards Open-Vocabulary 3D Multi-Object Tracking
To address this limitation, we introduce open-vocabulary 3D tracking, which extends the scope of 3D tracking to include objects beyond predefined categories.
CDChat: A Large Multimodal Model for Remote Sensing Change Description
In order to achieve this, we introduce a change description instruction dataset that can be utilized to finetune an LMM and provide better change descriptions for RS images.
CONDA: Condensed Deep Association Learning for Co-Salient Object Detection
They directly rely on raw associations which are not reliable in complex scenarios, and their image feature optimization approach is not explicit for inter-image association modeling.
Medical Image Segmentation Using Directional Window Attention
The focus of our design is the introduction of Dwin block within DwinFormer that effectively captures local and global information along the horizontal, vertical, and depthwise directions of the input feature map by separately performing attention in each of these directional volumes.
Efficient 3D-Aware Facial Image Editing via Attribute-Specific Prompt Learning
The LAE harnesses a pre-trained vision-language model to find text-guided attribute-specific editing direction in the latent space of any pre-trained 3D-aware GAN.
Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentation
To this end, we propose a fast yet accurate open-vocabulary 3D instance segmentation approach, named Open-YOLO 3D, that effectively leverages only 2D object detection from multi-view RGB images for open-vocabulary 3D instance segmentation.
3D Instance Segmentation
3D Open-Vocabulary Instance Segmentation
+4
Multi-modal Generation via Cross-Modal In-Context Learning
Moreover, they often result in misaligned image generation for prompt sequences featuring multiple objects.
ChangeBind: A Hybrid Change Encoder for Remote Sensing Change Detection
Change detection (CD) is a fundamental task in remote sensing (RS) which aims to detect the semantic changes between the same geographical regions at different time stamps.
PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Model
We introduce a novel segmentation task known as reasoning part segmentation for 3D objects, aiming to output a segmentation mask based on complex and implicit textual queries about specific parts of a 3D object.
ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detection
Deep learning has shown remarkable success in remote sensing change detection (CD), aiming to identify semantic change regions between co-registered satellite image pairs acquired at distinct time stamps.
Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery
Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks by pre-training on large amount of unlabelled data.
Semi-supervised Open-World Object Detection
We demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations.
BiMediX: Bilingual Medical Mixture of Experts LLM
In this paper, we introduce BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic.
Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM
To this end, we propose a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability.
GLaMM: Pixel Grounding Large Multimodal Model
In this work, we present Grounding LMM (GLaMM), the first model that can generate natural language responses seamlessly intertwined with corresponding object segmentation masks.
DDAM-PS: Diligent Domain Adaptive Mixer for Person Search
The objective of the two bridge losses is to guide the moderate mixed-domain representations to maintain an appropriate distance from both the source and target domain representations.
TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation
Scene understanding plays an essential role in enabling autonomous driving and maintaining high standards of performance and safety.
SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentation
Specifically, we propose scale-aware attention (SA2) module designed to capture inherent variations in scales and shapes of microscopic regions, such as cells, for accurate segmentation.
3D Indoor Instance Segmentation in an Open-World
We argue that such a closed-world assumption is restrictive and explore for the first time 3D indoor instance segmentation in an open-world setting, where the model is allowed to distinguish a set of known classes as well as identify an unknown object as unknown and then later incrementally learning the semantic category of the unknown when the corresponding category labels are available.
Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation
We decompose the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance, respectively.
Foundational Models Defining a New Era in Vision: A Survey and Outlook
Vision systems to see and reason about the compositional nature of visual scenes are fundamental to understanding our world.
XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models
The latest breakthroughs in large vision-language models, such as Bard and GPT-4, have showcased extraordinary abilities in performing a wide range of tasks.
Salient Mask-Guided Vision Transformer for Fine-Grained Classification
Fine-grained visual classification (FGVC) is a challenging computer vision problem, where the task is to automatically recognise objects from subordinate categories.
Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
Then, we use two types of pre-defined tokens to mine co-saliency and background information via our proposed contrast-induced pixel-to-token correlation and co-saliency token-to-token correlation modules.
Ranked #1 on
Co-Salient Object Detection
on CoSal2015
RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model
This improvement was associated with the increasing use of LiDAR sensors and point cloud data to facilitate the task of object detection and recognition in autonomous driving.
Remote Sensing Change Detection With Transformers Trained from Scratch
Current transformer-based change detection (CD) approaches either employ a pre-trained model trained on large-scale image classification ImageNet dataset or rely on first pre-training on another CD dataset and then fine-tuning on the target benchmark.
Cross-modulated Few-shot Image Generation for Colorectal Tissue Classification
In this work, we propose a few-shot colorectal tissue image generation method for addressing the scarcity of histopathological training data for rare cancer tissues.
Generative Multiplane Neural Radiance for 3D-Aware Image Generation
We present a method to efficiently generate 3D-aware high-resolution images that are view-consistent across multiple target views.
Video Instance Segmentation in an Open-World
Open-world formulation relaxes the close-world static-learning assumption as follows: (a) first, it distinguishes a set of known categories as well as labels an unknown object as `unknown' and then (b) it incrementally learns the class of an unknown as and when the corresponding semantic labels become available.
PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search
Our PS-ARM achieves state-of-the-art performance on both datasets.
CMR3D: Contextualized Multi-Stage Refinement for 3D Object Detection
In this work, we propose Contextualized Multi-Stage Refinement for 3D Object Detection (CMR3D) framework, which takes a 3D scene as input and strives to explicitly integrate useful contextual information of the scene at multiple levels to predict a set of object bounding-boxes along with their corresponding semantic labels.
Transformers in Remote Sensing: A Survey
Deep learning-based algorithms have seen a massive popularity in different areas of remote sensing image analysis over the past decade.
AVisT: A Benchmark for Visual Object Tracking in Adverse Visibility
While being greatly benefiting to the tracking research, existing benchmarks do not pose the same difficulty as before with recent trackers achieving higher performance mainly due to (i) the introduction of more sophisticated transformers-based methods and (ii) the lack of diverse scenarios with adverse visibility such as, severe weather conditions, camouflage and imaging effects.
Multi-scale Feature Aggregation for Crowd Counting
The SkipAgg module directly propagates features with small receptive fields to features with much larger receptive fields.
3D Vision with Transformers: A Survey
The success of the transformer architecture in natural language processing has recently triggered attention in the computer vision field.
On the Robustness of 3D Object Detectors
To the best of our knowledge, we are the first to investigate the robustness of point-based 3D object detectors.
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications
Our EdgeNeXt model with 1. 3M parameters achieves 71. 2% top-1 accuracy on ImageNet-1K, outperforming MobileViT with an absolute gain of 2. 2% with 28% reduction in FLOPs.
Ranked #29 on
Semantic Segmentation
on PASCAL VOC 2012 test
PSTR: End-to-End One-Step Person Search With Transformers
We propose a novel one-step transformer-based person search framework, PSTR, that jointly performs person detection and re-identification (re-id) in a single architecture.
Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer
When using the ResNet50 backbone, our MS-STS achieves a mask AP of 50. 1 %, outperforming the best reported results in literature by 2. 7 % and by 4. 8 % at higher overlap threshold of AP_75, while being comparable in model size and speed on Youtube-VIS 2019 val.
DoodleFormer: Creative Sketch Drawing with Transformers
Creative sketch image generation is a challenging vision problem, where the task is to generate diverse, yet realistic creative sketches possessing the unseen composition of the visual-world objects.
Structured Latent Embeddings for Recognizing Unseen Classes in Unseen Domains
The need to address the scarcity of task-specific annotated data has resulted in concerted efforts in recent years for specific settings such as zero-shot learning (ZSL) and domain generalization (DG), to separately address the issues of semantic shift and domain shift, respectively.
Handwriting Transformers
We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns.
D2-Net: Weakly-Supervised Action Localization via Discriminative Embeddings and Denoised Activations
The proposed formulation comprises a discriminative and a denoising loss term for enhancing temporal action localization.
Ranked #3 on
Weakly Supervised Action Localization
on FineAction
SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation
In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3. 0% (mask AP) under similar settings, while operating at comparable speed on a Titan Xp.
Ranked #11 on
Real-time Instance Segmentation
on MSCOCO
PSC-Net: Learning Part Spatial Co-occurrence for Occluded Pedestrian Detection
On the heavy occluded (\textbf{HO}) set of CityPerosns test set, our PSC-Net obtains an absolute gain of 4. 0\% in terms of log-average miss rate over the state-of-the-art with same backbone, input scale and without using additional VBB supervision.
Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes
The main requisite for fine-grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other.
Ranked #20 on
Fine-Grained Image Classification
on Stanford Dogs
Towards Partial Supervision for Generic Object Counting in Natural Scenes
Our RLC framework further reduces the annotation cost arising from large numbers of object categories in a dataset by only using lower-count supervision for a subset of categories and class-labels for the remaining ones.
Image Classification
Image-level Supervised Instance Segmentation
+3
3C-Net: Category Count and Center Loss for Weakly-Supervised Action Localization
Our joint formulation has three terms: a classification term to ensure the separability of learned action features, an adapted multi-label center loss term to enhance the action feature discriminability and a counting loss term to delineate adjacent action sequences, leading to improved localization.
Ranked #1 on
Action Classification
on THUMOS'14
Action Classification
Weakly Supervised Action Localization
+2
Object Counting and Instance Segmentation with Image-level Supervision
Moreover, our approach improves state-of-the-art image-level supervised instance segmentation with a relative gain of 17. 8% in terms of average best overlap, on the PASCAL VOC 2012 dataset.
Ranked #1 on
Object Counting
on COCO count-test
L1-regularized Reconstruction Error as Alpha Matte
Sampling-based alpha matting methods have traditionally followed the compositing equation to estimate the alpha value at a pixel from a pair of foreground (F) and background (B) samples.
Backtracking Spatial Pyramid Pooling (SPP)-based Image Classifier for Weakly Supervised Top-down Salient Object Detection
First, the probabilistic contribution of each image region to the confidence of a CNN-based image classifier is computed through a backtracking strategy to produce top-down saliency.
Backtracking ScSPM Image Classifier for Weakly Supervised Top-Down Saliency
We propose a weakly supervised top-down saliency framework using only binary labels that indicate the presence/absence of an object in an image.
A Classifier-guided Approach for Top-down Salient Object Detection
Although the role of the classifier is to support salient object detection, we evaluate its performance in image classification and also illustrate the utility of thresholded saliency maps for image segmentation.
Sparse Coding for Alpha Matting
In this paper, the matting problem is reinterpreted as a sparse coding of pixel features, wherein the sum of the codes gives the estimate of the alpha matte from a set of unpaired F and B samples.
