Search Results for author:
Found 44 papers, 10 papers with code
MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection
This way, our method can eliminate the dependence on RGB modality at every feature granularity and better overcome sensor failures and environmental noises while ensuring the segmentation performance.
A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
We categorize the field into three dimensions: benchmarks, methodologies, and challenges.
Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts
The recent Segment Anything Model (SAM) represents a significant breakthrough in scaling segmentation models, delivering strong performance across various downstream applications in the RGB modality.
Learning Robust Anymodal Segmentor with Unimodal and Cross-modal Distillation
The cross-modal and unimodal distillation is then achieved in the multi scale representation space by transferring the feature level knowledge from multimodal to anymodal segmentors, aiming at addressing the unimodal bias and avoiding over-reliance on specific modalities.
SAVEn-Vid: Synergistic Audio-Visual Integration for Enhanced Understanding in Long Video Context
Endeavors have been made to explore Large Language Models for video analysis (Video-LLMs), particularly in understanding and interpreting long videos.
Explanation-Preserving Augmentation for Semi-Supervised Graph Representation Learning
Graph representation learning (GRL), enhanced by graph augmentation methods, has emerged as an effective technique achieving performance improvements in wide tasks such as node classification and graph classification.
F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI
This approach although efficient suffers the Out-of-Distribution (OOD) problem as the perturbed samples may no longer follow the original data distribution.
EventDance++: Language-guided Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition
This enables the creation of surrogate images to extract knowledge (i. e., labels) from the source model.
GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentation
The `out-of-the-box' insight of GoodSAM++ is to introduce a teacher assistant (TA) to provide semantic information for SAM, integrated with SAM to obtain reliable pseudo semantic maps to bridge both domain and capacity gaps.
Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation
On top, a unified arbitrary-modal selection module is proposed to utilize the aggregated features as the benchmark to rank the multi-modal features based on the similarity scores.
Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalities
Specifically, we first introduce a novel language-guided semantic correlation distillation (LSCD) module to transfer both inter-modal and intra-modal semantic knowledge in the embedding space from MVLMs, e. g., LanguageBind.
EIT-1M: One Million EEG-Image-Text Pairs for Human Visual-textual Recognition and More
To verify the effectiveness of EIT-1M, we provide an in-depth analysis of EEG data captured from multi-modal stimuli across different categories and participants, along with data quality scores for transparency.
OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All
Specifically, we propose Cross-modal Alignment Distillation (CAD) to address the unequal-scale problem between student and teacher modalities and effectively align student modalities into the teacher modalities' representation space in stage one.
TimeX++: Learning Time-Series Explanations with Information Bottleneck
The design of the objective function builds upon the principle of information bottleneck (IB), and modifies the IB objective function to avoid trivial solutions and distributional shift issues.
360SFUDA++: Towards Source-free UDA for Panoramic Segmentation by Learning Reliable Category Prototypes
However, as the distinct projections make it less possible to directly transfer knowledge between domains, we then propose Reliable Panoramic Prototype Adaptation Module (RP2AM) to transfer knowledge at both prediction and prototype levels.
GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentation
To this end, we propose a novel framework, called GoodSAM, that introduces a teacher assistant (TA) to provide semantic information, integrated with SAM to generate ensemble logits to achieve knowledge transfer.
EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition
To this end, we propose a novel framework, dubbed EventDance for this unsupervised source-free cross-modal adaptation problem.
Semantics, Distortion, and Style Matter: Towards Source-free UDA for Panoramic Segmentation
However, the distinct projection discrepancies between source and target domains impede the direct knowledge transfer; thus, we propose a panoramic prototype adaptation module (PPAM) to integrate panoramic prototypes from the extracted knowledge for adaptation.
UniBind: LLM-Augmented Unified and Balanced Representation Space to Bind Them All
To make this possible, we 1) construct a knowledge base of text embeddings with the help of LLMs and multi-modal LLMs; 2) adaptively build LLM-augmented class-wise embedding center on top of the knowledge base and encoded visual embeddings; 3) align all the embeddings to the LLM-augmented embedding center via contrastive learning to achieve a unified and balanced representation space.
ExACT: Language-guided Conceptual Reasoning and Uncertainty Estimation for Event-based Action Recognition and More
Then, we propose a conceptual reasoning-based uncertainty estimation module, which simulates the recognition process to enrich the semantic representation.
BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis
However, a hurdle of existing methods is the low efficiency, per-prompt optimization for a single 3D object.
Parametric Augmentation for Time Series Contrastive Learning
In this study, we address this gap by analyzing time series data augmentation using information theory and summarizing the most commonly adopted augmentations in a unified format.
PAC Learnability under Explanation-Preserving Graph Perturbations
It is shown that the sample complexity of explanation-assisted learning can be arbitrarily smaller than explanation-agnostic learning.
Image Anything: Towards Reasoning-coherent and Training-free Multi-modal Image Generation
It extracts entity features from the multi-modal representations powered by our specially constructed entity knowledge graph; 2) Attribute Fusion Branch adeptly preserves and processes the attributes.
Semantics Distortion and Style Matter: Towards Source-free UDA for Panoramic Segmentation
However the distinct projection discrepancies between source and target domains impede the direct knowledge transfer; thus we propose a panoramic prototype adaptation module (PPAM) to integrate panoramic prototypes from the extracted knowledge for adaptation.
Distilling Efficient Vision Transformers from CNNs for Semantic Segmentation
Due to the completely different characteristics of ViT and CNN and the long-existing capacity gap between teacher and student models in Knowledge Distillation (KD), directly transferring the cross-model knowledge is non-trivial.
CLIP Is Also a Good Teacher: A New Learning Framework for Inductive Zero-shot Semantic Segmentation
Moreover, to enhance the ability to discriminate unseen categories, PLM consisting of pseudo labels and weight generation is designed.
Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks
An explanation function for GNNs takes a pre-trained GNN along with a graph as input, to produce a `sufficient statistic' subgraph with respect to the graph label.
Chasing Day and Night: Towards Robust and Efficient All-Day Object Detection Guided by an Event Camera
The ability to detect objects in all lighting (i. e., normal-, over-, and under-exposed) conditions is crucial for real-world applications, such as self-driving. Traditional RGB-based detectors often fail under such varying lighting conditions. Therefore, recent works utilize novel event cameras to supplement or guide the RGB modality; however, these methods typically adopt asymmetric network structures that rely predominantly on the RGB modality, resulting in limited robustness for all-day detection.
Look at the Neighbor: Distortion-aware Unsupervised Domain Adaptation for Panoramic Semantic Segmentation
The aim is to tackle the domain gaps caused by the style disparities and distortion problem from the non-uniformly distributed pixels of equirectangular projection (ERP).
EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding
Accordingly, we first introduce a novel event encoder that subtly models the temporal information from events and meanwhile, generates event prompts for modality bridging.
A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation
In this paper, we strive to answer the question "how to collaboratively learn convolutional neural network (CNN)-based and vision transformer (ViT)-based models by selecting and exchanging the reliable knowledge between them for semantic segmentation?"
Both Style and Distortion Matter: Dual-Path Unsupervised Domain Adaptation for Panoramic Semantic Segmentation
Moreover, adversarial intra-projection training is proposed to reduce the inherent gap, between the features of the pinhole images and those of the ERP and TP images, respectively.
Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks
Event cameras are bio-inspired sensors that capture the per-pixel intensity changes asynchronously and produce event streams encoding the time, pixel position, and polarity (sign) of the intensity changes.
Evaluation of the Synthetic Electronic Health Records
Generative models have been found effective for data synthesis due to their ability to capture complex underlying data distributions.
Transformer-CNN Cohort: Semi-supervised Semantic Segmentation by the Best of Both Students
To this end, we propose class-aware feature consistency distillation (CFCD) that first leverages the outputs of each student as the pseudo labels and generates class-aware feature (CF) maps for knowledge transfer between the two students.
Priors in Deep Image Restoration and Enhancement: A Survey
Image restoration and enhancement is a process of improving the image quality by removing degradations, such as noise, blur, and resolution degradation.
Synthesising Electronic Health Records: Cystic Fibrosis Patient Group
Deep generative models are effective data synthesisers due to their ability to capture complex underlying distributions.
Uncertainty-Aware Deep Co-training for Semi-supervised Medical Image Segmentation
However, due to the scarcity of labeled data, the features extracted by the models are limited in supervised learning, and the quality of predictions for unlabeled data also cannot be guaranteed.
A Graph-based Imputation Method for Sparse Medical Records
Our work can facilitate the diagnosis of novel diseases based on the clinical history of past events, with the potential to increase our understanding of the landscape of comorbidities.
Network Generation with Differential Privacy
Differential privacy is a gold standard for data privacy, and the introduction of the differentially private stochastic gradient descent (DP-SGD) algorithm has facilitated the training of private neural models in a number of domains.
Fröhlich Condensate of Phonons in Optomechanical Systems
We propose that the optomechanical systems can be potential platforms to implement the Fr\"{o}hlich condensate of phonons.
Quantum Physics Mesoscale and Nanoscale Physics
STaDA: Style Transfer as Data Augmentation
The success of training deep Convolutional Neural Networks (CNNs) heavily depends on a significant amount of labelled data.
Beyond Keywords and Relevance: A Personalized Ad Retrieval Framework in E-Commerce Sponsored Search
Given a search request, ad retrieval module rewrites the query into bidding keywords, and uses these keywords as keys to select Top N ads through inverted indexes.
