Search Results for author:
Found 21 papers, 13 papers with code
Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation
Furthermore, we also visualize the process of explicit cooperation and surprisingly find that each LoRA head has certain audio-visual understanding ability.
Adaptive Prompt Learning with SAM for Few-shot Scanning Probe Microscope Image Segmentation
To address these challenges, we propose an Adaptive Prompt Learning with SAM (APL-SAM) framework tailored for few-shot SPM image segmentation.
Multi-weather Cross-view Geo-localization Using Denoising Diffusion Models
For image restoration, MCGF incorporates a shared encoder and a lightweight restoration module to help the backbone eliminate weather-specific information.
Boosting Audio Visual Question Answering via Key Semantic-Aware Cues
The Audio Visual Question Answering (AVQA) task aims to answer questions related to various visual objects, sounds, and their interactions in videos.
Audio-visual Question Answering
Audio-Visual Question Answering (AVQA)
+2
Ref-AVS: Refer and Segment Objects in Audio-Visual Scenes
In this work, we introduce a novel task called Reference Audio-Visual Segmentation (Ref-AVS), which seeks to segment objects within the visual domain based on expressions containing multimodal cues.
Audio-Visual Instance Segmentation
In this paper, we propose a new multi-modal task, termed audio-visual instance segmentation (AVIS), which aims to simultaneously identify, segment and track individual sounding object instances in audible videos.
CM-PIE: Cross-modal perception for interactive-enhanced audio-visual video parsing
Audio-visual video parsing is the task of categorizing a video at the segment level with weak labels, and predicting them as audible or visible events.
Progressive Spatio-temporal Perception for Audio-Visual Question Answering
Such naturally multi-modal videos are composed of rich and complex dynamic audio-visual components, where most of which could be unrelated to the given questions, or even play as interference in answering the content of interest.
Ranked #2 on
Audio-Visual Question Answering (AVQA)
on AVQA
Audio-visual Question Answering
Audio-Visual Question Answering (AVQA)
+2
Multi-Scale Attention for Audio Question Answering
Audio question answering (AQA), acting as a widely used proxy task to explore scene understanding, has got more attention.
MECPformer: Multi-estimations Complementary Patch with CNN-Transformers for Weakly Supervised Semantic Segmentation
Given a class, the initial seeds generated based on the transformer may invade regions belonging to other classes.
Weakly supervised Semantic Segmentation
Weakly-Supervised Semantic Segmentation
Heterogeneous Federated Knowledge Graph Embedding Learning and Unlearning
To cope with the drift between local optimization and global convergence caused by data heterogeneity, we propose mutual knowledge distillation to transfer local knowledge to global, and absorb global knowledge back.
EventEA: Benchmarking Entity Alignment for Event-centric Knowledge Graphs
Towards a critical evaluation of embedding-based entity alignment methods, we construct a new dataset with heterogeneous relations and attributes based on event-centric KGs.
I Know What You Do Not Know: Knowledge Graph Embedding via Co-distillation Learning
To this end, we propose CoLE, a Co-distillation Learning method for KG Embedding that exploits the complementarity of graph structures and text information.
Learning to Answer Questions in Dynamic Audio-Visual Scenarios
In this paper, we focus on the Audio-Visual Question Answering (AVQA) task, which aims to answer questions regarding different visual objects, sounds, and their associations in videos.
audio-visual learning
Audio-Visual Question Answering (AVQA)
+4
WegFormer: Transformers for Weakly Supervised Semantic Segmentation
Although convolutional neural networks (CNNs) have achieved remarkable progress in weakly supervised semantic segmentation (WSSS), the effective receptive field of CNN is insufficient to capture global context information, leading to sub-optimal results.
Self-supervised Audiovisual Representation Learning for Remote Sensing Data
By fine-tuning the models on a number of commonly used remote sensing datasets, we show that our approach outperforms existing pre-training strategies for remote sensing imagery.
Ranked #2 on
Cross-Modal Retrieval
on SoundingEarth
Rule-Guided Graph Neural Networks for Recommender Systems
However, existing work incorporated with KGs cannot capture the explicit long-range semantics between users and items meanwhile consider various connectivity between items.
Deep Inception Generative Network for Cognitive Image Inpainting
Recent advances in deep learning have shown exciting promise in filling large holes and lead to another orientation for image inpainting.
Scene Text Detection with Supervised Pyramid Context Network
We propose a supervised pyramid context network (SPCNET) to precisely locate text regions while suppressing false positives.
Ranked #2 on
Scene Text Detection
on ICDAR 2013
Real-world plant species identification based on deep convolutional neural networks and visual attention
We propose a novel framework and an effective data augmentation method for deep learning in this paper.
