Search Results for author:
Found 22 papers, 5 papers with code
Cross-Modal Few-Shot Learning: a Generative Transfer Learning Framework
Most existing studies on few-shot learning focus on unimodal settings, where models are trained to generalize on unseen data using only a small number of labeled examples from the same modality.
Cross-Modal Denoising: A Novel Training Paradigm for Enhancing Speech-Image Retrieval
To address this issue, we introduce an effective framework and a novel learning task named cross-modal denoising (CMD) to enhance cross-modal interaction to achieve finer-level cross-modal alignment.
Coarse-to-fine Alignment Makes Better Speech-image Retrieval
We utilize speech-image contrastive (SIC) learning tasks to align speech and image representations at a coarse level and speech-image matching (SIM) learning tasks to further refine the fine-grained cross-modal alignment.
Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation
In this work, we introduce a Unified Modality Separation (UniMoS) framework for unsupervised domain adaptation.
Learning Causal Domain-Invariant Temporal Dynamics for Few-Shot Action Recognition
Few-shot action recognition aims at quickly adapting a pre-trained model to the novel data with a distribution shift using only a limited number of samples.
HaltingVT: Adaptive Token Halting Transformer for Efficient Video Recognition
Action recognition in videos poses a challenge due to its high computational cost, especially for Joint Space-Time video transformers (Joint VT).
Learning Socio-Temporal Graphs for Multi-Agent Trajectory Prediction
In order to predict a pedestrian's trajectory in a crowd accurately, one has to take into account her/his underlying socio-temporal interactions with other pedestrians consistently.
Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion
To address this issue, we introduce bio-inspired event cameras and propose a novel Structure-aware Fusion Network (SFNet) that extracts sharp and complete object structures from the event stream to compensate for the lost information in images through cross-modality fusion, enabling the network to obtain illumination-robust representations for traffic object detection.
Boosting Multi-Speaker Expressive Speech Synthesis with Semi-supervised Contrastive Learning
This paper aims to build a multi-speaker expressive TTS system, synthesizing a target speaker's speech with multiple styles and emotions.
HPL-ViT: A Unified Perception Framework for Heterogeneous Parallel LiDARs in V2V
It uses a graph-attention Transformer to extract domain-specific features for each agent, coupled with a cross-attention mechanism for the final fusion.
Differentiable Resolution Compression and Alignment for Efficient Video Classification and Retrieval
To address these issues, we propose an efficient video representation network with Differentiable Resolution Compression and Alignment mechanism, which compresses non-essential information in the early stage of the network to reduce computational costs while maintaining consistent temporal correlations.
Part-Aware Transformer for Generalizable Person Re-identification
Based on the local similarity obtained in CSL, a Part-guided Self-Distillation (PSD) is proposed to further improve the generalization of global features.
Domain Generalization
Generalizable Person Re-identification
Enhancing the Unified Streaming and Non-streaming Model with Contrastive Learning
Specifically, the top-layer hidden representation at the same frame of the streaming and non-streaming modes are regarded as a positive pair, encouraging the representation of the streaming mode close to its non-streaming counterpart.
3D-CSL: self-supervised 3D context similarity learning for Near-Duplicate Video Retrieval
In this paper, we introduce 3D-CSL, a compact pipeline for Near-Duplicate Video Retrieval (NDVR), and explore a novel self-supervised learning strategy for video similarity learning.
Improving CTC-based ASR Models with Gated Interlayer Collaboration
The CTC-based automatic speech recognition (ASR) models without the external language model usually lack the capacity to model conditional dependencies and textual interactions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition
Thus only considering the writing of Chinese characters as modeling units is insufficient to capture speech features.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
BBS-KWS:The Mandarin Keyword Spotting System Won the Video Keyword Wakeup Challenge
We propose a mandarin keyword spotting system (KWS) with several novel and effective improvements, including a big backbone (B) model, a keyword biasing (B) mechanism and the introduction of syllable modeling units (S).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Transformer-based Network for RGB-D Saliency Detection
TFFM conducts a sufficient feature fusion by integrating features from multiple scales and two modalities over all positions simultaneously.
Intervention-based Recurrent Casual Model for Non-stationary Video Causal Discovery
Non-stationary casual structures are prevalent in real-world physical systems.
Synergistic saliency and depth prediction for RGB-D saliency detection
Evaluation on seven RGB-D datasets demonstrates that even without saliency ground truth for RGB-D datasets and using only the RGB data of RGB-D datasets at inference, our semi-supervised system performs favorable against state-of-the-art fully-supervised RGB-D saliency detection methods that use saliency ground truth for RGB-D datasets at training and depth data at inference on two largest testing datasets.
Class-Conditional Domain Adaptation on Semantic Segmentation
We address this problem by introducing a Class-Conditional Domain Adaptation method (CCDA).
Which Way Are You Going? Imitative Decision Learning for Path Forecasting in Dynamic Scenes
A policy is then generated by taking the sampled latent decision into account to predict the future.
