Search Results for author:
Found 45 papers, 25 papers with code
Asymmetric Two-Stream Architecture for Accurate RGB-D Saliency Detection
In this paper, we propose an asymmetric two-stream architecture taking account of the inherent differences between RGB and depth data for saliency detection.
Ranked #19 on
Thermal Image Segmentation
on RGB-T-Glass-Segmentation
Bridging the Gap between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation
Here, we deal with these issues on two aspects: (1) We enhance the prior selection module with the necessary posterior information obtained from the specially designed Posterior Information Prediction Module (PIPM); (2) We propose a Knowledge Distillation Based Training Strategy (KDBTS) to train the decoder with the knowledge selected from the prior distribution, removing the exposure bias of knowledge selection.
MobileVLM V2: Faster and Stronger Baseline for Vision Language Model
We introduce MobileVLM V2, a family of significantly improved vision language models upon MobileVLM, which proves that a delicate orchestration of novel architectural design, an improved training scheme tailored for mobile VLMs, and rich high-quality dataset curation can substantially benefit VLMs' performance.
MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
We present MobileVLM, a competent multimodal vision language model (MMVLM) targeted to run on mobile devices.
ReFusion: Learning Image Fusion from Reconstruction with Learnable Loss via Meta-Learning
To ensure the fusion module maximally preserves the information from the source images, enabling the reconstruction of the source images from the fused image, we adopt a meta-learning strategy to train the loss proposal module using reconstruction loss.
RobustCalib: Robust Lidar-Camera Extrinsic Calibration with Consistency Learning
Current traditional methods for LiDAR-camera extrinsics estimation depend on offline targets and human efforts, while learning-based approaches resort to iterative refinement for calibration results, posing constraints on their generalization and application in on-board systems.
Neural Gradient Regularizer
NGR is applicable to various image types and different image processing tasks, functioning in a zero-shot learning fashion, making it a versatile and plug-and-play regularizer.
A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction
However, these generative methods output a whole sequence consisting of term-status pairs in one stage and ignore integrating prior knowledge, which demands a deeper understanding to model the relationship between terms and infer the status of each term.
Equivariant Multi-Modality Image Fusion
These components enable the net training to follow the principles of the natural sensing-imaging process while satisfying the equivariant imaging prior.
X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages
(3) Integrating multiple modalities: all single-modal encoders are aligned with the LLM through X2L interfaces to integrate multimodal capabilities into the LLM.
Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution
Then, the extracted features are mapped to the spherical space to complete the separation of private features and the alignment of shared features.
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
To leverage strong generative priors and address challenges such as unstable training and lack of interpretability for GAN-based generative methods, we propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM).
Matching-based Term Semantics Pre-training for Spoken Patient Query Understanding
Medical Slot Filling (MSF) task aims to convert medical queries into structured information, playing an essential role in diagnosis dialogue systems.
Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation
Large-scale pre-trained language models (PLMs) have shown great potential in natural language processing tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
We then introduce a dual-branch Transformer-CNN feature extractor with Lite Transformer (LT) blocks leveraging long-range attention to handle low-frequency global features and Invertible Neural Networks (INN) blocks focusing on extracting high-frequency local information.
Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learning
Visual Dialog is a challenging vision-language task since the visual dialog agent needs to answer a series of questions after reasoning over both the image content and dialog history.
Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue
We further develop a new method for supervised contrastive learning, referred to as two-level supervised contrastive learning, and employ the method in response selection in multi-turn dialogue.
Ranked #2 on
Conversational Response Selection
on E-commerce
Counterfactual Supporting Facts Extraction for Explainable Medical Record Based Diagnosis with Graph Network
Specifically, we first structure the sequence of EMR into a hierarchical graph network and then obtain the causal relationship between multi-granularity features and diagnosis results through counterfactual intervention on the graph.
Discrete Cosine Transform Network for Guided Depth Map Super-Resolution
Guided depth super-resolution (GDSR) is an essential topic in multi-modal image processing, which reconstructs high-resolution (HR) depth maps from low-resolution ones collected with suboptimal conditions with the help of HR RGB images of the same scene.
Deep Convolutional Sparse Coding Network for Pansharpening with Guidance of Side Information
Pansharpening is a fundamental issue in remote sensing field.
Deep Gradient Projection Networks for Pan-sharpening
Specifically, two optimization problems regularized by the deep prior are formulated, and they are separately responsible for the generative models for panchromatic images and low resolution multispectral images.
FGF-GAN: A Lightweight Generative Adversarial Network for Pansharpening via Fast Guided Filter
In addition, the latent information of features can be preserved effectively through adversarial training.
DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection
The success of learning-based light field saliency detection is heavily dependent on how a comprehensive dataset can be constructed for higher generalizability of models, how high dimensional light field data can be effectively exploited, and how a flexible model can be designed to achieve versatility for desktop computers and mobile devices.
Towards Reducing Severe Defocus Spread Effects for Multi-Focus Image Fusion via an Optimization Based Strategy
According to the idea that each local region in the fused image should be similar to the sharpest one among source images, this paper presents an optimization-based approach to reduce defocus spread effects.
Knowledge Aware Emotion Recognition in Textual Conversations via Multi-Task Incremental Transformer
For one thing, speakers often rely on the context and commonsense knowledge to express emotions; for another, most utterances contain neutral emotion in conversations, as a result, the confusion between a few non-neutral utterances and much more neutral ones restrains the emotion recognition performance.
Consecutive Decoding for Speech-to-text Translation
The key idea is to generate source transcript and target translation text with a single decoder.
"Listen, Understand and Translate": Triple Supervision Decouples End-to-end Speech-to-text Translation
Can we build a system to fully utilize signals in a parallel ST corpus?
MFIF-GAN: A New Generative Adversarial Network for Multi-Focus Image Fusion
Multi-Focus Image Fusion (MFIF) is a promising image enhancement technique to obtain all-in-focus images meeting visual needs and it is a precondition of other computer vision tasks.
When Image Decomposition Meets Deep Learning: A Novel Infrared and Visible Image Fusion Method
The core idea is that the encoder decomposes an image into base and detail feature maps with low- and high-frequency information, respectively, and that the decoder is responsible for the original image reconstruction.
A Comparison of Label-Synchronous and Frame-Synchronous End-to-End Models for Speech Recognition
End-to-end models are gaining wider attention in the field of automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Deep Convolutional Sparse Coding Networks for Image Fusion
Image fusion is a significant problem in many fields including digital photography, computational imaging and remote sensing, to name but a few.
Infrared And Visible Image Fusion
Multi-Exposure Image Fusion
Bayesian Fusion for Infrared and Visible Images
In this paper, a novel Bayesian fusion model is established for infrared and visible images.
Efficient and Model-Based Infrared and Visible Image Fusion Via Algorithm Unrolling
The proposed AUIF model starts with the iterative formulas of two traditional optimization models, which are established to accomplish two-scale decomposition, i. e., separating low-frequency base information and high-frequency detail information from source images.
Infrared And Visible Image Fusion
Rolling Shutter Correction
DIDFuse: Deep Image Decomposition for Infrared and Visible Image Fusion
Infrared and visible image fusion, a hot topic in the field of image processing, aims at obtaining fused images keeping the advantages of source images.
Ranked #5 on
Semantic Segmentation
on FMB Dataset
MFFW: A new dataset for multi-focus image fusion
It is found that current methods are evaluated on simulated image sets or Lytro dataset.
Inverse Projection Representation and Category Contribution Rate for Robust Tumor Recognition
In this paper, a stable inverse projection representation based classification (IPRC) is presented to tackle these problems by effectively using test samples.
Adaptive Quantile Low-Rank Matrix Factorization
By assuming noise to come from a Gaussian, Laplace or mixture of Gaussian distributions, significant efforts have been made on optimizing the (weighted) $L_1$ or $L_2$-norm loss between an observed matrix and its bilinear factorization.
Variational Bayesian Weighted Complex Network Reconstruction
Complex network reconstruction is a hot topic in many fields.
Semi-Supervised Disfluency Detection
While the disfluency detection has achieved notable success in the past years, it still severely suffers from the data scarcity.
Single-channel Speech Dereverberation via Generative Adversarial Training
In this paper, we propose a single-channel speech dereverberation system (DeReGAT) based on convolutional, bidirectional long short-term memory and deep feed-forward neural network (CBLDNN) with generative adversarial training (GAT).
Multilingual End-to-End Speech Recognition with A Single Transformer on Low-Resource Languages
Experiments on CALLHOME datasets demonstrate that the multilingual ASR Transformer with the language symbol at the end performs better and can obtain relatively 10. 5\% average word error rate (WER) reduction compared to SHL-MLSTM with residual learning.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
A Comparison of Modeling Units in Sequence-to-Sequence Speech Recognition with the Transformer on Mandarin Chinese
Experiments on HKUST datasets demonstrate that the lexicon free modeling units can outperform lexicon related modeling units in terms of character error rate (CER).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Syllable-Based Sequence-to-Sequence Speech Recognition with the Transformer in Mandarin Chinese
Furthermore, we investigate a comparison between syllable based model and context-independent phoneme (CI-phoneme) based model with the Transformer in Mandarin Chinese.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Towards Compact and Fast Neural Machine Translation Using a Combined Method
Neural Machine Translation (NMT) lays intensive burden on computation and memory cost.
