Search Results for author:
Found 43 papers, 7 papers with code
MTPareto: A MultiModal Targeted Pareto Framework for Fake News Detection
Multimodal fake news detection is essential for maintaining the authenticity of Internet multimedia information.
Neural Codec Source Tracing: Toward Comprehensive Attribution in Open-Set Condition
Current research in audio deepfake detection is gradually transitioning from binary classification to multi-class tasks, referred as audio deepfake source tracing task.
LetsTalk: Latent Diffusion Transformer for Talking Video Synthesis
Then we propose a suitable solution according to the modality differences of image, audio, and video generation.
DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speech
In recent years, speech diffusion models have advanced rapidly.
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation
Additionally, the Sound Event Reference Style Transfer Dataset (SERST) is introduced for the proposed target style audio generation task, enabling dual-prompt audio generation using both text and audio references.
Does Current Deepfake Audio Detection Model Effectively Detect ALM-based Deepfake Audio?
Currently, Audio Language Models (ALMs) are rapidly advancing due to the developments in large language models and audio neural codecs.
Temporal Variability and Multi-Viewed Self-Supervised Representations to Tackle the ASVspoof5 Deepfake Challenge
At first, we comprehensively investigate various CM on ASVspoof5, including data expansion, data augmentation, and self-supervised learning (SSL) features.
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic speech recognition (ASR), a cross-modal fine-grained (frame-level) sequence representation is desired, emphasizing the semantic content of the text modality while de-emphasizing the paralinguistic information of the speech modality.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
MDPE: A Multimodal Deception Dataset with Personality and Emotional Characteristics
We believe that MDPE will become a valuable resource for promoting research in the field of affective computing.
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation
The results indicate that the ASRRL method significantly outperforms traditional fine-tuning approaches, achieving higher speaker similarity and better overall speech quality with limited reference speeches.
Fake News Detection and Manipulation Reasoning via Large Vision-Language Models
To support the research, we introduce a benchmark for fake news detection and manipulation reasoning, referred to as Human-centric and Fact-related Fake News (HFFN).
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation
Despite advancements in AIGC technologies for text and image generation, the foley audio dubbing remains rudimentary due to difficulties in cross-modal scene matching and content correlation.
Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audio
To effectively detect LLM-based deepfake audio, we focus on the core of the generation process, the conversion from neural codec to waveform.
Generalized Source Tracing: Detecting Novel Audio Deepfake Algorithm with Real Emphasis and Fake Dispersion Strategy
For effective OOD detection, we first explore current post-hoc OOD methods and propose NSD, a novel OOD approach in identifying novel deepfake algorithms through the similarity consideration of both feature and logits scores.
The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio
With the proliferation of Audio Language Model (ALM) based deepfake audio, there is an urgent need for generalized detection methods.
Learning From Yourself: A Self-Distillation Method for Fake Speech Detection
To address this problem, we propose using the deepest network instruct shallow network for enhancing shallow networks.
UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversion
Text-to-speech (TTS) and voice conversion (VC) are two different tasks both aiming at generating high quality speaking voice according to different input modality.
Emotion Selectable End-to-End Text-based Speech Editing
To achieve this task, we propose Emo-CampNet (emotion CampNet), which can provide the option of emotional attributes for the generated speech in text-based speech editing and has the one-shot ability to edit unseen speakers' speech.
Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Features
Meanwhile, to make full use of the phase and full-band information, we also propose to use real and imaginary spectrogram features as complementary input features and model the disjoint subbands separately.
NeuralDPS: Neural Deterministic Plus Stochastic Model with Multiband Excitation for Noise-Controllable Waveform Generation
We have also verified through experiments that this method can effectively control the noise components in the predicted speech and adjust the SNR of speech.
CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing
It can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript.
ADD 2022: the First Audio Deep Synthesis Detection Challenge
Audio deepfake detection is an emerging topic, which was included in the ASVspoof 2021.
Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis
Firstly, we propose a global duration control attention mechanism for the SVS model.
TSNAT: Two-Step Non-Autoregressvie Transformer Models for Speech Recognition
To address these two problems, we propose a new model named the two-step non-autoregressive transformer(TSNAT), which improves the performance and accelerating the convergence of the NAR model by learning prior knowledge from a parameters-sharing AR model.
Fast End-to-End Speech Recognition via Non-Autoregressive Models and Cross-Modal Knowledge Transferring from BERT
Based on this idea, we propose a non-autoregressive speech recognition model called LASO (Listen Attentively, and Spell Once).
Deep Time Delay Neural Network for Speech Enhancement with Full Data Learning
This paper proposes a deep time delay neural network (TDNN) for speech enhancement with full data learning.
Gated Recurrent Fusion with Joint Training Framework for Robust End-to-End Speech Recognition
The joint training framework for speech enhancement and recognition methods have obtained quite good performances for robust end-to-end automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
One In A Hundred: Select The Best Predicted Sequence from Numerous Candidates for Streaming Speech Recognition
Inspired by the success of two-pass end-to-end models, we introduce a transformer decoder and the two-stage inference method into the streaming CTC model.
Decoupling Pronunciation and Language for End-to-end Code-switching Automatic Speech Recognition
In this paper, we propose a decoupled transformer model to use monolingual paired data and unpaired text data to alleviate the problem of code-switching data shortage.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition
To address this problem and improve the inference speed, we propose a spike-triggered non-autoregressive transformer model for end-to-end speech recognition, which introduces a CTC module to predict the length of the target sequence and accelerate the convergence.
Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition
Without beam-search, the one-pass propagation much reduces inference time cost of LASO.
Simultaneous Denoising and Dereverberation Using Deep Embedding Features
In this paper, we propose a joint training method for simultaneous speech denoising and dereverberation using deep embedding features, which is based on the deep clustering (DC).
Deep Attention Fusion Feature for Speech Separation with End-to-End Post-filter Method
Secondly, to pay more attention to the outputs of the pre-separation stage, an attention module is applied to acquire deep attention fusion features, which are extracted by computing the similarity between the mixture and the pre-separated speech.
Spatial and spectral deep attention fusion for multi-channel speech separation using deep embedding features
Specifically, we apply the deep clustering network to extract deep embedding features.
Synchronous Transformers for End-to-End Speech Recognition
Once a fixed-length chunk of the input sequence is processed by the encoder, the decoder begins to predict symbols immediately.
Integrating Knowledge into End-to-End Speech Recognition from External Text-Only Data
To alleviate the above two issues, we propose a unified method called LST (Learn Spelling from Teachers) to integrate knowledge into an AED model from the external text-only data and leverage the whole context in a sentence.
Self-Attention Transducers for End-to-End Speech Recognition
Furthermore, a path-aware regularization is proposed to assist SA-T to learn alignments and improve the performance.
Discriminative Learning for Monaural Speech Separation Using Deep Embedding Features
Firstly, a DC network is trained to extract deep embedding features, which contain each source's information and have an advantage in discriminating each target speakers.
Forward-Backward Decoding for Regularizing End-to-End TTS
Experimental results show our proposed methods especially the second one (bidirectional decoder regularization), leads a significantly improvement on both robustness and overall naturalness, as outperforming baseline (the revised version of Tacotron2) with a MOS gap of 0. 14 in a challenging test, and achieving close to human quality (4. 42 vs. 4. 49 in MOS) on general test.
Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
Integrating an external language model into a sequence-to-sequence speech recognition system is non-trivial.
Distilling Knowledge Using Parallel Data for Far-field Speech Recognition
The close-talking model is called the teacher model.
Audio Visual Emotion Recognition with Temporal Alignment and Perception Attention
The other one is locating and re-weighting the perception attentions in the whole audio-visual stream for better recognition.
