Search Results for author:
Found 30 papers, 6 papers with code
LetsTalk: Latent Diffusion Transformer for Talking Video Synthesis
Then we propose a suitable solution according to the modality differences of image, audio, and video generation.
DPI-TTS: Directional Patch Interaction for Fast-Converging and Style Temporal Modeling in Text-to-Speech
In recent years, speech diffusion models have advanced rapidly.
Towards Diverse and Efficient Audio Captioning via Diffusion Models
We introduce Diffusion-based Audio Captioning (DAC), a non-autoregressive diffusion model tailored for diverse and efficient audio captioning.
Text Prompt is Not Enough: Sound Event Enhanced Prompt Adapter for Target Style Audio Generation
Additionally, the Sound Event Reference Style Transfer Dataset (SERST) is introduced for the proposed target style audio generation task, enabling dual-prompt audio generation using both text and audio references.
Does Current Deepfake Audio Detection Model Effectively Detect ALM-based Deepfake Audio?
Currently, Audio Language Models (ALMs) are rapidly advancing due to the developments in large language models and audio neural codecs.
Temporal Variability and Multi-Viewed Self-Supervised Representations to Tackle the ASVspoof5 Deepfake Challenge
At first, we comprehensively investigate various CM on ASVspoof5, including data expansion, data augmentation, and self-supervised learning (SSL) features.
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing
For tasks such as text-to-speech (TTS), voice conversion (VC), and automatic speech recognition (ASR), a cross-modal fine-grained (frame-level) sequence representation is desired, emphasizing the semantic content of the text modality while de-emphasizing the paralinguistic information of the speech modality.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
MDPE: A Multimodal Deception Dataset with Personality and Emotional Characteristics
We believe that MDPE will become a valuable resource for promoting research in the field of affective computing.
ASRRL-TTS: Agile Speaker Representation Reinforcement Learning for Text-to-Speech Speaker Adaptation
The results indicate that the ASRRL method significantly outperforms traditional fine-tuning approaches, achieving higher speaker similarity and better overall speech quality with limited reference speeches.
Fake News Detection and Manipulation Reasoning via Large Vision-Language Models
To support the research, we introduce a benchmark for fake news detection and manipulation reasoning, referred to as Human-centric and Fact-related Fake News (HFFN).
A multi-speaker multi-lingual voice cloning system based on vits2 for limmits 2024 challenge
The objective of the challenge is to establish a multi-speaker, multi-lingual Indic Text-to-Speech system with voice cloning capabilities, covering seven Indian languages with both male and female speakers.
MINT: a Multi-modal Image and Narrative Text Dubbing Dataset for Foley Audio Content Planning and Generation
Despite advancements in AIGC technologies for text and image generation, the foley audio dubbing remains rudimentary due to difficulties in cross-modal scene matching and content correlation.
Codecfake: An Initial Dataset for Detecting LLM-based Deepfake Audio
To effectively detect LLM-based deepfake audio, we focus on the core of the generation process, the conversion from neural codec to waveform.
Generalized Source Tracing: Detecting Novel Audio Deepfake Algorithm with Real Emphasis and Fake Dispersion Strategy
For effective OOD detection, we first explore current post-hoc OOD methods and propose NSD, a novel OOD approach in identifying novel deepfake algorithms through the similarity consideration of both feature and logits scores.
The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio
With the proliferation of Audio Language Model (ALM) based deepfake audio, there is an urgent need for generalized detection methods.
Learning Speech Representation From Contrastive Token-Acoustic Pretraining
However, existing contrastive learning methods in the audio field focus on extracting global descriptive information for downstream audio classification tasks, making them unsuitable for TTS, VC, and ASR tasks.
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding
However, existing methods suffer from three problems: the high dimensionality and waveform distortion of discrete speech representations, the prosodic averaging problem caused by the duration prediction model in non-autoregressive frameworks, and the information redundancy and dimension explosion problems of existing semantic encoding methods.
Low-rank Adaptation Method for Wav2vec2-based Fake Audio Detection
Self-supervised speech models are a rapidly developing research topic in fake audio detection.
Adaptive Fake Audio Detection with Low-Rank Model Squeezing
During the inference stage, these adaptation matrices are combined with the existing model to generate the final prediction output.
UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversion
Text-to-speech (TTS) and voice conversion (VC) are two different tasks both aiming at generating high quality speaking voice according to different input modality.
Emotion Selectable End-to-End Text-based Speech Editing
To achieve this task, we propose Emo-CampNet (emotion CampNet), which can provide the option of emotional attributes for the generated speech in text-based speech editing and has the one-shot ability to edit unseen speakers' speech.
SceneFake: An Initial Dataset and Benchmarks for Scene Fake Audio Detection
Some scene fake audio detection benchmark results on the SceneFake dataset are reported in this paper.
An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
Speech is the fundamental mode of human communication, and its synthesis has long been a core priority in human-computer interaction research.
Fully Automated End-to-End Fake Audio Detection
The existing fake audio detection systems often rely on expert experience to design the acoustic features or manually design the hyperparameters of the network structure.
An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio
Many effective attempts have been made for fake audio detection.
NeuralDPS: Neural Deterministic Plus Stochastic Model with Multiband Excitation for Noise-Controllable Waveform Generation
We have also verified through experiments that this method can effectively control the noise components in the predicted speech and adjust the SNR of speech.
CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing
It can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript.
ADD 2022: the First Audio Deep Synthesis Detection Challenge
Audio deepfake detection is an emerging topic, which was included in the ASVspoof 2021.
Singing-Tacotron: Global duration control attention and dynamic filter for End-to-end singing voice synthesis
Firstly, we propose a global duration control attention mechanism for the SVS model.
Half-Truth: A Partially Fake Audio Detection Dataset
Therefore, this paper develops such a dataset for half-truth audio detection (HAD).
