Search Results for author:
Found 61 papers, 25 papers with code
Unsupervised Multi-scale Expressive Speaking Style Modeling with Hierarchical Context Information for Audiobook Speech Synthesis
On top of these, a bi-reference attention mechanism is used to align both local-scale reference style embedding sequence and local-scale context style embedding sequence with corresponding phoneme embedding sequence.
Can We Edit Factual Knowledge by In-Context Learning?
Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge.
QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation
Levenshtein distance based on audio quantization as a similarity metric of corresponding speech of gestures helps match more appropriate gestures with speech, and solves the alignment problem of speech and gestures well.
ZeroPrompt: Streaming Acoustic Encoders are Zero-Shot Masked LMs
In this paper, we present ZeroPrompt (Figure 1-(a)) and the corresponding Prompt-and-Refine strategy (Figure 3), two simple but effective \textbf{training-free} methods to decrease the Token Display Time (TDT) of streaming ASR models \textbf{without any accuracy loss}.
Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation Based Voice Conversion
Experimental results show that our proposed method achieves comparable similarity and higher naturalness than the supervised method, which needs a huge amount of annotated corpora for training and is applicable to improve similarity for VC methods with other SSL representations as input.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
CB-Conformer: Contextual biasing Conformer for biased word recognition
In this work, we propose CB-Conformer to improve biased word recognition by introducing the Contextual Biasing Module and the Self-Adaptive Language Model to vanilla Conformer.
OpenICL: An Open-Source Framework for In-context Learning
However, the implementation of ICL is sophisticated due to the diverse retrieval and inference methods involved, as well as the varying pre-processing requirements for different models, datasets, and tasks.
Compositional Exemplars for In-context Learning
The performance of ICL is highly dominated by the quality of the selected in-context examples.
In-Context Learning with Many Demonstration Examples
Based on EVALM, we scale up the size of examples efficiently in both instruction tuning and in-context learning to explore the boundary of the benefits from more annotated data.
A Survey on In-context Learning
With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples.
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering
Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context.
Explanation Regeneration via Information Bottleneck
Explaining the black-box predictions of NLP models naturally and accurately is an important open problem in natural language generation.
Lexicon-injected Semantic Parsing for Task-Oriented Dialog
Then we present a novel lexicon-injected semantic parser, which collects slot labels of tree representation as a lexicon, and injects lexical features to the span representation of parser.
Unsupervised Explanation Generation via Correct Instantiations
The major difficulty is finding the conflict point, where the statement contradicts our real world.
TrimTail: Low-Latency Streaming ASR with Simple but Effective Spectrogram-Level Length Penalty
In this paper, we present TrimTail, a simple but effective emission regularization method to improve the latency of streaming ASR models.
Disentangled Speech Representation Learning for One-Shot Cross-lingual Voice Conversion Using $β$-VAE
We propose an unsupervised learning method to disentangle speech into content representation and speaker identity representation.
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
To improve the quality of dataset synthesis, we propose a progressive zero-shot dataset generation framework, ProGen, which leverages the feedback from the task-specific model to guide the generation of new training data via in-context examples.
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks.
COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization
Traditional training paradigms for extractive and abstractive summarization systems always only use token-level or sentence-level training objectives.
The ReprGesture entry to the GENEA Challenge 2022
This paper describes the ReprGesture entry to the Generation and Evaluation of Non-verbal Behaviour for Embodied Agents (GENEA) challenge 2022.
Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversion
One-shot voice conversion (VC) with only a single target speaker's speech for reference has become a hot research topic.
Towards Cross-speaker Reading Style Transfer on Audiobook Dataset
This paper aims to introduce a chunk-wise multi-scale cross-speaker style model to capture both the global genre and the local prosody in audiobook speeches.
Ordinal Regression via Binary Preference vs Simple Regression: Statistical and Experimental Perspectives
The ORARS addresses the MOS prediction problem by pairing a test sample with each of the pre-scored anchored reference samples.
Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
However, previous works show that state-of-the-art ASV models are seriously vulnerable to voice spoofing attacks, and the recently proposed high-performance spoofing countermeasure (CM) models only focus solely on the standalone anti-spoofing tasks, and ignore the subsequent speaker verification process.
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
In this paradigm, the synthesized data from the PLM acts as the carrier of knowledge, which is used to train a task-specific model with orders of magnitude fewer parameters than the PLM, achieving both higher performance and efficiency than prompt-based zero-shot learning methods on PLMs.
Lexical Knowledge Internalization for Neural Dialog Generation
We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models.
A Character-level Span-based Model for Mandarin Prosodic Structure Prediction
In this paper, we propose a span-based Mandarin prosodic structure prediction model to obtain an optimal prosodic structure tree, which can be converted to corresponding prosodic label sequence.
An End-to-end Chinese Text Normalization Model based on Rule-guided Flat-Lattice Transformer
Inspired by Flat-LAttice Transformer (FLAT), we propose an end-to-end Chinese text normalization model, which accepts Chinese characters as direct input and integrates expert knowledge contained in rules into the neural network, both contribute to the superior performance of proposed model for the text normalization task.
Neural Architecture Search for Speech Emotion Recognition
Deep neural networks have brought significant advancements to speech emotion recognition (SER).
Disentangleing Content and Fine-grained Prosody Information via Hybrid ASR Bottleneck Features for Voice Conversion
In this paper, we proposed an any-to-one VC method using hybrid bottleneck features extracted from CTC-BNFs and CE-BNFs to complement each other advantages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Towards Expressive Speaking Style Modelling with Hierarchical Context Information for Mandarin Speech Synthesis
In this paper, we propose a hierarchical framework to model speaking style from context.
FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement
Previously proposed FullSubNet has achieved outstanding performance in Deep Noise Suppression (DNS) Challenge and attracted much attention.
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs).
An Approach to Mispronunciation Detection and Diagnosis with Acoustic, Phonetic and Linguistic (APL) Embeddings
These embeddings, when used as implicit phonetic supplementary information, can alleviate the data shortage of explicit phoneme annotations.
Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding
Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languages, the augmented data sets are often noisy, and thus impede the performance of SLU models.
Adversarial Sample Detection for Speaker Verification by Neural Vocoders
We also show that the neural vocoder adopted in the detection framework is dataset-independent.
Voting for the right answer: Adversarial defense for speaker verification
Automatic speaker verification (ASV) is a well developed technology for biometric identification, and has been ubiquitous implemented in security-critic applications, such as banking and access control.
Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling
However, state-of-the-art context modeling methods in conversational TTS only model the textual information in context with a recurrent neural network (RNN).
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms.
Cascaded Head-colliding Attention
Transformers have advanced the field of natural language processing (NLP) on a variety of important tasks.
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
A neural multimodal machine translation (MMT) system is one that aims to perform better translation by extending conventional text-only translation models with multimodal information.
Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis
Exploiting rich linguistic information in raw text is crucial for expressive text-to-speech (TTS).
Towards Multi-Scale Style Control for Expressive Speech Synthesis
This paper introduces a multi-scale speech style modeling method for end-to-end expressive speech synthesis.
The Multi-speaker Multi-style Voice Cloning Challenge 2021
The challenge consists of two tracks, namely few-shot track and one-shot track, where the participants are required to clone multiple target voices with 100 and 5 samples respectively.
Adversarial defense for automatic speaker verification by cascaded self-supervised learning models
Automatic speaker verification (ASV) is one of the core technologies in biometric identification.
Adversarially learning disentangled speech representations for robust multi-factor voice conversion
To increase the robustness of highly controllable style transfer on multiple factors in VC, we propose a disentangled speech representation learning framework based on adversarial learning.
Good for Misconceived Reasons: Revisiting Neural Multimodal Machine Translation
A neural multimodal machine translation (MMT) system is one that aims to perform better translation by extending conventional text-only translation models with multimodal information.
Unsupervised Cross-Lingual Speech Emotion Recognition Using DomainAdversarial Neural Network
By using deep learning approaches, Speech Emotion Recog-nition (SER) on a single domain has achieved many excellentresults.
Syntactic representation learning for neural network based TTS with syntactic parse tree traversal
Meanwhile, nuclear-norm maximization loss is introduced to enhance the discriminability and diversity of the embeddings of constituent labels.
FERNet: Fine-grained Extraction and Reasoning Network for Emotion Recognition in Dialogues
Unlike non-conversation scenes, emotion recognition in dialogues (ERD) poses more complicated challenges due to its interactive nature and intricate contextual information.
Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input
Non-autoregressive (NAR) transformer models have achieved significantly inference speedup but at the cost of inferior accuracy compared to autoregressive (AR) models in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Improving pronunciation assessment via ordinal regression with anchored reference samples
Traditional speech pronunciation assessment, based on the Goodness of Pronunciation (GOP) algorithm, has some weakness in assessing a speech utterance: 1) Phoneme GOP scores cannot be easily translated into a sentence score with a simple average for effective assessment; 2) The rank ordering information has not been well exploited in GOP scoring for delivering a robust assessment and correlate well with a human rater's evaluations.
Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams
Recent approaches mainly have following limitations: 1) most speaker-independent methods need handcrafted features that are time-consuming to design or unreliable; 2) there is no convincing method to support multilingual or mixlingual speech as input.
Noise Robust TTS for Low Resource Speakers using Pre-trained Model and Speech Enhancement
Firstly, the speech synthesis model is pre-trained with both multi-speaker clean data and noisy augmented data; then the pre-trained model is adapted on noisy low-resource new speaker data; finally, by setting the clean speech condition, the model can synthesize the new speaker's clean voice.
Walking with Perception: Efficient Random Walk Sampling via Common Neighbor Awareness
Random walk is widely applied to sample large-scale graphs due to its simplicity of implementation and solid theoretical foundations of bias analysis.
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
However, this approach of evaluating a language model is undermined by the uncertainty of the amount of knowledge that is learned by the probe itself.
Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks
Our best systems achieve a relative improvement of 11. 9% and 8. 3% on the TIMIT and WSJ tasks respectively.
NEXT: A Neural Network Framework for Next POI Recommendation
Inspired by the recent success of neural networks in many areas, in this paper, we present a simple but effective neural network framework for next POI recommendation, named NEXT.
Study on Feature Subspace of Archetypal Emotions for Speech Emotion Recognition
Hence, traditional methods may fail to distinguish some of the emotions with just one global feature subspace.
Feature Learning with Gaussian Restricted Boltzmann Machine for Robust Speech Recognition
In this paper, we first present a new variant of Gaussian restricted Boltzmann machine (GRBM) called multivariate Gaussian restricted Boltzmann machine (MGRBM), with its definition and learning algorithm.
