Search Results for author:
Found 68 papers, 25 papers with code
Clozer”:" Adaptable Data Augmentation for Cloze-style Reading Comprehension
Task-adaptive pre-training (TAPT) alleviates the lack of labelled data and provides performance lift by adapting unlabelled data to downstream task.
Dimsum @LaySumm 20
Lay summarization aims to generate lay summaries of scientific papers automatically.
A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
It is, for example, better at deductive than inductive reasoning.
NusaCrowd: Open Source Initiative for Indonesian NLP Resources
We present NusaCrowd, a collaborative initiative to collect and unite existing resources for Indonesian languages, including opening access to previously non-public resources.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
TriNet: stabilizing self-supervised learning from complete or slow collapse on ASR
Self-supervised learning (SSL) models confront challenges of abrupt informational collapse or slow dimensional collapse.
Generative Long-form Question Answering: Relevance, Faithfulness and Succinctness
We pioneered the research direction to improve the answer quality in terms of 1) query-relevance, 2) answer faithfulness, and 3) answer succinctness.
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training
Large-scale vision-language pre-trained (VLP) models are prone to hallucinate non-existent visual objects when generating text based on visual information.
Context Generation Improves Open Domain Question Answering
Prior work on closed-book QA either directly finetunes or prompts a pretrained language model (LM) to leverage the stored knowledge.
Cross-Age Speaker Verification: Learning Age-Invariant Speaker Embeddings
In this paper, we mine cross-age test sets based on the VoxCeleb dataset and propose our age-invariant speaker representation(AISR) learning method.
End-to-End Voice Conversion with Information Perturbation
The ideal goal of voice conversion is to convert the source speaker's speech to sound naturally like the target speaker while maintaining the linguistic content and the prosody of the source speech.
Towards Answering Open-ended Ethical Quandary Questions
We explore the current capability of LLMs in providing an answer with a deliberative exchange of different perspectives to an ethical quandary, in the approach of Socratic philosophy, instead of providing a closed answer like an oracle.
FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis
Also, FastDiff enables a sampling speed of 58x faster than real-time on a V100 GPU, making diffusion models practically applicable to speech synthesis deployment for the first time.
Ranked #6 on
Text-To-Speech Synthesis
on LJSpeech
3M: Multi-loss, Multi-path and Multi-level Neural Networks for speech recognition
Recently, Conformer based CTC/AED model has become a mainstream architecture for ASR.
Ranked #1 on
Speech Recognition
on WenetSpeech
BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis
We propose a new bilateral denoising diffusion model (BDDM) that parameterizes both the forward and reverse processes with a schedule network and a score network, which can train with a novel bilateral modeling objective.
Ranked #1 on
Speech Synthesis
on LJSpeech
Read before Generate! Faithful Long Form Question Answering with Machine Reading
Long-form question answering (LFQA) aims to generate a paragraph-length answer for a given question.
Ranked #1 on
Question Answering
on KILT: ELI5
VCVTS: Multi-speaker Video-to-Speech synthesis via cross-modal knowledge transfer from voice conversion
Though significant progress has been made for speaker-dependent Video-to-Speech (VTS) synthesis, little attention is devoted to multi-speaker VTS that can map silent video to speech, while allowing flexible control of speaker identity, all in a single system.
QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation
Multi-hop question generation (MQG) aims to generate complex questions which require reasoning over multiple pieces of information of the input passage.
Survey of Hallucination in Natural Language Generation
This advancement has led to more fluent and coherent NLG, leading to improved development in downstream tasks such as abstractive summarization, dialogue generation and data-to-text generation.
The CUHK-TENCENT speaker diarization system for the ICASSP 2022 multi-channel multi-party meeting transcription challenge
This paper describes our speaker diarization system submitted to the Multi-channel Multi-party Meeting Transcription (M2MeT) challenge, where Mandarin meeting data were recorded in multi-channel format for diarization and automatic speech recognition (ASR) tasks.
DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
Denoising diffusion probabilistic models (DDPMs) are expressive generative models that have been used to solve a variety of speech synthesis problems.
Consistent Training and Decoding For End-to-end Speech Recognition Using Lattice-free MMI
Recently, End-to-End (E2E) frameworks have achieved remarkable results on various Automatic Speech Recognition (ASR) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SpeechMoE2: Mixture-of-Experts Model with Improved Routing
Mixture-of-experts based acoustic models with dynamic routing mechanisms have proved promising results for speech recognition.
Meta-Voice: Fast few-shot style transfer for expressive voice cloning using meta learning
The task of few-shot style transfer for voice cloning in text-to-speech (TTS) synthesis aims at transferring speaking styles of an arbitrary source speaker to a target speaker's voice using very limited amount of neutral data.
SynCLR: A Synthesis Framework for Contrastive Learning of out-of-domain Speech Representations
Learning generalizable speech representations for unseen samples in different domains has been a challenge with ever increasing importance to date.
AppQ: Warm-starting App Recommendation Based on View Graphs
In this work, we propose AppQ, a novel app quality grading and recommendation system that extracts inborn features of apps based on app source code.
Referee: Towards reference-free cross-speaker style transfer with low-quality data for expressive speech synthesis
The S2W model is trained with high-quality target data, which is adopted to effectively aggregate style descriptors and generate high-fidelity speech in the target speaker's voice.
Bilateral Denoising Diffusion Models
In this paper, we propose novel bilateral denoising diffusion models (BDDMs), which take significantly fewer steps to generate high-quality samples.
Glow-WaveGAN: Learning Speech Representations from GAN-based Variational Auto-Encoder For High Fidelity Flow-based Speech Synthesis
Current two-stage TTS framework typically integrates an acoustic model with a vocoder -- the acoustic model predicts a low resolution intermediate representation such as Mel-spectrum while the vocoder generates waveform from the intermediate representation.
Controllable Context-aware Conversational Speech Synthesis
Specifically, we use explicit labels to represent two typical spontaneous behaviors filled-pause and prolongation in the acoustic model and develop a neural network based predictor to predict the occurrences of the two behaviors from text.
GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio
This paper introduces GigaSpeech, an evolving, multi-domain English speech recognition corpus with 10, 000 hours of high quality labeled audio suitable for supervised training, and 40, 000 hours of total audio suitable for semi-supervised and unsupervised training.
Ranked #1 on
Speech Recognition
on GigaSpeech
Enhancing Speaking Styles in Conversational Text-to-Speech Synthesis with Graph-based Multi-modal Context Modeling
However, state-of-the-art context modeling methods in conversational TTS only model the textual information in context with a recurrent neural network (RNN).
Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition
End-to-end speech recognition generally uses hand-engineered acoustic features as input and excludes the feature extraction module from its joint optimization.
DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion
Singing voice conversion (SVC) is one promising technique which can enrich the way of human-computer interaction by endowing a computer the ability to produce high-fidelity and expressive singing voice.
Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance
Query focused summarization (QFS) models aim to generate summaries from source documents that can answer the given query.
Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters
This paper proposes KnowExpert, a framework to bypass the explicit retrieval process and inject knowledge into the pre-trained language models with lightweight adapters and adapt to the knowledge-grounded dialogue task.
Latency-Controlled Neural Architecture Search for Streaming Speech Recognition
Extensive experiments show that: 1) Based on the proposed neural architecture, the neural networks with a medium latency of 550ms (millisecond) and a low latency of 190ms can be learned in the vanilla and revised operation space respectively.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts
Recently, Mixture of Experts (MoE) based Transformer has shown promising results in many domains.
Enhancing Word-Level Semantic Representation via Dependency Structure for Expressive Text-to-Speech Synthesis
Exploiting rich linguistic information in raw text is crucial for expressive text-to-speech (TTS).
TeCANet: Temporal-Contextual Attention Network for Environment-Aware Speech Dereverberation
In this paper, we exploit the effective way to leverage contextual information to improve the speech dereverberation performance in real-world reverberant environments.
Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect
We study the cocktail party problem and propose a novel attention network called Tune-In, abbreviated for training under negative environments with interference.
Contrastive Separative Coding for Self-supervised Representation Learning
To extract robust deep representations from long sequential modeling of speech data, we propose a self-supervised learning approach, namely Contrastive Separative Coding (CSC).
Sandglasset: A Light Multi-Granularity Self-attentive Network For Time-Domain Speech Separation
One of the leading single-channel speech separation (SS) models is based on a TasNet with a dual-path segmentation technique, where the size of each segment remains unchanged throughout all layers.
Ranked #6 on
Speech Separation
on WSJ0-3mix
VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attention
This paper proposes VARA-TTS, a non-autoregressive (non-AR) text-to-speech (TTS) model using a very deep Variational Autoencoder (VDVAE) with Residual Attention mechanism, which refines the textual-to-acoustic alignment layer-wisely.
Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks
Recent research on the time-domain audio separation networks (TasNets) has brought great success to speech separation.
Ranked #9 on
Speech Separation
on WSJ0-2mix
Phonetic Posteriorgrams based Many-to-Many Singing Voice Conversion via Adversarial Training
In order to make timbre conversion more stable and controllable, speaker embedding is further decomposed to the weighted sum of a group of trainable vectors representing different timbre clusters.
FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulation
This paper presents FastSVC, a light-weight cross-domain singing voice conversion (SVC) system, which can achieve high conversion performance, with inference speed 4x faster than real-time on CPUs.
Non-Autoregressive Transformer ASR with CTC-Enhanced Decoder Input
Non-autoregressive (NAR) transformer models have achieved significantly inference speedup but at the cost of inferior accuracy compared to autoregressive (AR) models in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Replay and Synthetic Speech Detection with Res2net Architecture
This multiple scaling mechanism significantly improves the countermeasure's generalizability to unseen spoofing attacks.
Dimsum @LaySumm 20: BART-based Approach for Scientific Document Summarization
Lay summarization aims to generate lay summaries of scientific papers automatically.
Multi-hop Question Generation with Graph Convolutional Network
Multi-hop Question Generation (QG) aims to generate answer-related questions by aggregating and reasoning over multiple scattered evidence from different paragraphs.
Learned Transferable Architectures Can Surpass Hand-Designed Architectures for Large Scale Speech Recognition
Extensive experiments show that: (i) the architecture searched on the small proxy dataset can be transferred to the large dataset for the speech recognition tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Independent and Multilingual/Mixlingual Speech-Driven Talking Head Generation Using Phonetic Posteriorgrams
Recent approaches mainly have following limitations: 1) most speaker-independent methods need handcrafted features that are time-consuming to design or unreliable; 2) there is no convincing method to support multilingual or mixlingual speech as input.
Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification
Orthogonal to prior approaches, this work proposes to defend ASV systems against adversarial attacks with a separate detection network, rather than augmenting adversarial data into ASV training.
Audio-visual Multi-channel Recognition of Overlapped Speech
Automatic speech recognition (ASR) of overlapped speech remains a highly challenging task to date.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
CAiRE-COVID: A Question Answering and Query-focused Multi-Document Summarization System for COVID-19 Scholarly Information Management
We present CAiRE-COVID, a real-time question answering (QA) and multi-document summarization system, which won one of the 10 tasks in the Kaggle COVID-19 Open Research Dataset Challenge, judged by medical experts.
Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning
Hand-crafted spatial features (e. g., inter-channel phase difference, IPD) play a fundamental role in recent deep learning based multi-channel speech separation (MCSS) methods.
Generalizing Question Answering System with Pre-trained Language Model Fine-tuning
With a large number of datasets being released and new techniques being proposed, Question answering (QA) systems have witnessed great breakthroughs in reading comprehension (RC)tasks.
Mixup-breakdown: a consistency training method for improving generalization of speech separation models
Deep-learning based speech separation models confront poor generalization problem that even the state-of-the-art models could abruptly fail when evaluating them in mismatch conditions.
DFSMN-SAN with Persistent Memory Model for Automatic Speech Recognition
Self-attention networks (SAN) have been introduced into automatic speech recognition (ASR) and achieved state-of-the-art performance owing to its superior ability in capturing long term dependency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networks
Our best systems achieve a relative improvement of 11. 9% and 8. 3% on the TIMIT and WSJ tasks respectively.
A Random Gossip BMUF Process for Neural Language Modeling
One important challenge of training an NNLM is to leverage between scaling the learning process and handling big data.
DurIAN: Duration Informed Attention Network For Multimodal Synthesis
In this paper, we present a generic and robust multimodal synthesis system that produces highly natural speech and facial expression simultaneously.
Phrase-Level Class based Language Model for Mandarin Smart Speaker Query Recognition
In this work, we propose to train pruned language models for the word classes to replace the slots in the root n-gram.
Maximizing Mutual Information for Tacotron
End-to-end speech synthesis methods already achieve close-to-human quality performance.
Teach an all-rounder with experts in different domains
First, for each domain, a teacher model (domain-dependent model) is trained by fine-tuning a multi-condition model with domain-specific subset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Learning discriminative features in sequence training without requiring framewise labelled data
We propose a novel method which simultaneously models both the sequence discriminative training and the feature discriminative learning within a single network architecture, so that it can learn discriminative deep features in sequence training that obviates the need for presegmented training data.
End-to-End Multi-Channel Speech Separation
This paper extended the previous approach and proposed a new end-to-end model for multi-channel speech separation.
RGB-D Salient Object Detection Based on Discriminative Cross-modal Transfer Learning
In the proposed approach, we leverage the auxiliary data from the source modality effectively by training the RGB saliency detection network to obtain the task-specific pre-understanding layers for the target modality.
