Search Results for author:
Found 41 papers, 25 papers with code
Marginal Structured SVM with Hidden Variables
In this work, we propose the marginal structured SVM (MSSVM) for structured prediction with hidden variables.
Decomposition Bounds for Marginal MAP
Marginal MAP inference involves making MAP predictions in systems defined with latent variables or missing information.
Belief Propagation in Conditional RBMs for Structured Prediction
We demonstrate that, in both maximum likelihood and max-margin learning, training conditional RBMs with BP as the inference routine can provide significantly better results than current state-of-the-art CD methods on structured prediction problems.
Deep Voice 2: Multi-Speaker Neural Text-to-Speech
We introduce Deep Voice 2, which is based on a similar pipeline with Deep Voice 1, but constructed with higher performance building blocks and demonstrates a significant audio quality improvement over Deep Voice 1.
Learning Infinite RBMs with Frank-Wolfe
In this work, we propose an infinite restricted Boltzmann machine~(RBM), whose maximum likelihood estimation~(MLE) corresponds to a constrained convex optimization.
Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
We present Deep Voice 3, a fully-convolutional attention-based neural text-to-speech (TTS) system.
Topic Compositional Neural Language Model
The TCNLM learns the global semantic coherence of a document via a neural topic model, and the probability of each learned latent topic is further used to build a Mixture-of-Experts (MoE) language model, where each expert (corresponding to one topic) is a recurrent neural network (RNN) that accounts for learning the local structure of a word sequence.
HybridNet: A Hybrid Neural Architecture to Speed-up Autoregressive Models
This paper introduces HybridNet, a hybrid neural network to speed-up autoregressive models for raw audio waveform generation.
Neural Voice Cloning with a Few Samples
Speaker adaptation is based on fine-tuning a multi-speaker generative model with a few cloning samples.
Cancer Metastasis Detection With Neural Conditional Random Field
Compared to the baseline method without considering spatial correlations, we show that the proposed NCRF framework obtains probability maps of patch predictions with better visual quality.
ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
In this work, we propose a new solution for parallel wave generation by WaveNet.
Large Margin Neural Language Model
We propose a large margin criterion for training neural language models.
Non-Autoregressive Neural Text-to-Speech
In this work, we propose ParaNet, a non-autoregressive seq2seq model that converts text to spectrogram.
Multi-Speaker End-to-End Speech Synthesis
In this work, we extend ClariNet (Ping et al., 2019), a fully end-to-end speech synthesis model (i. e., text-to-wave), to generate high-fidelity speech from multiple speakers.
WaveFlow: A Compact Flow-based Model for Raw Audio
WaveFlow provides a unified view of likelihood-based models for 1-D data, including WaveNet and WaveGlow as special cases.
Ranked #9 on
Speech Synthesis
on LibriTTS
Parallel Neural Text-to-Speech
In this work, we first propose ParaNet, a non-autoregressive seq2seq model that converts text to spectrogram.
DiffWave: A Versatile Diffusion Model for Audio Synthesis
In this work, we propose DiffWave, a versatile diffusion probabilistic model for conditional and unconditional waveform generation.
Ranked #2 on
Speech Synthesis
on LJSpeech
Local Knowledge Powered Conversational Agents
State-of-the-art conversational agents have advanced significantly in conjunction with the use of large transformer-based language models.
End-to-End Training of Neural Retrievers for Open-Domain Question Answering
We also explore two approaches for end-to-end supervised training of the reader and retriever components in OpenQA models.
On Fast Sampling of Diffusion Probabilistic Models
In this work, we propose FastDPM, a unified framework for fast sampling in diffusion probabilistic models.
RAD-TTS: Parallel Flow-Based TTS with Robust Alignment Learning and Diverse Synthesis
This work introduces a predominantly parallel, end-to-end TTS model based on normalizing flows.
Long-Short Transformer: Efficient Transformers for Language and Vision
For instance, Transformer-LS achieves 0. 97 test BPC on enwik8 using half the number of parameters than previous method, while being faster and is able to handle 3x as long sequences compared to its full-attention version on the same hardware.
Ranked #1 on
Language Modelling
on enwik8 dev
One TTS Alignment To Rule Them All
However, these alignments tend to be brittle and often fail to generalize to long utterances and out-of-domain text, leading to missing or repeating words.
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models.
Speech Denoising in the Waveform Domain with Self-Attention
In this work, we present CleanUNet, a causal speech denoising model on the raw waveform.
Multi-Stage Prompting for Knowledgeable Dialogue Generation
We propose a multi-stage prompting approach to generate knowledgeable responses from a single pretrained LM.
BigVGAN: A Universal Neural Vocoder with Large-Scale Training
Despite recent progress in generative adversarial network (GAN)-based vocoders, where the model generates raw waveform conditioned on acoustic features, it is challenging to synthesize high-fidelity audio for numerous speakers across various recording environments.
Ranked #5 on
Speech Synthesis
on LibriTTS
Factuality Enhanced Language Models for Open-Ended Text Generation
In this work, we measure and improve the factual accuracy of large-scale LMs for open-ended text generation.
Evaluating Parameter Efficient Learning for Generation
For cross-domain and cross-dataset cases, we show that (a) Adapter (Houlsby et al., 2019) performs the best amongst all the PERMs studied here, and (b) it outperforms finetuning if the task dataset is below a certain size.
FlowGrad: Controlling the Output of Generative ODEs With Gradients
To further accelerate the computation of the back-propagation, we propose to use a non-uniform discretization to approximate the ODE trajectory, where we measure how straight the trajectory is and gather the straight parts into one discretization step.
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
Augmenting pretrained language models (LMs) with a vision encoder (e. g., Flamingo) has obtained the state-of-the-art results in image-to-text generation.
Defending against Adversarial Audio via Diffusion Model
In this paper, we propose an adversarial purification-based defense pipeline, AudioPure, for acoustic systems via off-the-shelf diffusion models.
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Thus, it is still an open question: shall we pretrain large autoregressive LMs with retrieval?
Defending against Insertion-based Textual Backdoor Attacks via Attribution
Textual backdoor attack, as a novel attack model, has been shown to be effective in adding a backdoor to the model during training.
RAVEN: In-Context Learning with Retrieval-Augmented Encoder-Decoder Language Models
In this paper, we investigate the in-context learning ability of retrieval-augmented encoder-decoder language models.
CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram
In this work, we present CleanUNet 2, a speech denoising model that combines the advantages of waveform denoiser and spectrogram denoiser and achieves the best of both worlds.
Retrieval meets Long Context Large Language Models
Perhaps surprisingly, we find that LLM with 4K context window using simple retrieval-augmentation at generation can achieve comparable performance to finetuned LLM with 16K context window via positional interpolation on long context tasks, while taking much less computation.
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
After instruction tuning on Retro, InstructRetro demonstrates significant improvement over the instruction tuned GPT on a wide range of zero-shot tasks.
VILA: On Pre-training for Visual Language Models
Visual language models (VLMs) rapidly progressed with the recent success of large language models.
Ranked #21 on
Visual Question Answering
on MM-Vet
ChatQA: Building GPT-4 Level Conversational QA Models
In this work, we introduce ChatQA, a family of conversational question answering (QA) models that obtain GPT-4 level accuracies.
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities
Augmenting large language models (LLMs) to understand audio -- including non-speech sounds and non-verbal speech -- is critically important for diverse real-world applications of LLMs.
