Search Results for author:
Found 53 papers, 18 papers with code
Tacotron: Towards End-to-End Speech Synthesis
A text-to-speech synthesis system typically consists of multiple stages, such as a text analysis frontend, an acoustic model and an audio synthesis module.
Ranked #5 on
Speech Synthesis
on North American English
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions
This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text.
Ranked #2 on
Speech Synthesis
on North American English
Adversarial Autoencoders
In this paper, we propose the "adversarial autoencoder" (AAE), which is a probabilistic autoencoder that uses the recently proposed generative adversarial networks (GAN) to perform variational inference by matching the aggregated posterior of the hidden code vector of the autoencoder with an arbitrary prior distribution.
Ranked #6 on
Unsupervised Image Classification
on MNIST
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Listen, Attend and Spell
Unlike traditional DNN-HMM models, this model learns all the components of a speech recognizer jointly.
State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Recurrent Neural Networks can be trained to produce sequences of tokens given some input, as exemplified by recent results in machine translation and image captioning.
Pointer Networks
It differs from the previous attention attempts in that, instead of using attention to blend hidden units of an encoder to a context vector at each decoder step, it uses attention as a pointer to select a member of the input sequence as the output.
Ranked #9 on
Point Cloud Completion
on ShapeNet
(using extra training data)
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
Learning long term dependencies in recurrent networks is difficult due to vanishing and exploding gradients.
Ranked #27 on
Sequential Image Classification
on Sequential MNIST
RNN Approaches to Text Normalization: A Challenge
Though our conclusions are largely negative on this point, we are actually not arguing that the text normalization problem is intractable using an pure RNN approach, merely that it is not going to be something that can be solved merely by having huge amounts of annotated text data and feeding that to a general RNN model.
Imputer: Sequence Modelling via Imputation and Dynamic Programming
This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations.
An Online Sequence-to-Sequence Model Using Partial Conditioning
However, they are unsuitable for tasks that require incremental predictions to be made as more data arrives or tasks that have long input sequences and output sequences.
PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation.
How Far Are We from Intelligent Visual Deductive Reasoning?
Vision-Language Models (VLMs) such as GPT-4V have recently demonstrated incredible strides on diverse vision language tasks.
Sequence-to-Sequence Models Can Directly Translate Foreign Speech
We present a recurrent encoder-decoder deep neural network architecture that directly translates speech in one language into text in another.
Machine Translation
Sequence-To-Sequence Speech Recognition
+2
Very Deep Convolutional Networks for End-to-End Speech Recognition
Sequence-to-sequence models have shown success in end-to-end speech recognition.
Position Prediction as an Effective Pretraining Strategy
This pretraining strategy which has been used in BERT models in NLP, Wav2Vec models in Speech and, recently, in MAE models in Vision, forces the model to learn about relationships between the content in different parts of the input using autoencoding related objectives.
Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games
In this paper, we offer a surrogate problem which assesses an LLMs's capability to deduce an entity unknown to itself, but revealed to a judge, by asking the judge a series of queries.
Discrete Sequential Prediction of Continuous Actions for Deep RL
Specifically, we show how Q-values and policies over continuous spaces can be modeled using a next step prediction model over discretized dimensions.
Speech recognition for medical conversations
We explored both CTC and LAS systems for building speech recognition models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Learning Hard Alignments with Variational Inference
There has recently been significant interest in hard attention models for tasks such as object recognition, visual captioning and speech recognition.
An online sequence-to-sequence model for noisy speech recognition
This is because the models require that the entirety of the input sequence be available at the beginning of inference, an assumption that is not valid for instantaneous speech recognition.
Next-Step Conditioned Deep Convolutional Neural Networks Improve Protein Secondary Structure Prediction
This sequential model achieves 70. 3% Q8 accuracy on CB513 with a single model; an ensemble of these models produces 71. 4% Q8 accuracy on the same test set, improving upon the previous overall state of the art for the eight-class secondary structure problem.
Reward Augmented Maximum Likelihood for Neural Structured Prediction
A key problem in structured output prediction is direct optimization of the task reward function that matters for test evaluation.
Towards better decoding and language model integration in sequence to sequence models
The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Protein Secondary Structure Prediction Using Deep Multi-scale Convolutional Neural Networks and Next-Step Conditioning
We first train a series of deep neural networks to predict eight-class secondary structure labels given a protein's amino acid sequence information and find that using recent methods for regularization, such as dropout and weight-norm constraining, leads to measurable gains in accuracy.
Protein Secondary Structure Prediction
Protein Structure Prediction
Chained Predictions Using Convolutional Neural Networks
In this model the output variables for a given input are predicted sequentially using neural networks.
A Neural Transducer
However, they are unsuitable for tasks that require incremental predictions to be made as more data arrives or tasks that have long input sequences and output sequences.
Learning Online Alignments with Continuous Rewards Policy Gradient
Though capable and easy to use, they require that the entirety of the input sequence is available at the beginning of inference, an assumption that is not valid for instantaneous translation and speech recognition.
Object Recognition from Short Videos for Robotic Perception
Videos, unlike still images, are temporally coherent which makes the application of deep networks non-trivial.
Occlusion Edge Detection in RGB-D Frames using Deep Convolutional Networks
Occlusion edges in images which correspond to range discontinuity in the scene from the point of view of the observer are an important prerequisite for many vision and mobile robot tasks.
Multi-task Neural Networks for QSAR Predictions
Although artificial neural networks have occasionally been used for Quantitative Structure-Activity/Property Relationship (QSAR/QSPR) studies in the past, the literature has of late been dominated by other machine learning techniques such as random forests.
Peptide-Spectra Matching from Weak Supervision
As in many other scientific domains, we face a fundamental problem when using machine learning to identify proteins from mass spectrometry data: large ground truth datasets mapping inputs to correct outputs are extremely difficult to obtain.
SPIN: A High Speed, High Resolution Vision Dataset for Tracking and Action Recognition in Ping Pong
The corpus consists of ping pong play with three main annotation streams that can be used to learn tracking and action recognition models -- tracking of the ping pong ball and poses of humans in the videos and the spin of the ball being hit by humans.
Robotic Table Tennis with Model-Free Reinforcement Learning
We propose a model-free algorithm for learning efficient policies capable of returning table tennis balls by controlling robot joints at a rate of 100Hz.
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Most current speech recognition systems use hidden Markov models (HMMs) to deal with the temporal variability of speech and Gaussian mixture models to determine how well each state of each HMM fits a frame or a short window of frames of coefficients that represents the acoustic input.
Towards End-To-End Speech Recognition with Recurrent Neural Networks
A modification to the objective function is introduced that trains the network to minimise the expectation of an arbitrary transcription loss function.
RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions
On a long-form YouTube test set, when the nonstreaming RNN-T model is trained with shorter segments of data, the proposed combination improves word error rate (WER) from 22. 3% to 14. 8%; when the streaming RNN-T model trained on short Search queries, the proposed techniques improve WER on the YouTube set from 67. 0% to 25. 3%.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Policy Optimization by Local Improvement through Search
On the other end of the spectrum, approaches rooted in Policy Iteration, such as Dual Policy Iteration do not choose next step actions based on an expert, but instead use planning or search over the policy to choose an action distribution to train towards.
Efficient Representation Learning via Adaptive Context Pooling
Inspired by the success of ConvNets that are combined with pooling to capture long-range dependencies, we learn to pool neighboring features for each token before computing attention in a given attention layer.
Continuous Pseudo-Labeling from the Start
Nevertheless, these approaches still rely on bootstrapping the ST using an initial supervised learning phase where the model is trained on labeled data alone.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
More Speaking or More Speakers?
We perform a systematic analysis on both labeled and unlabeled data by varying the number of speakers while keeping the number of hours fixed and vice versa.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Continuous Soft Pseudo-Labeling in ASR
Continuous pseudo-labeling (PL) algorithms such as slimIPL have recently emerged as a powerful strategy for semi-supervised learning in speech recognition.
Understanding the Robustness of Multi-Exit Models under Common Corruptions
Furthermore, the lack of calibration increases the inconsistency in the predictions of the model across exits, leading to both inefficient inference and more misclassifications compared with evaluation on in-distribution data.
REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation
In online F-TTA, a pre-trained model is adapted using a stream of test samples by minimizing a self-supervised objective, such as entropy minimization.
Robotic Table Tennis: A Case Study into a High Speed Learning System
We present a deep-dive into a real-world robotic learning system that, in previous work, was shown to be capable of hundreds of table tennis rallies with a human and has the ability to precisely return the ball to desired targets.
Construction of Paired Knowledge Graph-Text Datasets Informed by Cyclic Evaluation
Guided by these observations, we construct a new, improved dataset called LAGRANGE using heuristics meant to improve equivalence between KG and text and show the impact of each of the heuristics on cyclic evaluation.
Matryoshka Diffusion Models
Diffusion models are the de facto approach for generating high-quality images and videos, but learning high-dimensional models remains a formidable task due to computational and optimization challenges.
Generating Molecular Conformer Fields
In this paper we tackle the problem of generating conformers of a molecule in 3D space given its molecular graph.
KGLens: A Parameterized Knowledge Graph Solution to Assess What an LLM Does and Doesn't Know
Measuring the alignment between a Knowledge Graph (KG) and Large Language Models (LLMs) is an effective method to assess the factualness and identify the knowledge blind spots of LLMs.
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling
Large language models are trained on massive scrapes of the web, which are often unstructured, noisy, and poorly phrased.
Divide-or-Conquer? Which Part Should You Distill Your LLM?
Recent methods have demonstrated that Large Language Models (LLMs) can solve reasoning tasks better when they are encouraged to solve subtasks of the main task first.
