Search Results for author:
Found 26 papers, 13 papers with code
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance
While autoregressive speech token generation models produce speech with remarkable variety and naturalness, their inherent lack of controllability often results in issues such as hallucinations and undesired vocalizations that do not conform to conditioning inputs.
TTS-Transducer: End-to-End Speech Synthesis with Neural Transducer
The proposed system first uses a transducer architecture to learn monotonic alignments between tokenized text and speech codec tokens for the first codebook.
Low Frame-rate Speech Codec: a Codec Designed for Fast High-quality Speech LLM Training and Inference
Large language models (LLMs) have significantly advanced audio processing through audio codecs that convert audio into discrete tokens, enabling the application of language modeling techniques to audio data.
Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment
Large Language Model (LLM) based text-to-speech (TTS) systems have demonstrated remarkable capabilities in handling large speech datasets and generating natural speech for new speakers.
Mechanistic Interpretability of Binary and Ternary Transformers
Recent research (arXiv:2310. 11453, arXiv:2402. 17764) has proposed binary and ternary transformer networks as a way to significantly reduce memory and improve inference speed in Large Language Models (LLMs) while maintaining accuracy.
zkLLM: Zero Knowledge Proofs for Large Language Models
The recent surge in artificial intelligence (AI), characterized by the prominence of large language models (LLMs), has ushered in fundamental transformations across the globe.
SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation
We present a novel Speech Augmented Language Model (SALM) with {\em multitask} and {\em in-context} learning capabilities.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Exploring Embeddings for Measuring Text Relatedness: Unveiling Sentiments and Relationships in Online Comments
This paper investigates sentiment and semantic relationships among comments across various social media platforms, as well as discusses the importance of shared opinions across these different media platforms, using word embeddings to analyze components in sentences and documents.
AstroLLaMA: Towards Specialized Foundation Models in Astronomy
Large language models excel in many human-language tasks but often falter in highly specialized domains like scholarly astronomy.
zkDL: Efficient Zero-Knowledge Proofs of Deep Learning Training
In response to this challenge, we present zero-knowledge deep learning (zkDL), an efficient zero-knowledge proof for deep learning training.
NutritionVerse-Thin: An Optimized Strategy for Enabling Improved Rendering of 3D Thin Food Models
With the growth in capabilities of generative models, there has been growing interest in using photo-realistic renders of common 3D food items to improve downstream tasks such as food printing, nutrition prediction, or management of food wastage.
ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representations
In this work, we propose a zero-shot voice conversion method using speech representations trained with self-supervised learning.
Modeling Human Eye Movements with Neural Networks in a Maze-Solving Task
This not only provides a generative model of eye movements in this task but also suggests a computational theory for how humans solve the task, namely that humans use mental simulation.
Adapting TTS models For New Speakers using Transfer Learning
We address this challenge by proposing transfer-learning guidelines for adapting high quality single-speaker TTS models for a new speaker, using only a few minutes of speech data.
Offensive Language and Hate Speech Detection with Deep Learning and Transfer Learning
Differentiating if a text message belongs to hate speech and offensive language is a key challenge in automatic detection of toxic text content.
A Lightweight Algorithm to Uncover Deep Relationships in Data Tables
The key to our approach is a computationally lightweight forward addition algorithm that we developed to recursively extract the functional dependencies between table columns that are scalable to tables with many columns.
Cycle Text-To-Image GAN with BERT
We explore novel approaches to the task of image generation from their respective captions, building on state-of-the-art GAN architectures.
UniVL: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation
However, most of the existing multimodal models are pre-trained for understanding tasks, leading to a pretrain-finetune discrepancy for generation tasks.
Ranked #2 on
Action Segmentation
on COIN
(using extra training data)
Training Deep Networks with Stochastic Gradient Normalized by Layerwise Adaptive Second Moments
We propose NovoGrad, an adaptive stochastic gradient descent method with layer-wise gradient normalization and decoupled weight decay.
Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens
Mellotron is a multispeaker voice synthesis model based on Tacotron 2 GST that can make a voice emote and sing without emotive or singing training data.
QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions
We propose a new end-to-end neural acoustic model for automatic speech recognition.
Ranked #41 on
Speech Recognition
on LibriSpeech test-clean
Speech Recognition
Audio and Speech Processing
NeMo: a toolkit for building AI applications using Neural Modules
NeMo (Neural Modules) is a Python framework-agnostic toolkit for creating AI applications through re-usability, abstraction, and composition.
Ranked #1 on
Speech Recognition
on Common Voice Spanish
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Stochastic Gradient Methods with Layer-wise Adaptive Moments for Training of Deep Networks
We propose NovoGrad, an adaptive stochastic gradient descent method with layer-wise gradient normalization and decoupled weight decay.
Jasper: An End-to-End Convolutional Neural Acoustic Model
In this paper, we report state-of-the-art results on LibriSpeech among end-to-end speech recognition models without any external training data.
Ranked #3 on
Speech Recognition
on Hub5'00 SwitchBoard
Training Neural Speech Recognition Systems with Synthetic Speech Augmentation
Building an accurate automatic speech recognition (ASR) system requires a large dataset that contains many hours of labeled speech samples produced by a diverse set of speakers.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Mixed-Precision Training for NLP and Speech Recognition with OpenSeq2Seq
We present OpenSeq2Seq - a TensorFlow-based toolkit for training sequence-to-sequence models that features distributed and mixed-precision training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
