Search Results for author:
Found 33 papers, 17 papers with code
Need a Small Specialized Language Model? Plan Early!
In the first scenario, we propose an effective solution based on importance sampling: we resample the pretraining set to imitate the specialization data and train a small model on it.
Ranked #1 on
Language Modelling
on The Pile
(Test perplexity metric)
Adaptive Training Distributions with Scalable Online Bilevel Optimization
Large neural networks pretrained on web-scale corpora are central to modern machine learning.
Transfer Learning for Structured Pruning under Limited Task Data
We propose a framework which combines structured pruning with transfer learning to reduce the need for task-specific data.
Flashlight: Enabling Innovation in Tools for Machine Learning
This is in part due to the difficulties involved in prototyping new computational paradigms with existing frameworks.
Star Temporal Classification: Sequence Classification with Partially Labeled Data
These experiments show that STC can recover most of the performance of supervised baseline when up to 70% of the labels are missing.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Parallel Composition of Weighted Finite-State Transducers
We benchmark our parallel algorithm on the composition of random graphs and the composition of graphs commonly used in speech recognition.
CrypTen: Secure Multi-Party Computation Meets Machine Learning
To foster adoption of secure MPC in machine learning, we present CrypTen: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks.
The History of Speech Recognition to the Year 2030
Given the remarkable changes in the state of speech recognition over the previous decade, what can we expect over the coming decade?
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
The Role of Evolution in Machine Intelligence
I argue that the alternative, evolution, is important to the development of machine intelligence and underinvested in terms of research allocation.
Gradient Matching for Domain Generalization
We perform experiments on both the Wilds benchmark, which captures distribution shift in the real world, as well as datasets in DomainBed benchmark that focuses more on synthetic-to-real transfer.
Ranked #6 on
Image Classification
on iWildCam2020-WILDS
Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems
Machine-learning systems such as self-driving cars or virtual assistants are composed of a large number of machine-learning models that recognize image content, transcribe speech, analyze natural language, infer preferences, rank options, etc.
Measuring Data Leakage in Machine-Learning Models with Fisher Information
This information leaks either through the model itself or through predictions made by the model.
Data Appraisal Without Data Sharing
We show that in private, forward influence functions provide an appealing trade-off between high quality appraisal and required computation, in spite of label noise, class imbalance, and missing data.
Differentiable Weighted Finite-State Transducers
We introduce a framework for automatic differentiation with weighted finite-state transducers (WFSTs) allowing them to be used dynamically at training time.
The Trade-Offs of Private Prediction
This is problematic when the training data needs to remain private.
Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters
We study training a single acoustic model for multiple languages with the aim of improving automatic speech recognition (ASR) performance on low-resource languages, and over-all simplifying deployment of ASR systems that support diverse languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Iterative Pseudo-Labeling for Speech Recognition
In particular, IPL fine-tunes an existing model at each iteration using both labeled data and a subset of unlabeled data.
Ranked #13 on
Speech Recognition
on LibriSpeech test-other
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Semi-Supervised Speech Recognition via Local Prior Matching
For sequence transduction tasks like speech recognition, a strong structured prior model encodes rich information about the target space, implicitly ruling out invalid sequences by assigning them low probability.
Ranked #51 on
Speech Recognition
on LibriSpeech test-other
Scaling Up Online Speech Recognition Using ConvNets
We design an online end-to-end speech recognition system based on Time-Depth Separable (TDS) convolutions and Connectionist Temporal Classification (CTC).
Secure multiparty computations in floating-point arithmetic
Secure multiparty computations enable the distribution of so-called shares of sensitive data to multiple parties such that the multiple parties can effectively process the data while being unable to glean much information about the data (at least not without collusion among all parties to put back together all the shares).
Certified Data Removal from Machine Learning Models
Good data stewardship requires removal of data at the request of the data's owner.
Lead2Gold: Towards exploiting the full potential of noisy transcriptions for speech recognition
The transcriptions used to train an Automatic Speech Recognition (ASR) system may contain errors.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Privacy-Preserving Multi-Party Contextual Bandits
This paper considers a learning setting in which multiple parties aim to train a contextual bandit together in a private way: the parties aim to maximize the total reward but do not want to share any of the relevant information they possess with the other parties.
Self-Supervised Speech Recognition via Local Prior Matching
We propose local prior matching (LPM), a self-supervised objective for speech recognition.
Self-Training for End-to-End Speech Recognition
We revisit self-training in the context of end-to-end speech recognition.
Word-level Speech Recognition with a Letter to Word Encoder
We propose a direct-to-word sequence model which uses a word network to learn word embeddings from letters.
Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions
Coupled with a convolutional language model, our time-depth separable convolution architecture improves by more than 22% relative WER over the best previously reported sequence-to-sequence results on the noisy LibriSpeech test set.
A Fully Differentiable Beam Search Decoder
We demonstrate our approach scales by applying it to speech recognition, jointly training acoustic and word-level language models.
wav2letter++: The Fastest Open-source Speech Recognition System
This paper introduces wav2letter++, the fastest open-source deep learning speech recognition framework.
An End-to-End Architecture for Keyword Spotting and Voice Activity Detection
We propose a single neural network architecture for two tasks: on-line keyword spotting and voice activity detection.
Learning Multiscale Features Directly From Waveforms
Deep learning has dramatically improved the performance of speech recognition systems through learning hierarchies of features optimized for the task at hand.
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
We show that an end-to-end deep learning approach can be used to recognize either English or Mandarin Chinese speech--two vastly different languages.
Deep Speech: Scaling up end-to-end speech recognition
We present a state-of-the-art speech recognition system developed using end-to-end deep learning.
