Search Results for author: Awni Hannun

Found 33 papers, 17 papers with code

Semi-Supervised Speech Recognition via Local Prior Matching

1 code implementation24 Feb 2020 Wei-Ning Hsu, Ann Lee, Gabriel Synnaeve, Awni Hannun

For sequence transduction tasks like speech recognition, a strong structured prior model encodes rich information about the target space, implicitly ruling out invalid sequences by assigning them low probability.

Knowledge Distillation Language Modelling +2

Iterative Pseudo-Labeling for Speech Recognition

1 code implementation19 May 2020 Qiantong Xu, Tatiana Likhomanenko, Jacob Kahn, Awni Hannun, Gabriel Synnaeve, Ronan Collobert

In particular, IPL fine-tunes an existing model at each iteration using both labeled data and a subset of unlabeled data.

Ranked #11 on Speech Recognition on LibriSpeech test-other (using extra training data)

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

CrypTen: Secure Multi-Party Computation Meets Machine Learning

1 code implementation NeurIPS 2021 Brian Knott, Shobha Venkataraman, Awni Hannun, Shubho Sengupta, Mark Ibrahim, Laurens van der Maaten

To foster adoption of secure MPC in machine learning, we present CrypTen: a software framework that exposes popular secure MPC primitives via abstractions that are common in modern machine-learning frameworks, such as tensor computations, automatic differentiation, and modular neural networks.

BIG-bench Machine Learning Image Classification +4

Gradient Matching for Domain Generalization

2 code implementations ICLR 2022 Yuge Shi, Jeffrey Seely, Philip H. S. Torr, N. Siddharth, Awni Hannun, Nicolas Usunier, Gabriel Synnaeve

We perform experiments on both the Wilds benchmark, which captures distribution shift in the real world, as well as datasets in DomainBed benchmark that focuses more on synthetic-to-real transfer.

Domain Generalization

An End-to-End Architecture for Keyword Spotting and Voice Activity Detection

3 code implementations28 Nov 2016 Chris Lengerich, Awni Hannun

We propose a single neural network architecture for two tasks: on-line keyword spotting and voice activity detection.

Action Detection Activity Detection +2

A Fully Differentiable Beam Search Decoder

1 code implementation16 Feb 2019 Ronan Collobert, Awni Hannun, Gabriel Synnaeve

We demonstrate our approach scales by applying it to speech recognition, jointly training acoustic and word-level language models.

Language Modelling speech-recognition +1

Differentiable Weighted Finite-State Transducers

1 code implementation2 Oct 2020 Awni Hannun, Vineel Pratap, Jacob Kahn, Wei-Ning Hsu

We introduce a framework for automatic differentiation with weighted finite-state transducers (WFSTs) allowing them to be used dynamically at training time.

Handwriting Recognition speech-recognition +1

Measuring Data Leakage in Machine-Learning Models with Fisher Information

1 code implementation23 Feb 2021 Awni Hannun, Chuan Guo, Laurens van der Maaten

This information leaks either through the model itself or through predictions made by the model.

BIG-bench Machine Learning

The Trade-Offs of Private Prediction

1 code implementation9 Jul 2020 Laurens van der Maaten, Awni Hannun

This is problematic when the training data needs to remain private.

The History of Speech Recognition to the Year 2030

1 code implementation30 Jul 2021 Awni Hannun

Given the remarkable changes in the state of speech recognition over the previous decade, what can we expect over the coming decade?

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Fixes That Fail: Self-Defeating Improvements in Machine-Learning Systems

1 code implementation NeurIPS 2021 Ruihan Wu, Chuan Guo, Awni Hannun, Laurens van der Maaten

Machine-learning systems such as self-driving cars or virtual assistants are composed of a large number of machine-learning models that recognize image content, transcribe speech, analyze natural language, infer preferences, rank options, etc.

BIG-bench Machine Learning Object Detection +1

Learning Multiscale Features Directly From Waveforms

no code implementations31 Mar 2016 Zhenyao Zhu, Jesse H. Engel, Awni Hannun

Deep learning has dramatically improved the performance of speech recognition systems through learning hierarchies of features optimized for the task at hand.

speech-recognition Speech Recognition

Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions

no code implementations4 Apr 2019 Awni Hannun, Ann Lee, Qiantong Xu, Ronan Collobert

Coupled with a convolutional language model, our time-depth separable convolution architecture improves by more than 22% relative WER over the best previously reported sequence-to-sequence results on the noisy LibriSpeech test set.

Language Modelling Sequence-To-Sequence Speech Recognition +1

Privacy-Preserving Multi-Party Contextual Bandits

no code implementations11 Oct 2019 Awni Hannun, Brian Knott, Shubho Sengupta, Laurens van der Maaten

This paper considers a learning setting in which multiple parties aim to train a contextual bandit together in a private way: the parties aim to maximize the total reward but do not want to share any of the relevant information they possess with the other parties.

Multi-Armed Bandits Privacy Preserving

Secure multiparty computations in floating-point arithmetic

no code implementations9 Jan 2020 Chuan Guo, Awni Hannun, Brian Knott, Laurens van der Maaten, Mark Tygert, Ruiyu Zhu

Secure multiparty computations enable the distribution of so-called shares of sensitive data to multiple parties such that the multiple parties can effectively process the data while being unable to glean much information about the data (at least not without collusion among all parties to put back together all the shares).

Mathematical Proofs Privacy Preserving +1

Scaling Up Online Speech Recognition Using ConvNets

no code implementations27 Jan 2020 Vineel Pratap, Qiantong Xu, Jacob Kahn, Gilad Avidov, Tatiana Likhomanenko, Awni Hannun, Vitaliy Liptchinsky, Gabriel Synnaeve, Ronan Collobert

We design an online end-to-end speech recognition system based on Time-Depth Separable (TDS) convolutions and Connectionist Temporal Classification (CTC).

speech-recognition Speech Recognition

Massively Multilingual ASR: 50 Languages, 1 Model, 1 Billion Parameters

no code implementations6 Jul 2020 Vineel Pratap, Anuroop Sriram, Paden Tomasello, Awni Hannun, Vitaliy Liptchinsky, Gabriel Synnaeve, Ronan Collobert

We study training a single acoustic model for multiple languages with the aim of improving automatic speech recognition (ASR) performance on low-resource languages, and over-all simplifying deployment of ASR systems that support diverse languages.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Data Appraisal Without Data Sharing

no code implementations11 Dec 2020 Mimee Xu, Laurens van der Maaten, Awni Hannun

We show that in private, forward influence functions provide an appealing trade-off between high quality appraisal and required computation, in spite of label noise, class imbalance, and missing data.

The Role of Evolution in Machine Intelligence

no code implementations21 Jun 2021 Awni Hannun

I argue that the alternative, evolution, is important to the development of machine intelligence and underinvested in terms of research allocation.

Evolutionary Algorithms

Parallel Composition of Weighted Finite-State Transducers

no code implementations6 Oct 2021 Shubho Sengupta, Vineel Pratap, Awni Hannun

We benchmark our parallel algorithm on the composition of random graphs and the composition of graphs commonly used in speech recognition.

speech-recognition Speech Recognition

Transfer Learning for Structured Pruning under Limited Task Data

no code implementations10 Nov 2023 Lucio Dery, David Grangier, Awni Hannun

We propose a framework which combines structured pruning with transfer learning to reduce the need for task-specific data.

Transfer Learning

Specialized Language Models with Cheap Inference from Limited Domain Data

no code implementations2 Feb 2024 David Grangier, Angelos Katharopoulos, Pierre Ablin, Awni Hannun

Large language models have emerged as a versatile tool but are challenging to apply to tasks lacking large inference budgets and large in-domain training sets.

 Ranked #1 on Language Modelling on The Pile (Test perplexity metric)

Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.