Search Results for author: Huanru Henry Mao

Found 8 papers, 7 papers with code

Fine-Tuning Pre-trained Transformers into Decaying Fast Weights

1 code implementation9 Oct 2022 Huanru Henry Mao

Autoregressive Transformers are strong language models but incur O(T) complexity during per-token generation due to the self-attention mechanism.

A Survey on Self-supervised Pre-training for Sequential Transfer Learning in Neural Networks

no code implementations1 Jul 2020 Huanru Henry Mao

It involves first pre-training a model on a large amount of unlabeled data, then adapting the model to target tasks of interest.

Self-Supervised Learning Transfer Learning

Speech Recognition and Multi-Speaker Diarization of Long Conversations

3 code implementations16 May 2020 Huanru Henry Mao, Shuyang Li, Julian McAuley, Garrison Cottrell

Speech recognition (ASR) and speaker diarization (SD) models have traditionally been trained separately to produce rich conversation transcripts with speaker labels.

Data Augmentation speaker-diarization +3

ReZero is All You Need: Fast Convergence at Large Depth

13 code implementations10 Mar 2020 Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W. Cottrell, Julian McAuley

Deep networks often suffer from vanishing or exploding gradients due to inefficient signal propagation, leading to long training times or convergence difficulties.

Language Modelling

The NES Music Database: A multi-instrumental dataset with expressive performance attributes

2 code implementations12 Jun 2018 Chris Donahue, Huanru Henry Mao, Julian McAuley

Existing research on music generation focuses on composition, but often ignores the expressive performance characteristics required for plausible renditions of resultant pieces.

Music Generation

DeepJ: Style-Specific Music Generation

1 code implementation3 Jan 2018 Huanru Henry Mao, Taylor Shin, Garrison W. Cottrell

Recent advances in deep neural networks have enabled algorithms to compose music that is comparable to music composed by humans.

Sound Audio and Speech Processing

Cannot find the paper you are looking for? You can Submit a new open access paper.