Search Results for author:
Found 43 papers, 16 papers with code
TryOnDiffusion: A Tale of Two UNets
Given two images depicting a person and a garment worn by another person, our goal is to generate a visualization of how the garment might look on the input person.
Noise2Music: Text-conditioned Music Generation with Diffusion Models
We introduce Noise2Music, where a series of diffusion models is trained to generate high-quality 30-second music clips from text prompts.
Ranked #6 on
Text-to-Music Generation
on MusicCaps
(FAD metric)
Character-Aware Models Improve Visual Text Rendering
In the text-only domain, we find that character-aware models provide large gains on a novel spelling task (WikiSpell).
Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
Through extensive human evaluation on EditBench, we find that object-masking during training leads to across-the-board improvements in text-image alignment -- such that Imagen Editor is preferred over DALL-E 2 and Stable Diffusion -- and, as a cohort, these models are better at object-rendering than text-rendering, and handle material/color/size attributes better than count/shape attributes.
Novel View Synthesis with Diffusion Models
We demonstrate that stochastic conditioning significantly improves the 3D consistency of a naive sampler for an image-to-image diffusion model, which involves conditioning on a single fixed view.
Imagen Video: High Definition Video Generation with Diffusion Models
We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models.
Ranked #1 on
Video Generation
on LAION-400M
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding.
Ranked #16 on
Text-to-Image Generation
on MS COCO
(using extra training data)
Video Diffusion Models
Generating temporally coherent high fidelity video is an important milestone in generative modeling research.
Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality
We introduce Differentiable Diffusion Sampler Search (DDSS): a method that optimizes fast samplers for any pre-trained diffusion model by differentiating through sample quality scores.
Palette: Image-to-Image Diffusion Models
We expect this standardized evaluation protocol to play a role in advancing image-to-image translation research.
Ranked #1 on
Colorization
on ImageNet ctest10k
Optimizing Few-Step Diffusion Samplers by Gradient Descent
We propose Generalized Gaussian Diffusion Processes (GGDP), a family of non-Markovian samplers for diffusion models, and we show how to improve the generated samples of pre-trained DDPMs by optimizing the degrees of freedom of the GGDP sampler family with respect to a perceptual loss.
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a million hours of audio.
Ranked #1 on
Speech Recognition
on Common Voice
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis
The model takes an input phoneme sequence, and through an iterative refinement process, generates an audio waveform.
Learning to Efficiently Sample from Diffusion Probabilistic Models
Key advantages of DDPMs include ease of training, in contrast to generative adversarial networks, and speed of generation, in contrast to autoregressive models.
Cascaded Diffusion Models for High Fidelity Image Generation
We show that cascaded diffusion models are capable of generating high fidelity images on the class-conditional ImageNet generation benchmark, without any assistance from auxiliary image classifiers to boost sample quality.
Ranked #9 on
Image Generation
on ImageNet 64x64
Image Super-Resolution via Iterative Refinement
We present SR3, an approach to image Super-Resolution via Repeated Refinement.
Pushing the Limits of Non-Autoregressive Speech Recognition
We combine recent advancements in end-to-end speech recognition to non-autoregressive automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network
We present SpeechStew, a speech recognition model that is trained on a combination of various publicly available speech recognition datasets: AMI, Broadcast News, Common Voice, LibriSpeech, Switchboard/Fisher, Tedlium, and Wall Street Journal.
Ranked #1 on
Speech Recognition
on Switchboard CallHome
Multichannel Generative Language Model: Learning All Possible Factorizations Within and Across Channels
MGLM is a generative joint distribution model over channels.
WaveGrad: Estimating Gradients for Waveform Generation
This paper introduces WaveGrad, a conditional model for waveform generation which estimates gradients of the data density.
Non-Autoregressive Machine Translation with Latent Alignments
In addition, we adapt the Imputer model for non-autoregressive machine translation and demonstrate that Imputer with just 4 generation steps can match the performance of an autoregressive Transformer baseline.
Imputer: Sequence Modelling via Imputation and Dynamic Programming
This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations.
Insertion-Deletion Transformer
We propose the Insertion-Deletion Transformer, a novel transformer-based neural architecture and training method for sequence generation.
SpecAugment on Large Scale Datasets
Recently, SpecAugment, an augmentation scheme for automatic speech recognition that acts directly on the spectrogram of input utterances, has shown to be highly effective in enhancing the performance of end-to-end networks on public datasets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
An Empirical Study of Generation Order for Machine Translation
In this work, we present an empirical study of generation order for machine translation.
Big Bidirectional Insertion Representations for Documents
The Insertion Transformer is well suited for long form text generation due to its parallel generation capabilities, requiring $O(\log_2 n)$ generation steps to generate $n$ tokens.
Multichannel Generative Language Models
For conditional generation, the model is given a fully observed channel, and generates the k-1 channels in parallel.
Speaker-Targeted Audio-Visual Models for Speech Recognition in Cocktail-Party Environments
Speech recognition in cocktail-party environments remains a significant challenge for state-of-the-art speech recognition systems, as it is extremely difficult to extract an acoustic signal of an individual speaker from a background of overlapping speech with similar frequency and temporal characteristics.
KERMIT: Generative Insertion-Based Modeling for Sequences
During training, one can feed KERMIT paired data $(x, y)$ to learn the joint distribution $p(x, y)$, and optionally mix in unpaired data $x$ or $y$ to refine the marginals $p(x)$ or $p(y)$.
Ranked #38 on
Machine Translation
on WMT2014 English-German
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
On LibriSpeech, we achieve 6. 8% WER on test-other without the use of a language model, and 5. 8% WER with shallow fusion with a language model.
Ranked #1 on
Speech Recognition
on Hub5'00 SwitchBoard
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
Insertion Transformer: Flexible Sequence Generation via Insertion Operations
We present the Insertion Transformer, an iterative, partially autoregressive model for sequence generation based on insertion operations.
Privacy Partitioning: Protecting User Data During the Deep Learning Inference Phase
We present a practical method for protecting data during the inference phase of deep learning based on bipartite topology threat modeling and an interactive adversarial deep network construction.
Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytes
We present two end-to-end models: Audio-to-Byte (A2B) and Byte-to-Audio (B2A), for multilingual speech recognition and synthesis.
Optimal Completion Distillation for Sequence Learning
We present Optimal Completion Distillation (OCD), a training procedure for optimizing sequence to sequence models based on edit distance.
InferLite: Simple Universal Sentence Representations from Natural Language Inference Data
Natural language inference has been shown to be an effective supervised task for learning generic sentence embeddings.
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search
We introduce Picturebook, a large-scale lookup operation to ground language via {`}snapshots{'} of our physical world accessed through image search.
Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LM
The CTC network sits on top of the encoder and is jointly trained with the attention-based decoder.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Very Deep Convolutional Networks for End-to-End Speech Recognition
Sequence-to-sequence models have shown success in end-to-end speech recognition.
Listen, Attend and Spell
Unlike traditional DNN-HMM models, this model learns all the components of a speech recognizer jointly.
Deep Recurrent Neural Networks for Acoustic Modelling
We present a novel deep Recurrent Neural Network (RNN) model for acoustic modelling in Automatic Speech Recognition (ASR).
Ranked #11 on
Speech Recognition
on WSJ eval92
Transferring Knowledge from a RNN to a DNN
The small DNN trained on the soft RNN alignments achieved a 3. 93 WER on the Wall Street Journal (WSJ) eval92 task compared to a baseline 4. 54 WER or more than 13% relative improvement.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
