As a result, over 1% of the unprompted output of language models trained on these datasets is copied verbatim from the training data.
We then train agents to minimize the difference between the attention weights that they apply to the environment at each timestep, and the attention of other agents.
We analyze the reasons for this deficiency, and show that by imposing constraints on the training environment and introducing a model-based auxiliary loss we are able to obtain generalized social learning policies which enable agents to: i) discover complex skills that are not learned from single-agent training, and ii) adapt online to novel environments by taking cues from experts present in the new environment.
Recent advancements in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text.
We explore models for translating abstract musical ideas (scores, rhythms) into expressive performances using Seq2Seq and recurrent Variational Information Bottleneck (VIB) models.
Dramatic advances in generative models have resulted in near photographic quality for artificially rendered faces, animals and other objects in the natural world.
Generating musical audio directly with neural networks is notoriously difficult because it requires coherently modeling structure at many different timescales.
Music generation has generally been focused on either creating scores or interpreting them.
The Variational Autoencoder (VAE) has proven to be an effective model for producing semantically meaningful latent representations for natural data.
We use a Latent Constraints GAN (LC-GAN) to learn from the facial feedback of a small group of viewers, by optimizing the model to produce sketches that it predicts will lead to more positive facial expressions.
We advance the state of the art in polyphonic piano music transcription by using a deep convolutional and recurrent neural network which is trained to jointly predict onsets and frames.
GANs provide a framework for training generative models which mimic a data distribution.
Generative models in vision have seen rapid progress due to algorithmic improvements and the availability of high-quality image datasets.
Recurrent neural network models with an attention mechanism have proven to be extremely effective on a wide variety of sequence-to-sequence problems.
Ranked #17 on Speech Recognition on TIMIT
This paper proposes a general method for improving the structure and quality of sequences generated by a recurrent neural network (RNN), while maintaining information originally learned from data, as well as sample diversity.
The Indian Buffet Process is a Bayesian nonparametric approach that models objects as arising from an infinite number of latent factors.