Search Results for author:
Found 186 papers, 107 papers with code
Unsupervised Dependency Graph Network
We introduce a new model, the Unsupervised Dependency Graph Network (UDGN), that can induce dependency structures from raw corpora and the masked language modeling task.
Meta-Value Learning: a General Framework for Learning with Learning Awareness
Gradient-based learning in multi-agent systems is difficult because the gradient derives from a first-order model which does not account for the interaction between agents' learning processes.
Bigger, Better, Faster: Human-level Atari with human-level efficiency
We introduce a value-based RL agent, which we call BBF, that achieves super-human performance in the Atari 100K benchmark.
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
Combinatorial optimization (CO) problems are often NP-hard and thus out of reach for exact algorithms, making them a tempting domain to apply machine learning methods.
Distributional GFlowNets with Quantile Flows
Generative Flow Networks (GFlowNets) are a new family of probabilistic samplers where an agent learns a stochastic policy for generating complex combinatorial structure through a series of decision-making steps.
Versatile Energy-Based Probabilistic Models for High Energy Physics
As a classical generative modeling approach, energy-based models have the natural advantage of flexibility in the form of the energy function.
Teaching Algorithmic Reasoning via In-context Learning
Large language models (LLMs) have shown increasing in-context learning capabilities through scaling up model and data size.
On the Compositional Generalization Gap of In-Context Learning
In this work, we look at the gap between the in-distribution (ID) and out-of-distribution (OOD) performance of such models in semantic parsing tasks with in-context learning.
Generative Augmented Flow Networks
We specify intermediate rewards by intrinsic motivation to tackle the exploration problem in sparse reward environments.
Latent State Marginalization as a Low-cost Approach for Improving Exploration
While the maximum entropy (MaxEnt) reinforcement learning (RL) framework -- often touted for its exploration and robustness capabilities -- is usually motivated from a probabilistic perspective, the use of deep probabilistic models has not gained much traction in practice due to their inherent complexity.
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
In this work, we study the URLB and propose a new method to solve it, using unsupervised model-based RL, for pre-training the agent, and a task-aware fine-tuning strategy combined with a new proposed hybrid planner, Dyna-MPC, to adapt the agent for downstream tasks.
Riemannian Diffusion Models
In this work, we generalize continuous-time diffusion models to arbitrary Riemannian manifolds and derive a variational framework for likelihood estimation.
R-MelNet: Reduced Mel-Spectral Modeling for Neural TTS
This paper introduces R-MelNet, a two-part autoregressive architecture with a frontend based on the first tier of MelNet and a backend WaveRNN-style audio decoder for neural text-to-speech synthesis.
Building Robust Ensembles via Margin Boosting
Consequently, an emerging line of work has focused on learning an ensemble of neural networks to defend against adversarial attacks.
Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress
To address these issues, we present reincarnating RL as an alternative workflow or class of problem settings, where prior computational work (e. g., learned policies) is reused or transferred between design iterations of an RL agent, or from one RL agent to another.
Expressiveness and Learnability: A Unifying View for Evaluating Self-Supervised Learning
We propose to use the Intrinsic Dimension (ID) to assess expressiveness and introduce Cluster Learnability (CL) to assess learnability.
Cascaded Video Generation for Videos In-the-Wild
Videos can be created by first outlining a global view of the scene and then adding local details.
The Primacy Bias in Deep Reinforcement Learning
This work identifies a common flaw of deep reinforcement learning (RL) algorithms: a tendency to rely on early interactions and ignore useful evidence encountered later.
Ranked #4 on
Atari Games 100k
on Atari 100k
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into $L$ simplices of $V$ dimensions each using a softmax operation.
Generative Flow Networks for Discrete Probabilistic Modeling
We present energy-based generative flow networks (EB-GFN), a novel probabilistic modeling algorithm for high-dimensional discrete data.
Fortuitous Forgetting in Connectionist Networks
Forgetting is often seen as an unwanted characteristic in both human and machine learning.
Invariant Representation Driven Neural Classifier for Anti-QCD Jet Tagging
We leverage representation learning and the inductive bias in neural-net-based Standard Model jet classification tasks, to detect non-QCD signal jets.
MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modeling
Musical expression requires control of both what notes are played, and how they are performed.
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
In this paper, we discuss how the implicit regularization effect of SGD seen in supervised learning could in fact be harmful in the offline deep RL setting, leading to poor generalization and degenerate feature representations.
Multi-label Iterated Learning for Image Classification with Label Ambiguity
We also show that MILe is effective reducing label noise, achieving state-of-the-art performance on real-world large-scale noisy data such as WebVision.
Ranked #6 on
Image Classification
on WebVision-1000
Chunked Autoregressive GAN for Conditional Waveform Synthesis
We show that simple pitch and periodicity conditioning is insufficient for reducing this error relative to using autoregression.
Unifying Likelihood-free Inference with Black-box Optimization and Beyond
Black-box optimization formulations for biological sequence design have drawn recent attention due to their promising potential impact on the pharmaceutical industry.
Inducing Reusable Skills From Demonstrations with Option-Controller Network
With the isolation of information and the synchronous calling mechanism, we can impose a division of works between the controller and options in an end-to-end training regime.
Learning to Dequantise with Truncated Flows
Addtionally, since the support of the marginal $q(z)$ is bounded and the support of prior $p(z)$ is not, we propose renormalising the prior distribution over the support of $q(z)$.
INFERNO: Inferring Object-Centric 3D Scene Representations without Supervision
Each object representation defines a localized neural radiance field that is used to generate 2D views of the scene through a differentiable rendering process.
Ranked #6 on
Video Object Tracking
on CATER
Overcoming Label Ambiguity with Multi-label Iterated Learning
This ambiguity biases models towards a single prediction, which could result in the suppression of classes that tend to co-occur in the data.
Learnability and Expressiveness in Self-Supervised Learning
In this work, we argue that representations induced by self-supervised learning (SSL) methods should both be expressive and learnable.
On Bonus-Based Exploration Methods in the Arcade Learning Environment
Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration problems such as Montezuma's Revenge (Bellemare et al., 2016).
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks, ignoring the statistical uncertainty implied by the use of a finite number of training runs.
Pretraining Representations for Data-Efficient Reinforcement Learning
Data efficiency is a key challenge for deep reinforcement learning.
Ranked #2 on
Atari Games 100k
on Atari 100k
(using extra training data)
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?
Can models with particular structure avoid being biased towards spurious correlation in out-of-distribution (OOD) generalization?
A Variational Perspective on Diffusion-Based Generative Models and Score Matching
Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by Song et al. (2021), bridging the theoretical gap.
Hierarchical Video Generation for Complex Data
Inspired by this we propose a hierarchical model for video generation which follows a coarse to fine approach.
Understanding by Understanding Not: Modeling Negation in Language Models
To improve language models in this regard, we propose to augment the language modeling objective with an unlikelihood objective that is based on negated generic sentences from a raw text corpus.
Iterated learning for emergent systematicity in VQA
Although neural module networks have an architectural bias towards compositionality, they require gold standard layouts to generalize systematically in practice.
Touch-based Curiosity for Sparse-Reward Tasks
Robots in many real-world settings have access to force/torque sensors in their gripper and tactile sensing is often necessary in tasks that involve contact-rich motion.
Learning Task Decomposition with Ordered Memory Policy Network
The discovered subtask hierarchy could be used to perform task decomposition, recovering the subtask boundaries in an unstruc-tured demonstration.
Pretraining Reward-Free Representations for Data-Efficient Reinforcement Learning
Data efficiency poses a major challenge for deep reinforcement learning.
Continuous Coordination As a Realistic Scenario for Lifelong Learning
Its large strategy space makes it a desirable environment for lifelong RL tasks.
Emergent Communication under Competition
First, we show that communication is proportional to cooperation, and it can occur for partially competitive scenarios using standard learning algorithms.
Neural Approximate Sufficient Statistics for Likelihood-free Inference
We consider the fundamental problem of how to automatically construct summary statistics for likelihood-free inference where the evaluation of likelihood function is intractable but sampling / simulating data from the model is possible.
SSW-GAN: Scalable Stage-wise Training of Video GANs
Current state-of-the-art generative models for videos have high computational requirements that impede high resolution generations beyond a few frames.
Systematic generalisation with group invariant predictions
We consider situations where the presence of dominant simpler correlations with the target variable in a training set can cause an SGD-trained neural network to be less reliant on more persistently-correlating complex features.
Learning Task Decomposition with Order-Memory Policy Network
Many complex real-world tasks are composed of several levels of sub-tasks.
Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
Flow-based models are powerful tools for designing probabilistic models with tractable density.
StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling
There are two major classes of natural language grammar -- the dependency grammar that models one-to-one correspondences between words and the constituency grammar that models the assembly of one or several corresponded words.
Bijective-Contrastive Estimation
In this work, we propose Bijective-Contrastive Estimation (BCE), a classification-based learning criterion for energy-based models.
Gradient Starvation: A Learning Proclivity in Neural Networks
We identify and formalize a fundamental gradient descent phenomenon resulting in a learning proclivity in over-parameterized neural networks.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
Unsupervised Learning of Dense Visual Representations
VADeR provides a natural representation for dense prediction tasks and transfers well to downstream tasks.
NU-GAN: High resolution neural upsampling with GAN
In this paper, we propose NU-GAN, a new method for resampling audio from lower to higher sampling rates (upsampling).
Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
In the present work, we propose a new syntax-aware language model: Syntactic Ordered Memory (SOM).
Neural Approximate Sufficient Statistics for Implicit Models
We consider the fundamental problem of how to automatically construct summary statistics for implicit generative models where the evaluation of the likelihood function is intractable, but sampling data from the model is possible.
Recursive Top-Down Production for Sentence Generation with Latent Trees
We model the recursive production property of context-free grammars for natural and synthetic languages.
Supervised Seeded Iterated Learning for Interactive Language Learning
Language drift has been one of the major obstacles to train language models through interaction.
Integrating Categorical Semantics into Unsupervised Domain Translation
While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical semantic features could broaden its applicability.
Data-Efficient Reinforcement Learning with Self-Predictive Representations
We further improve performance by adding data augmentation to the future prediction loss, which forces the agent's representations to be consistent across multiple views of an observation.
Ranked #5 on
Atari Games 100k
on Atari 100k
Generative Compositional Augmentations for Scene Graph Prediction
However, test images might contain zero- and few-shot compositions of objects and relationships, e. g. <cup, on, surfboard>.
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation
Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary continuous random variable.
Graph Density-Aware Losses for Novel Compositions in Scene Graph Generation
We show that such models can suffer the most in their ability to generalize to rare compositions, evaluating two different models on the Visual Genome dataset and its more recent, improved version, GQA.
A Large-Scale, Open-Domain, Mixed-Interface Dialogue-Based ITS for STEM
We present Korbit, a large-scale, open-domain, mixed-interface, dialogue-based intelligent tutoring system (ITS).
Countering Language Drift with Seeded Iterated Learning
At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion.
Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation
We propose Pix2Shape, an approach to solve this problem with four components: (i) an encoder that infers the latent 3D representation from an image, (ii) a decoder that generates an explicit 2. 5D surfel-based reconstruction of a scene from the latent code (iii) a differentiable renderer that synthesizes a 2D image from the surfel representation, and (iv) a critic network trained to discriminate between images generated by the decoder-renderer and those from a training distribution.
Out-of-Distribution Generalization via Risk Extrapolation (REx)
Distributional shift is one of the major obstacles when transferring machine learning prediction systems from the lab to the real world.
Solving ODE with Universal Flows: Approximation Theory for Flow-Based Models
Normalizing flows are powerful invertible probabilistic models that can be used to translate two probability distributions, in a way that allows us to efficiently track the change of probability density.
Augmented Normalizing Flows: Bridging the Gap Between Generative Flows and Latent Variable Models
In this work, we propose a new family of generative flows on an augmented data space, with an aim to improve expressivity without drastically increasing the computational cost of sampling and evaluation of a lower bound on the likelihood.
Ranked #6 on
Image Generation
on CelebA 256x256
On Bonus Based Exploration Methods In The Arcade Learning Environment
Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration problems such as Montezuma's Revenge (Bellemare et al., 2016).
CLOSURE: Assessing Systematic Generalization of CLEVR Models
In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs.
What Do Compressed Deep Neural Networks Forget?
However, this measure of performance conceals significant differences in how different classes and images are impacted by model compression techniques.
Ordered Memory
Inspired by Ordered Neurons (Shen et al., 2018), we introduce a new attention-based mechanism and use its cumulative probability to control the writing and erasing operation of the memory.
Icentia11K: An Unsupervised Representation Learning Dataset for Arrhythmia Subtype Discovery
We release the largest public ECG dataset of continuous raw signals for representation learning containing 11 thousand patients and 2 billion labelled beats.
MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
In this paper, we show that it is possible to train GANs reliably to generate high quality coherent waveforms by introducing a set of architectural changes and simple training techniques.
Selective Brain Damage: Measuring the Disparate Impact of Model Pruning
Neural network pruning techniques have demonstrated it is possible to remove the majority of weights in a network with surprisingly little degradation to top-1 test set accuracy.
{COMPANYNAME}11K: An Unsupervised Representation Learning Dataset for Arrhythmia Subtype Discovery
We release the largest public ECG dataset of continuous raw signals for representation learning containing over 11k patients and 2 billion labelled beats.
Selfish Emergent Communication
Current literature in machine learning holds that unaligned, self-interested agents do not learn to use an emergent communication channel.
No Press Diplomacy: Modeling Multi-Agent Gameplay
Diplomacy is a seven-player non-stochastic, non-cooperative game, where agents acquire resources through a mix of teamwork and betrayal.
VideoNavQA: Bridging the Gap between Visual and Embodied Question Answering
The goal of this dataset is to assess question-answering performance from nearly-ideal navigation paths, while considering a much more complete variety of questions than current instantiations of the EQA task.
Detecting semantic anomalies
We critically appraise the recent interest in out-of-distribution (OOD) detection and question the practical relevance of existing benchmarks.
Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
This paper provides an empirical evaluation of recently developed exploration algorithms within the Arcade Learning Environment (ALE).
Adversarial Computation of Optimal Transport Maps
We show that during training, our generator follows the $W_2$-geodesic between the initial and the target distributions.
Investigating Biases in Textual Entailment Datasets
The ability to understand logical relationships between sentences is an important task in language understanding.
Stochastic Neural Network with Kronecker Flow
Recent advances in variational inference enable the modelling of highly structured joint distributions, but are limited in their capacity to scale to the high-dimensional setting of stochastic neural networks.
Note on the bias and variance of variational inference
In this note, we study the relationship between the variational gap and the variance of the (log) likelihood ratio.
Batch weight for domain adaptation with mass shift
We also provide rigorous probabilistic setting for domain transfer and new simplified objective for training transfer networks, an alternative to complex, multi-component loss functions used in the current state-of-the art image-to-image translation models.
Hierarchical Importance Weighted Autoencoders
We believe a joint proposal has the potential of reducing the number of redundant samples, and introduce a hierarchical structure to induce correlation.
Unsupervised one-to-many image translation
We perform completely unsupervised one-sided image to image translation between a source domain $X$ and a target domain $Y$ such that we preserve relevant underlying shared semantics (e. g., class, size, shape, etc).
EnGAN: Latent Space MCMC and Maximum Entropy Generators for Energy-based Models
Unsupervised learning is about capturing dependencies between variables and is driven by the contrast between the probable vs improbable configurations of these variables, often either via a generative model which only samples probable ones or with an energy function (unnormalized log-density) which is low for probable ones and high for improbable ones.
Manifold Mixup: Learning Better Representations by Interpolating Hidden States
Because the hidden states are learned, this has an important effect of encouraging the hidden states for a class to be concentrated in such a way so that interpolations within the same class or between two different classes do not intersect with the real data points from other classes.
Pix2Scene: Learning Implicit 3D Representations from Images
Modelling 3D scenes from 2D images is a long-standing problem in computer vision with implications in, e. g., simulation and robotics.
Improved Conditional VRNNs for Video Prediction
To address this issue, we propose to increase the expressiveness of the latent distributions and to use higher capacity likelihood models.
Ranked #2 on
Video Prediction
on Cityscapes 128x128
Counterpoint by Convolution
Machine learning models of music typically break up the task of composition into a chronological process, composing a piece of music in a single pass from beginning to end.
Ranked #4 on
Music Modeling
on JSB Chorales
Maximum Entropy Generators for Energy-Based Models
Maximum likelihood estimation of energy-based models is a challenging problem due to the intractability of the log-likelihood gradient.
Deep Generative Modeling of LiDAR Data
In this work, we show that one can adapt deep generative models for this task by unravelling lidar scans into a 2D point map.
Systematic Generalization: What Is Required and Can It Be Learned?
Numerous models for grounded language understanding have been recently proposed, including (i) generic models that can be easily adapted to any given task and (ii) intuitively appealing modular models that require background knowledge to be instantiated.
Planning in Dynamic Environments with Conditional Autoregressive Models
We demonstrate the use of conditional autoregressive generative models (van den Oord et al., 2016a) over a discrete latent space (van den Oord et al., 2017b) for forward planning with MCTS.
Harmonic Recomposition using Conditional Autoregressive Modeling
We demonstrate a conditional autoregressive pipeline for efficient music recomposition, based on methods presented in van den Oord et al.(2017).
Representation Mixing for TTS Synthesis
Recent character and phoneme-based parametric TTS systems using deep learning have shown strong performance in natural speech generation.
Blindfold Baselines for Embodied QA
We explore blindfold (question-only) baselines for Embodied Question Answering.
Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
When a larger constituent ends, all of the smaller constituents that are nested within it must also be closed.
W2GAN: RECOVERING AN OPTIMAL TRANSPORT MAP WITH A GAN
In this work we address the converse question: is it possible to recover an optimal map in a GAN fashion?
Convergence Properties of Deep Neural Networks on Separable Data
While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely misunderstood.
On Difficulties of Probability Distillation
Probability distillation has recently been of interest to deep learning practitioners as it presents a practical solution for sampling from autoregressive models for deployment in real-time applications.
On the Learning Dynamics of Deep Neural Networks
While a lot of progress has been made in recent years, the dynamics of learning in deep nonlinear neural networks remain to this day largely misunderstood.
Improving Explorability in Variational Inference with Annealed Variational Objectives
Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned.
Approximate Exploration through State Abstraction
Next, we show how a given density model can be related to an abstraction and that the corresponding pseudo-count bonus can act as a substitute in MBIE-EB combined with this abstraction, but may lead to either under- or over-exploration.
Visual Reasoning with Multi-hop Feature Modulation
Recent breakthroughs in computer vision and natural language processing have spurred interest in challenging multi-modal tasks such as visual question-answering and visual dialogue.
Mutual Information Neural Estimation
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks.
On the Spectral Bias of Neural Networks
Neural networks are known to be a class of highly expressive functions able to fit even random input-output mappings with $100\%$ accuracy.
Learning Distributed Representations from Reviews for Collaborative Filtering
However, interestingly, the greater modeling power offered by the recurrent neural network appears to undermine the model's ability to act as a regularizer of the product representations.
Manifold Mixup: Better Representations by Interpolating Hidden States
Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples.
Ranked #19 on
Image Classification
on OmniBenchmark
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
In this work, we propose a novel constituency parsing scheme.
Neural Autoregressive Flows
Normalizing flows and autoregressive models have been successfully combined to produce state-of-the-art results in density estimation, via Masked Autoregressive Flows (MAF), and to accelerate state-of-the-art WaveNet-based speech synthesis to 20x faster than real-time, via Inverse Autoregressive Flows (IAF).
Generating Contradictory, Neutral, and Entailing Sentences
Learning distributed sentence representations remains an interesting problem in the field of Natural Language Processing (NLP).
Augmented CycleGAN: Learning Many-to-Many Mappings from Unpaired Data
Learning inter-domain mappings from unpaired data can improve performance in structured prediction tasks, such as image segmentation, by reducing the need for paired data.
Hierarchical Adversarially Learned Inference
We propose a novel hierarchical generative model with a simple Markovian structure and a corresponding inference model.
MINE: Mutual Information Neural Estimation
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks.
Learning Generative Models with Locally Disentangled Latent Factors
One of the most successful techniques in generative models has been decomposing a complicated generation task into a series of simpler generation tasks.
GibbsNet: Iterative Adversarial Inference for Deep Graphical Models
Directed latent variable models that formulate the joint distribution as $p(x, z) = p(z) p(x \mid z)$ have the advantage of fast and exact sampling.
HoME: a Household Multimodal Environment
We introduce HoME: a Household Multimodal Environment for artificial agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context.
Neural Language Modeling by Jointly Learning Syntax and Lexicon
In this paper, We propose a novel neural language model, called the Parsing-Reading-Predict Networks (PRPN), that can simultaneously induce the syntactic structure from unannotated sentences and leverage the inferred structure to learn a better language model.
Ranked #12 on
Constituency Grammar Induction
on PTB Diagnostic ECG Database
(Max F1 (WSJ) metric)
Bayesian Hypernetworks
We study Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks.
Learnable Explicit Density for Continuous Latent Space and Variational Inference
In this paper, we study two aspects of the variational autoencoder (VAE): the prior distribution over the latent variables and its corresponding posterior.
FiLM: Visual Reasoning with a General Conditioning Layer
We introduce a general-purpose conditioning method for neural networks called FiLM: Feature-wise Linear Modulation.
Ranked #3 on
Visual Question Answering (VQA)
on CLEVR-Humans
Image Retrieval with Multi-Modal Query
Visual Question Answering (VQA)
+1
Self-organized Hierarchical Softmax
We propose a new self-organizing hierarchical softmax formulation for neural-network-based language models over large vocabularies.
Learning Visual Reasoning Without Strong Priors
Previous work has operated under the assumption that visual reasoning calls for a specialized architecture, but we show that a general architecture with proper conditioning can learn to visually reason effectively.
Modulating early visual processing by language
It is commonly assumed that language refers to high-level visual concepts while leaving low-level visual processing unaffected.
A Closer Look at Memorization in Deep Networks
We examine the role of memorization in deep learning, drawing connections to capacity, generalization, and adversarial robustness.
Adversarial Generation of Natural Language
Generative Adversarial Networks (GANs) have gathered a lot of attention from the computer vision community, yielding impressive results for image generation.
Improved Training of Wasserstein GANs
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability.
Ranked #3 on
Image Generation
on CAT 256x256
End-to-end optimization of goal-driven and visually grounded dialogue systems
End-to-end design of dialogue systems has recently become a popular research topic thanks to powerful tools such as encoder-decoder architectures for sequence-to-sequence learning.
Calibrating Energy-based Generative Adversarial Networks
In this paper, we propose to equip Generative Adversarial Networks with the ability to produce direct energy estimates for samples. Specifically, we propose a flexible adversarial training framework, and prove this framework not only ensures the generator converges to the true data distribution, but also enables the discriminator to retain the density information at the global optimal.
Ranked #17 on
Conditional Image Generation
on CIFAR-10
(Inception score metric)
Towards End-to-End Speech Recognition with Deep Convolutional Neural Networks
Meanwhile, Connectionist Temporal Classification (CTC) with Recurrent Neural Networks (RNNs), which is proposed for labeling unsegmented sequences, makes it feasible to train an end-to-end speech recognition system instead of hybrid settings.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
In this paper we propose a novel model for unconditional audio generation based on generating one audio sample at a time.
Generalizable Features From Unsupervised Learning
In this work, we explore the potential of unsupervised learning to find features that promote better generalization to settings outside the supervised training distribution.
A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Images
Colorectal cancer (CRC) is the third cause of cancer death worldwide.
Piecewise Latent Variables for Neural Variational Text Processing
Advances in neural variational inference have facilitated the learning of powerful directed graphical models with continuous latent variables, such as variational autoencoders.
A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering
In addition to presenting statistics and a description of the dataset, we perform a detailed analysis of 5 different models' predictions, and compare these with human performance.
GuessWhat?! Visual object discovery through multi-modal dialogue
Our key contribution is the collection of a large-scale dataset consisting of 150K human-played games with a total of 800K visual question-answer pairs on 66K images.
PixelVAE: A Latent Variable Model for Natural Images
Natural image modeling is a landmark challenge of unsupervised learning.
Professor Forcing: A New Algorithm for Training Recurrent Networks
We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps.
An Actor-Critic Algorithm for Sequence Prediction
We present an approach to training neural networks to generate sequences using actor-critic methods from reinforcement learning (RL).
Ranked #8 on
Machine Translation
on IWSLT2015 English-German
First Result on Arabic Neural Machine Translation
Neural machine translation has become a major alternative to widely used phrase-based statistical machine translation.
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
We propose zoneout, a novel method for regularizing RNNs.
Adversarially Learned Inference
We introduce the adversarially learned inference (ALI) model, which jointly learns a generation network and an inference network using an adversarial process.
Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generation
We introduce the multiresolution recurrent neural network, which extends the sequence-to-sequence framework to model natural language generation as two parallel discrete stochastic processes: a sequence of high-level coarse tokens, and a sequence of natural language tokens.
Ranked #1 on
Dialogue Generation
on Ubuntu Dialogue (Activity)
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Sequential data often possesses a hierarchical structure with complex dependencies between subsequences, such as found between the utterances in a dialogue.
Movie Description
In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Recurrent Batch Normalization
We propose a reparameterization of LSTM that brings the benefits of batch normalization to recurrent neural networks.
Ranked #20 on
Language Modelling
on Text8
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Over the past decade, large-scale supervised learning corpora have enabled machine learning researchers to make substantial advances.
Discriminative Regularization for Generative Models
We propose to take advantage of this by using the representations from discriminative classifiers to augment the objective function corresponding to a generative model.
Dynamic Capacity Networks
The low-capacity sub-networks are applied across most of the input, but also provide a guide to select a few portions of the input on which to apply the high-capacity sub-networks.
ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
Moreover, ReNet layers are stacked on top of pre-trained convolutional layers, benefiting from generic local features.
Ranked #17 on
Semantic Segmentation
on CamVid
Variance Reduction in SGD by Distributed Importance Sampling
This leads the model to update using an unbiased estimate of the gradient which also has minimum variance when the sampling proposal is proportional to the L2-norm of the gradient.
Delving Deeper into Convolutional Networks for Learning Video Representations
We propose an approach to learn spatio-temporal features in videos from intermediate visual representations we call "percepts" using Gated-Recurrent-Unit Recurrent Networks (GRUs). Our method relies on percepts that are extracted from all level of a deep convolutional network trained on the large ImageNet dataset.
A Controller-Recognizer Framework: How necessary is recognition for control?
Recently there has been growing interest in building active visual object recognizers, as opposed to the usual passive recognizers which classifies a given static image into a predefined set of object categories.
Task Loss Estimation for Sequence Prediction
Our idea is that this score can be interpreted as an estimate of the task loss, and that the estimation error may be used as a consistent surrogate loss.