Search Results for author:
Found 426 papers, 215 papers with code
Learning to Navigate in Synthetically Accessible Chemical Space Using Reinforcement Learning
In this work, we propose a novel reinforcement learning (RL) setup for drug discovery that addresses this challenge by embedding the concept of synthetic accessibility directly into the de novo compound design system.
Learning Neural Causal Models with Active Interventions
Discovering causal structures from data is a challenging inference problem of fundamental importance in all areas of science.
Discrete-Valued Neural Communication
Deep learning has advanced from fully connected architectures to structured models organized into components, e. g., the transformer composed of positional elements, modular architectures divided into slots, and graph neural nets made up of nodes.
The Causal-Neural Connection: Expressiveness, Learnability, and Inference
Given this property, one may be tempted to surmise that a collection of neural nets is capable of learning any SCM by training on data generated by that SCM.
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
A central goal for AI and causality is thus the joint discovery of abstract representations and causal structure.
Predicting Unreliable Predictions by Shattering a Neural Network
Piecewise linear neural networks can be split into subfunctions, each with its own activation pattern, domain, and empirical error.
Variational Causal Networks: Approximate Bayesian Inference over Causal Structures
However, a crucial aspect to acting intelligently upon the knowledge about causal structure which has been inferred from finite data demands reasoning about its uncertainty.
Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization
To answer these questions, we revisit the fundamental assumptions in linear regression tasks, where invariance-based approaches were shown to provably generalize OOD.
Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation
Using insights from Temporal Difference learning, we propose GFlowNet, based on a view of the generative process as a flow network, making it possible to handle the tricky case where different trajectories can yield the same final state, e. g., there are many ways to sequentially add atoms to generate some molecular graph.
SpeechBrain: A General-Purpose Speech Toolkit
SpeechBrain is an open-source and all-in-one speech toolkit.
A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning
We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state, in order to plan and to generalize better out-of-distribution.
Fast and Slow Learning of Recurrent Independent Mechanisms
To study these ideas, we propose a particular training framework in which we assume that the pieces of knowledge an agent needs and its reward function are stationary and can be re-used across tasks.
An End-to-End Framework for Molecular Conformation Generation via Bilevel Programming
Specifically, the molecular graph is first encoded in a latent space, and then the 3D structures are generated by solving a principled bilevel optimization program.
Comparative Study of Learning Outcomes for Online Learning Platforms
Personalization and active learning are key aspects to successful learning.
hBert + BiasCorp -- Fighting Racism on the Web
In this work, we present hBERT, where we modify certain layers of the pretrained BERT model with the new Hopfield Layer.
Neural Production Systems
First, GNNs do not predispose interactions to be sparse, as relationships among independent entities are likely to be.
Coordination Among Neural Modules Through a Shared Global Workspace
We explore the use of such a communication channel in the context of deep learning for modeling the structure of complex environments.
Exploring the Wasserstein metric for survival analysis
Survival analysis is a type of semi-supervised task where the target output (the survival time) is often right-censored.
Transformers with Competitive Ensembles of Independent Mechanisms
In this work we explore a way in which the Transformer architecture is deficient: it represents each position with a large monolithic hidden representation and a single set of parameters which are applied over the entire hidden representation.
Towards Causal Representation Learning
The two fields of machine learning and graphical causality arose and developed separately.
Learning Neural Generative Dynamics for Molecular Conformation Generation
Inspired by the recent progress in deep generative models, in this paper, we propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
DEUP: Direct Epistemic Uncertainty Prediction
Epistemic uncertainty is the part of out-of-sample prediction error due to the lack of knowledge of the learner.
Structured Sparsity Inducing Adaptive Optimizers for Deep Learning
The parameters of a neural network are naturally organized in groups, some of which might not contribute to its overall performance.
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
Equilibrium Propagation (EP) is a biologically-inspired counterpart of Backpropagation Through Time (BPTT) which, owing to its strong theoretical guarantees and the locality in space of its learning rule, fosters the design of energy-efficient hardware dedicated to learning.
Neural Bayes: A Generic Parameterization Method for Unsupervised Learning
Disjoint Manifold Separation: Neural Bayes allows us to formulate an objective which can optimally label samples from disjoint manifolds present in the support of a continuous distribution.
Spatially Structured Recurrent Modules
Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalise well and are robust to changes in the input distribution.
Dependency Structure Discovery from Interventions
Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data.
Meta Attention Networks: Meta-Learning Attention to Modulate Information Between Recurrent Independent Mechanisms
Decomposing knowledge into interchangeable pieces promises a generalization advantage when there are changes in distribution.
Conditional Networks
In this work we tackle the problem of out-of-distribution generalization through conditional computation.
Systematic generalisation with group invariant predictions
We consider situations where the presence of dominant simpler correlations with the target variable in a training set can cause an SGD-trained neural network to be less reliant on more persistently-correlating complex features.
Factorizing Declarative and Procedural Knowledge in Structured, Dynamical Environments
To use a video game as an illustration, two enemies of the same type will share schemata but will have separate object files to encode their distinct state (e. g., health, position).
Machine Learning for Glacier Monitoring in the Hindu Kush Himalaya
Glacier mapping is key to ecological monitoring in the hkh region.
Untangling tradeoffs between recurrence and self-attention in artificial neural networks
Attention and self-attention mechanisms, are now central to state-of-the-art deep learning on sequential tasks.
Inductive Biases for Deep Learning of Higher-Level Cognition
A fascinating hypothesis is that human and animal intelligence could be explained by a few principles (rather than an encyclopedic list of heuristics).
RetroGNN: Approximating Retrosynthesis by Graph Neural Networks for De Novo Drug Design
A natural idea to mitigate this problem is to bias the search process towards more easily synthesizable molecules using a proxy for synthetic accessibility.
Gradient Starvation: A Learning Proclivity in Neural Networks
We identify and formalize a fundamental gradient descent phenomenon resulting in a learning proclivity in over-parameterized neural networks.
COVI-AgentSim: an Agent-based Model for Evaluating Methods of Digital Contact Tracing
The rapid global spread of COVID-19 has led to an unprecedented demand for effective methods to mitigate the spread of the disease, and various digital contact tracing (DCT) methods have emerged as a component of the solution.
Predicting Infectiousness for Proactive Contact Tracing
Predictions are used to provide personalized recommendations to the individual via an app, as well as to send anonymized messages to the individual's contacts, who use this information to better predict their own infectiousness, an approach we call proactive contact tracing (PCT).
NU-GAN: High resolution neural upsampling with GAN
In this paper, we propose NU-GAN, a new method for resampling audio from lower to higher sampling rates (upsampling).
Cross-Modal Information Maximization for Medical Imaging: CMIM
In hospitals, data are siloed to specific information systems that make the same information available under different modalities such as the different medical imaging exams the patient undergoes (CT scans, MRI, PET, Ultrasound, etc.)
Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers
Feed-forward neural networks consist of a sequence of layers, in which each layer performs some processing on the information from the previous layer.
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
To facilitate research addressing this problem, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment.
RNNLogic: Learning Logic Rules for Reasoning on Knowledge Graphs
Then in the E-step, we select a set of high-quality rules from all generated rules with both the rule generator and reasoning predictor via posterior inference; and in the M-step, the rule generator is updated with the rules selected in the E-step.
Visual Concept Reasoning Networks
It approximates sparsely connected networks by explicitly defining multiple branches to simultaneously learn representations with different visual concepts or properties.
Mastering Rate based Curriculum Learning
Recent automatic curriculum learning algorithms, and in particular Teacher-Student algorithms, rely on the notion of learning progress, making the assumption that the good next tasks are the ones on which the learner is making the fastest progress or digress.
Deriving Differential Target Propagation from Iterating Approximate Inverses
We show that a particular form of target propagation, i. e., relying on learned inverses of each layer, which is differential, i. e., where the target is a small perturbation of the forward propagation, gives rise to an update rule which corresponds to an approximate Gauss-Newton gradient-based optimization, without requiring the manipulation or inversion of large matrices.
Multi-Image Super-Resolution for Remote Sensing using Deep Recurrent Networks
High-resolution satellite imagery is critical for various earth observation applications related to environment monitoring, geoscience, forecasting, and land use analysis.
BabyAI 1.1
This increases reinforcement learning sample efficiency by up to 3 times and improves imitation learning performance on the hardest level from 77 % to 90. 4 %.
S2RMs: Spatially Structured Recurrent Modules
Capturing the structure of a data-generating process by means of appropriate inductive biases can help in learning models that generalize well and are robust to changes in the input distribution.
Revisiting Fundamentals of Experience Replay
Experience replay is central to off-policy algorithms in deep reinforcement learning (RL), but there remain significant gaps in our understanding.
Compositional Generalization by Factorizing Alignment and Translation
Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution.
Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules
To effectively utilize the wealth of potential top-down information available, and to prevent the cacophony of intermixed signals in a bidirectional architecture, mechanisms are needed to restrict information flow.
Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems
To use a video game as an illustration, two enemies of the same type will share schemata but will have separate object files to encode their distinct state (e. g., health, position).
Hybrid Models for Learning to Branch
First, in a more realistic setting where only a CPU is available, is the GNN model still competitive?
Image-to-image Mapping with Many Domains by Sparse Attribute Transfer
Unsupervised image-to-image translation consists of learning a pair of mappings between two domains without known pairwise correspondences between points.
Rethinking Distributional Matching Based Domain Adaptation
In this paper, in order to devise robust DA algorithms, we first systematically analyze the limitations of DM based methods, and then build new benchmarks with more realistic domain shifts to evaluate the well-accepted DM methods.
HNHN: Hypergraph Networks with Hyperedge Neurons
Hypergraphs provide a natural representation for many real world datasets.
Untangling tradeoffs between recurrence and self-attention in neural networks
Attention and self-attention mechanisms, are now central to state-of-the-art deep learning on sequential tasks.
Learning Causal Models Online
One solution for achieving strong generalization is to incorporate causal structures in the models; such structures constrain learning by ignoring correlations that contradict them.
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
In this work, we show that a bias in the gradient estimate of EP, inherent in the use of finite nudging, is responsible for this phenomenon and that cancelling it allows training deep ConvNets by EP.
Training End-to-End Analog Neural Networks with Equilibrium Propagation
We introduce a principled method to train end-to-end analog neural networks by stochastic gradient descent.
Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning
In this study, we present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images.
COVI White Paper
Manual contact tracing of Covid-19 cases has significant challenges that limit the ability of public health authorities to minimize community infections.
An Analysis of the Adaptation Speed of Causal Models
When the intervention is on the effect variable, we characterize the relative adaptation speed.
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
It is commonly believed that knowledge of syntactic structure should improve language modeling.
A Large-Scale, Open-Domain, Mixed-Interface Dialogue-Based ITS for STEM
We present Korbit, a large-scale, open-domain, mixed-interface, dialogue-based intelligent tutoring system (ITS).
Learning the Arrow of Time for Problems in Reinforcement Learning
We humans have an innate understanding of the asymmetric progression of time, which we use to efficiently and safely perceive and manipulate our environment.
Equilibrium Propagation with Continual Weight Updates
However, in existing implementations of EP, the learning rule is not local in time: the weight update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.
Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation
On the other hand, the biological plausibility of EP is limited by the fact that its learning rule is not local in time: the synapse update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.
DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning
Visual Similarity plays an important role in many computer vision applications.
Learning To Navigate The Synthetically Accessible Chemical Space Using Reinforcement Learning
Over the last decade, there has been significant progress in the field of machine learning for de novo drug design, particularly in deep generative models.
The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budget
This is typically the case when we have a standard conditioning input, such as a state observation, and a "privileged" input, which might correspond to the goal of a task, the output of a costly planning algorithm, or communication with another agent.
Experience Grounds Language
Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates.
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development.
Computers and Society
Object-Centric Image Generation from Layouts
In this paper, we start with the idea that a model must be able to understand individual objects and relationships between objects in order to generate complex scenes well.
Ranked #1 on
Layout-to-Image Generation
on COCO-Stuff 256x256
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling
To make that practical, we show that sampling from this modified density can be achieved by sampling in latent space according to an energy-based model induced by the sum of the latent prior log-density and the discriminator output score.
Continuous Domain Adaptation with Variational Domain-Agnostic Feature Replay
Learning in non-stationary environments is one of the biggest challenges in machine learning.
Benchmarking Graph Neural Networks
Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs.
Ranked #1 on
Link Prediction
on COLLAB
On Catastrophic Interference in Atari 2600 Games
Our study provides a clear empirical link between catastrophic interference and sample efficiency in reinforcement learning.
Neural Bayes: A Generic Parameterization Method for Unsupervised Representation Learning
Disjoint Manifold Labeling: Neural Bayes allows us to formulate an objective which can optimally label samples from disjoint manifolds present in the support of a continuous distribution.
HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery
Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views.
Ranked #4 on
Multi-Frame Super-Resolution
on PROBA-V
Parameterizing Branch-and-Bound Search Trees to Learn Branching Policies
We aim instead at learning a policy that generalizes across heterogeneous MILPs: our main hypothesis is that parameterizing the state of the B&B search tree can aid this type of generalization.
BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization
Neural networks have demonstrably achieved state-of-the art accuracy using low-bitlength integer quantization, yielding both execution time and energy benefits on existing hardware designs that support short bitlengths.
Meta-learning framework with applications to zero-shot time-series forecasting
Can meta-learning discover generic ways of processing time series (TS) from a diverse dataset so as to greatly improve generalization on new TS coming from different datasets?
Combating False Negatives in Adversarial Imitation Learning
In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior.
Using Simulated Data to Generate Images of Climate Change
In our paper, we explore the potential of using images from a simulated 3D environment to improve a domain adaptation task carried out by the MUNIT architecture, aiming to use the resulting images to raise awareness of the potential future impacts of climate change.
Multi-task self-supervised learning for Robust Speech Recognition
We then propose a revised encoder that better learns short- and long-term speech dynamics with an efficient combination of recurrent and convolutional networks.
Universal Successor Features for Transfer Reinforcement Learning
Transfer in Reinforcement Learning (RL) refers to the idea of applying knowledge gained from previous tasks to solving related tasks.
HighRes-net: Multi-Frame Super-Resolution by Recursive Fusion
Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views.
Ranked #4 on
Multi-Frame Super-Resolution
on PROBA-V
Learning from Learning Machines: Optimisation, Rules, and Social Norms
There is an analogy between machine learning systems and economic entities in that they are both adaptive, and their behaviour is specified in a more-or-less explicit way.
On the Morality of Artificial Intelligence
Much of the existing research on the social and ethical impact of Artificial Intelligence has been focused on defining ethical principles and guidelines surrounding Machine Learning (ML) and other Artificial Intelligence (AI) algorithms [IEEE, 2017, Jobin et al., 2019].
A learning-based algorithm to quickly compute good primal solutions for Stochastic Integer Programs
We propose a novel approach using supervised learning to obtain near-optimal primal solutions for two-stage stochastic integer programming (2SIP) problems with constraints in the first and second stages.
Joint Learning of Generative Translator and Classifier for Visually Similar Classes
In this paper, we propose a Generative Translation Classification Network (GTCN) for improving visual classification accuracy in settings where classes are visually similar and data is scarce.
CLOSURE: Assessing Systematic Generalization of CLEVR Models
In this work, we study how systematic the generalization of such models is, that is to which extent they are capable of handling novel combinations of known linguistic constructs.
The effect of task and training on intermediate representations in convolutional neural networks revealed with modified RV similarity analysis
Centered Kernel Alignment (CKA) was recently proposed as a similarity metric for comparing activation patterns in deep networks.
Applying Knowledge Transfer for Water Body Segmentation in Peru
In this work, we present the application of convolutional neural networks for segmenting water bodies in satellite images.
Automated curriculum generation for Policy Gradients from Demonstrations
In this paper, we present a technique that improves the process of training an agent (using RL) for instruction following.
Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models
We test the verifier network on out-of-distribution detection and adversarial example detection problems, as well as anomaly detection problems in structured prediction tasks such as image caption generation.
Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks
This reduced system combines the essential elements to have a working biologically abstracted analogue of backpropagation with a simple formulation and proofs of the associated results.
Small-GAN: Speeding Up GAN Training Using Core-sets
Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes.
Establishing an Evaluation Metric to Quantify Climate Change Image Realism
Because metrics for comparing the realism of different modes in a conditional generative model do not exist, we propose several automated and human-based methods for evaluation.
Icentia11K: An Unsupervised Representation Learning Dataset for Arrhythmia Subtype Discovery
We release the largest public ECG dataset of continuous raw signals for representation learning containing 11 thousand patients and 2 billion labelled beats.
Predicting ice flow using machine learning
Though machine learning has achieved notable success in modeling sequential and spatial data for speech recognition and in computer vision, applications to remote sensing and climate science problems are seldom considered.
MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis
In this paper, we show that it is possible to train GANs reliably to generate high quality coherent waveforms by introducing a set of architectural changes and simple training techniques.
Variational Temporal Abstraction
We introduce a variational approach to learning and inference of temporally hierarchical structure and representation for sequential data.
Learning Neural Causal Models from Unknown Interventions
Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data.
Saliency is a Possible Red Herring When Diagnosing Poor Generalization
In some prediction tasks, such as for medical images, one may have some images with masks drawn by a human expert, indicating a region of the image containing relevant information to make the prediction.
GraphMix: Improved Training of GNNs for Semi-Supervised Learning
We present GraphMix, a regularization method for Graph Neural Network based semi-supervised object classification, whereby we propose to train a fully-connected network jointly with the graph neural network via parameter sharing and interpolation-based regularization.
Ranked #1 on
Node Classification
on Cora random partition
Recurrent Independent Mechanisms
Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes.
Avoidance Learning Using Observational Reinforcement Learning
We define avoidance learning as the process of optimizing the agent's reward while avoiding dangerous behaviors given by a demonstrator.
Torchmeta: A Meta-Learning library for PyTorch
The constant introduction of standardized benchmarks in the literature has helped accelerating the recent advances in meta-learning research.
Data-Driven Approach to Encoding and Decoding 3-D Crystal Structures
We present a method to encode and decode the position of atoms in 3-D molecules from a dataset of nearly 50, 000 stable crystal unit cells that vary from containing 1 to over 100 atoms.
Interactive Language Learning by Question Answering
In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions.
Learning Fixed Points in Generative Adversarial Networks: From Image-to-Image Translation to Disease Detection and Localization
Qualitative and quantitative evaluations demonstrate that the proposed method outperforms the state of the art in multi-domain image-to-image translation and that it surpasses predominant weakly-supervised localization methods in both disease detection and localization.
Weakly-supervised Knowledge Graph Alignment with Adversarial Learning
Therefore, in this paper we propose to study aligning knowledge graphs in fully-unsupervised or weakly-supervised fashion, i. e., without or with only a few aligned triplets.
Learning the Arrow of Time
We humans seem to have an innate understanding of the asymmetric progression of time, which we use to efficiently and safely perceive and manipulate our environment.
Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives
Reinforcement learning agents that operate in diverse and complex environments can benefit from the structured decomposition of their behavior.
Perceptual Generative Autoencoders
We therefore propose to map both the generated and target distributions to a latent space using the encoder of a standard autoencoder, and train the generator (or decoder) to match the target distribution in the latent space.
Unsupervised State Representation Learning in Atari
State representation learning, or the ability to capture latent generative factors of an environment, is crucial for building intelligent agents that can perform a wide variety of tasks.
On the interplay between noise and curvature and its effect on optimization and generalization
The speed at which one can minimize an expected loss using stochastic methods depends on two properties: the curvature of the loss and the variance of the gradients.
Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracy
Adversarial robustness has become a central goal in deep learning, both in the theory and the practice.
Conditional Computation for Continual Learning
In this study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example.
Learning Powerful Policies by Using Consistent Dynamics Model
There is enough evidence that humans build a model of the environment, not only by observing the environment but also by interacting with the environment.
Tackling Climate Change with Machine Learning
Climate change is one of the greatest challenges facing humanity, and we, as machine learning experts, may wonder how we can help.
How to Initialize your Network? Robust Initialization for WeightNorm & ResNets
Finally, we show that using our initialization in conjunction with learning rate warmup is able to reduce the gap between the performance of weight normalized and batch normalized networks.
Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study
Neural generative models have been become increasingly popular when building conversational agents.
Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input
Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i. e. RNNs that are fed by a static input x and settle to a steady state.
Attention Based Pruning for Shift Networks
In many application domains such as computer vision, Convolutional Layers (CLs) are key to the accuracy of deep learning methods.
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics
A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over long time scales, is to constrain recurrent connectivity matrices to be orthogonal or unitary.
State-Reification Networks: Improving Generalization by Modeling the Distribution of Hidden Representations
Machine learning promises methods that generalize well from finite labeled data.
N-BEATS: Neural basis expansion analysis for interpretable time series forecasting
We focus on solving the univariate times series point forecasting problem using deep learning.
The Journey is the Reward: Unsupervised Learning of Influential Trajectories
Unsupervised exploration and representation learning become increasingly important when learning in diverse and sparse environments.
GMNN: Graph Markov Neural Networks
Statistical relational learning methods can effectively model the dependency of object labels through conditional random fields for collective classification, whereas graph neural networks learn effective object representations for classification through end-to-end training.
Visualizing the Consequences of Climate Change Using Cycle-Consistent Adversarial Networks
We present a project that aims to generate images that depict accurate, vivid, and personalized outcomes of climate change using Cycle-Consistent Adversarial Networks (CycleGANs).
A Walk with SGD: How SGD Explores Regions of Deep Network Loss?
The non-convex nature of the loss landscape of deep neural networks (DNN) lends them the intuition that over the course of training, stochastic optimization algorithms explore different regions of the loss surface by entering and escaping many local minima due to the noise induced by mini-batches.
Probabilistic Planning with Sequential Monte Carlo methods
In this work, we propose a novel formulation of planning which views it as a probabilistic inference problem over future optimal trajectories.
Reinforced Imitation Learning from Observations
Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse.
Modeling the Long Term Future in Model-Based Reinforcement Learning
This paper focuses on building a model that reasons about the long-term future and demonstrates how to use this for efficient planning and exploration.
Transfer and Exploration via the Information Bottleneck
In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
Manifold Mixup: Learning Better Representations by Interpolating Hidden States
Because the hidden states are learned, this has an important effect of encouraging the hidden states for a class to be concentrated in such a way so that interpolations within the same class or between two different classes do not intersect with the real data points from other classes.
EnGAN: Latent Space MCMC and Maximum Entropy Generators for Energy-based Models
Unsupervised learning is about capturing dependencies between variables and is driven by the contrast between the probable vs improbable configurations of these variables, often either via a generative model which only samples probable ones or with an energy function (unnormalized log-density) which is low for probable ones and high for improbable ones.
Unsupervised one-to-many image translation
We perform completely unsupervised one-sided image to image translation between a source domain $X$ and a target domain $Y$ such that we preserve relevant underlying shared semantics (e. g., class, size, shape, etc).
Compositional generalization in a deep seq2seq model by separating syntax and semantics
Standard methods in deep learning for natural language processing fail to capture the compositional structure of human language that allows for systematic generalization outside of the training distribution.
GradMask: Reduce Overfitting by Regularizing Saliency
With too few samples or too many model parameters, overfitting can inhibit the ability to generalise predictions to new data.
Speech Model Pre-training for End-to-End Spoken Language Understanding
Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map speech directly to intent through a single trainable model.
Ranked #2 on
Spoken Language Understanding
on Fluent Speech Commands
(using extra training data)
Learning Problem-agnostic Speech Representations from Multiple Self-supervised Tasks
Learning good representations without supervision is still an open issue in machine learning, and is particularly challenging for speech signals, which are often characterized by long sequences with a complex hierarchical structure.
Ranked #2 on
Distant Speech Recognition
on DIRHA English WSJ
Reinforced Imitation in Heterogeneous Action Space
Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse.
InfoMask: Masked Variational Latent Representation to Localize Chest Disease
The scarcity of richly annotated medical images is limiting supervised deep learning based solutions to medical image analysis tasks, such as localizing discriminatory radiomic disease signatures.
Wasserstein Dependency Measure for Representation Learning
Mutual information maximization has emerged as a powerful learning objective for unsupervised representation learning obtaining state-of-the-art performance in applications such as object recognition, speech recognition, and reinforcement learning.
Towards Standardization of Data Licenses: The Montreal Data License
This paper provides a taxonomy for the licensing of data in the fields of artificial intelligence and machine learning.
Gradient based sample selection for online continual learning
To prevent forgetting, a replay buffer is usually employed to store the previous data for the purpose of rehearsal.
Interpolation Consistency Training for Semi-Supervised Learning
We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm.
On Adversarial Mixup Resynthesis
In this paper, we explore new approaches to combining information encoded within the learned representations of auto-encoders.
Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
This paper focuses on building a model that reasons about the long-term future and demonstrates how to use this for efficient planning and exploration.
Hyperbolic Discounting and Learning over Multiple Horizons
Reinforcement learning (RL) typically defines a discount factor as part of the Markov Decision Process.
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms
We show that causal structures can be parameterized via continuous variables and learned end-to-end.
InfoBot: Transfer and Exploration via the Information Bottleneck
In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.
Maximum Entropy Generators for Energy-Based Models
Maximum likelihood estimation of energy-based models is a challenging problem due to the intractability of the log-likelihood gradient.
Towards Non-saturating Recurrent Units for Modelling Long-term Dependencies
Modelling long-term dependencies is a challenge for recurrent neural networks.
Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information
We formulate the problem as a two-stage optimal prediction stochastic program whose solution we predict with a supervised machine learning algorithm.
The Benefits of Over-parameterization at Initialization in Deep ReLU Networks
These results are derived using the PAC analysis framework, and hold true for finitely sized datasets such that the width of the ReLU network only needs to be larger than a certain finite lower bound.
Quantized Guided Pruning for Efficient Hardware Implementations of Convolutional Neural Networks
Convolutional Neural Networks (CNNs) are state-of-the-art in numerous computer vision tasks such as object classification and detection.
Speech and Speaker Recognition from Raw Waveform with SincNet
Deep neural networks can learn complex and abstract representations, that are progressively obtained by combining simpler ones.
An Empirical Study of Example Forgetting during Deep Neural Network Learning
Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks.