Search Results for author:
Found 191 papers, 114 papers with code
Involutive MCMC: One Way to Derive Them All
Markov Chain Monte Carlo (MCMC) is a computational approach to fundamental problems such as inference, integration, optimization, and simulation.
Binding Dynamics in Rotating Features
In human cognition, the binding problem describes the open question of how the brain flexibly integrates diverse information into cohesive object representations.
Position Paper: Bayesian Deep Learning in the Age of Large-Scale AI
In the current landscape of deep learning research, there is a predominant emphasis on achieving high predictive accuracy in supervised tasks involving large image and language datasets.
Protect Your Score: Contact Tracing With Differential Privacy Guarantees
The pandemic in 2020 and 2021 had enormous economic and societal consequences, and studies show that contact tracing algorithms can be key in the early containment of the virus.
Perspectives on the State and Future of Deep Learning -- 2023
The goal of this series is to chronicle opinions and issues in the field of machine learning as they stand today and as they change over time.
Image segmentation with traveling waves in an exactly solvable recurrent neural network
We study image segmentation using spatiotemporal dynamics in a recurrent neural network where the state of each unit is given by a complex number.
Lie Point Symmetry and Physics Informed Networks
Symmetries have been leveraged to improve the generalization of neural networks through different mechanisms from data augmentation to equivariant architectures.
GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers
As transformers are equivariant to the permutation of input tokens, encoding the positional information of tokens is necessary for many tasks.
Flow Factorized Representation Learning
A prominent goal of representation learning research is to achieve representations which are factorized in a useful manner with respect to the ground truth factors of variation.
Learning Objective-Specific Active Learning Strategies with Attentive Neural Processes
In this work, we propose a novel LAL method for classification that exploits symmetry and independence properties of the active learning problem with an Attentive Conditional Neural Process model.
Traveling Waves Encode the Recent Past and Enhance Sequence Learning
Traveling waves of neural activity have been observed throughout the brain at a diversity of regions and scales; however, their precise computational role is still debated.
Efficient Neural PDE-Solvers using Quantization Aware Training
In the past years, the application of neural networks as an alternative to classical numerical methods to solve Partial Differential Equations has emerged as a potential paradigm shift in this century-old mathematical field.
Latent Traversals in Generative Models as Potential Flows
In this work, we instead propose to model latent structures with a learned dynamic potential landscape, thereby performing latent traversals as the flow of samples down the landscape's gradient.
The END: An Equivariant Neural Decoder for Quantum Error Correction
Quantum error correction is a critical component for scaling up quantum computing.
Geometric Clifford Algebra Networks
GCANs are based on symmetry group transformations using geometric (Clifford) algebras.
Pixelated Reconstruction of Foreground Density and Background Surface Brightness in Gravitational Lensing Systems using Recurrent Inference Machines
In this work, we use a neural network based on the Recurrent Inference Machine (RIM) to simultaneously reconstruct an undistorted image of the background source and the lens mass density distribution as pixelated maps.
Structure-based Drug Design with Equivariant Diffusion Models
Structure-based drug design (SBDD) aims to design small-molecule ligands that bind with high affinity and specificity to pre-determined protein targets.
Equivariant 3D-Conditional Diffusion Models for Molecular Linker Design
Additionally, the model automatically determines the number of atoms in the linker and its attachment points to the input fragments.
SVNet: Where SO(3) Equivariance Meets Binarization on Point Cloud Representation
Precisely, the presence of scalar features makes the major part of the network binarizable, while vector features serve to retain rich structural information and ensure SO(3) equivariance.
Clifford Neural Layers for PDE Modeling
We empirically evaluate the benefit of Clifford neural layers by replacing convolution and Fourier operations in common neural PDE surrogates by their Clifford counterparts on 2D Navier-Stokes and weather modeling tasks, as well as 3D Maxwell equations.
Bayesian Optimization for Macro Placement
Macro placement is the problem of placing memory blocks on a chip canvas.
Stochastic Optimal Control for Collective Variable Free Sampling of Molecular Transition Paths
We consider the problem of sampling transition paths between two given metastable states of a molecular system, e. g. a folded and unfolded protein or products and reactants of a chemical reaction.
Complex-Valued Autoencoders for Object Discovery
Object-centric representations form the basis of human perception, and enable us to reason about the world and to systematically generalize to new settings.
Equivariant Diffusion for Molecule Generation in 3D
This work introduces a diffusion model for molecule generation in 3D that is equivariant to Euclidean transformations.
Adversarial Defense via Image Denoising with Chaotic Encryption
In the literature on adversarial examples, white box and black box attacks have received the most attention.
Alleviating Adversarial Attacks on Variational Autoencoders with MCMC
Variational autoencoders (VAEs) are latent variable models that can generate complex objects and provide meaningful latent representations.
Neural RF SLAM for unsupervised positioning and mapping with channel state information
The model is based on an encoder-decoder architecture.
Lie Point Symmetry Data Augmentation for Neural PDE Solvers
In this paper, we present a method, which can partially alleviate this problem, by improving neural PDE solver sample complexity -- Lie point symmetry data augmentation (LPSDA).
Message Passing Neural PDE Solvers
The numerical solution of partial differential equations (PDEs) is difficult, having led to a century of research so far.
Modality-Agnostic Topology Aware Localization
This work presents a data-driven approach for the indoor localization of an observer on a 2D topological map of the environment.
Particle Dynamics for Learning EBMs
We accomplish this by viewing the evolution of the modeling distribution as (i) the evolution of the energy function, and (ii) the evolution of the samples from this distribution along some vector field.
An Expectation-Maximization Perspective on Federated Learning
Federated learning describes the distributed training of models across multiple clients while keeping the data private on-device.
Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders
Category-selectivity in the brain describes the observation that certain spatially localized areas of the cerebral cortex tend to respond robustly and selectively to stimuli from specific limited categories.
Multi-Agent MDP Homomorphic Networks
This paper introduces Multi-Agent MDP Homomorphic Networks, a class of networks that allows distributed execution using only local information, yet is able to share experience between global symmetries in the joint state-action space of cooperative multi-agent systems.
Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
Including covariant information, such as position, force, velocity or spin is important in many tasks in computational physics and chemistry.
Neural Augmentation of Kalman Filter with Hypernetwork for Channel Tracking
Through extensive experiments on CDL-B channel model, we show that the HKF can be used for tracking the channel over a wide range of Doppler values, matching Kalman filter performance with genie Doppler information.
Topographic VAEs learn Equivariant Capsules
Finally, we demonstrate approximate equivariance to complex transformations, expanding upon the capabilities of existing group equivariant neural networks.
Federated Mixture of Experts
Federated learning (FL) has emerged as the predominant approach for collaborative training of neural network models across multiple users, without the need to gather the data at a central location.
Deterministic Gibbs Sampling via Ordinary Differential Equations
Deterministic dynamics is an essential part of many MCMC algorithms, e. g.
Learning Equivariant Energy Based Models with Equivariant Stein Variational Gradient Descent
We focus on the problem of efficient sampling and learning of probability densities by incorporating symmetries in probabilistic models.
Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
We argue that the particular choice of coordinatization should not affect a network's inference -- it should be coordinate independent.
E(n) Equivariant Normalizing Flows
This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs).
A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
Symmetries and equivariance are fundamental to the generalization of neural networks on domains such as images, graphs, and point clouds.
Federated Learning of User Verification Models Without Sharing Embeddings
We consider the problem of training User Verification (UV) models in federated setting, where each user has access to the data of only one class and user embeddings cannot be shared with the server or other users.
Diagnosing Vulnerability of Variational Auto-Encoders to Adversarial Attacks
In this work, we explore adversarial attacks on the Variational Autoencoders (VAE).
Combining Interventional and Observational Data Using Causal Reductions
Second, for continuous variables and assuming a linear-Gaussian model, we derive equality constraints for the parameters of the observational and interventional distributions.
The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning
In this work we develop a quantum field theory formalism for deep learning, where input signals are encoded in Gaussian states, a generalization of Gaussian processes which encode the agent's uncertainty about the input signal.
Batch Bayesian Optimization on Permutations using the Acquisition Weighted Kernel
In this work we propose a batch Bayesian optimization method for combinatorial problems on permutations, which is well suited for expensive-to-evaluate objectives.
Mixed Variable Bayesian Optimization with Frequency Modulated Kernels
In experiments, we demonstrate the improved sample efficiency of GP BO using FM kernels (BO-FM). On synthetic problems and hyperparameter optimization problems, BO-FM outperforms competitors consistently.
Deep Policy Dynamic Programming for Vehicle Routing Problems
Routing problems are a class of combinatorial problems with many practical applications.
E(n) Equivariant Graph Neural Networks
This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs).
Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions
Argmax Flows are defined by a composition of a continuous distribution (such as a normalizing flow), and an argmax function.
Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC
Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions.
Secure Federated Learning of User Verification Models
We consider the problem of training User Verification (UV) models in federated setup, where the conventional loss functions are not applicable due to the constraints that each user has access to the data of only one class and user embeddings cannot be shared with the server or other users.
Federated Averaging as Expectation Maximization
Federated averaging (FedAvg), despite its simplicity, has been the main approach in training neural networks in the federated learning setting.
Argmax Flows: Learning Categorical Distributions with Normalizing Flows
This paper introduces a new method to define and train continuous distributions such as normalizing flows directly on categorical data, for example text and image segmentation.
Self Normalizing Flows
Efficient gradient computation of the Jacobian determinant term is a core problem in many machine learning settings, and especially so in the normalizing flow framework.
Experimental design for MRI by greedy policy search
In today's clinical practice, magnetic resonance imaging (MRI) is routinely accelerated through subsampling of the associated Fourier domain.
Probabilistic Numeric Convolutional Neural Networks
Continuous input signals like images and time series that are irregularly sampled or have missing values are challenging for existing deep learning methods.
Quantum Deformed Neural Networks
We develop a new quantum neural network layer designed to run efficiently on a quantum computer but that can be simulated on a classical computer when restricted in the way it entangles input states.
Orbital MCMC
Markov Chain Monte Carlo (MCMC) algorithms ubiquitously employ complex deterministic transformations to generate proposal points that are then filtered by the Metropolis-Hastings-Green (MHG) test.
Natural Graph Networks
A key requirement for graph neural networks is that they must process a graph in a way that does not depend on how the graph is described.
Federated Learning of User Authentication Models
In this paper, we propose Federated User Authentication (FedUA), a framework for privacy-preserving training of UA models.
SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows
Normalizing flows and variational autoencoders are powerful generative models that can represent complicated density functions.
Involutive MCMC: a Unifying Framework
Markov Chain Monte Carlo (MCMC) is a computational approach to fundamental problems such as inference, integration, optimization, and simulation.
MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
MDP homomorphic networks are neural networks that are equivariant under symmetries in the joint state-action space of an MDP.
RE-MIMO: Recurrent and Permutation Equivariant Neural MIMO Detection
Each iterative unit is a neural computation module comprising of 3 sub-modules: the likelihood module, the encoder module, and the predictor module.
SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
We introduce the SE(3)-Transformer, a variant of the self-attention module for 3D point clouds and graphs, which is equivariant under continuous 3D roto-translations.
Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data
This enables us to train a single, amortized model that infers causal relations across samples with different underlying causal graphs, and thus leverages the shared dynamics information.
The Convolution Exponential and Generalized Sylvester Flows
Empirically, we show that the convolution exponential outperforms other linear transformations in generative flows on CIFAR10 and the graph convolution exponential improves the performance of graph normalizing flows.
Bayesian Bits: Unifying Quantization and Pruning
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization.
A Data and Compute Efficient Design for Limited-Resources Deep Learning
Thanks to their improved data efficiency, equivariant neural networks have gained increased interest in the deep learning community.
Guided Variational Autoencoder for Disentanglement Learning
We propose an algorithm, guided variational autoencoder (Guided-VAE), that is able to learn a controllable generative model by performing latent representation disentanglement learning.
Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
A common approach to define convolutions on meshes is to interpret them as a graph and apply graph convolutional networks (GCNs).
Neural Enhanced Belief Propagation on Factor Graphs
In this work we first extend graph neural networks to factor graphs (FG-GNN).
Plannable Approximations to MDP Homomorphisms: Equivariance under Actions
We introduce a contrastive loss function that enforces action equivariance on the learned representations.
Gradient $\ell_1$ Regularization for Quantization Robustness
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization.
Estimating Gradients for Discrete Random Variables by Sampling without Replacement
We derive an unbiased estimator for expectations over discrete random variables based on sampling without replacement, which reduces variance as it avoids duplicate samples.
Learning to Predict Error for MRI Reconstruction
In this paper, we demonstrate that predictive uncertainty estimated by the current methods does not highly correlate with prediction error by decomposing the latter into random and systematic errors, and showing that the former is equivalent to the variance of the random error.
Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks
The success of deep neural networks in many real-world applications is leading to new challenges in building more efficient architectures.
Learning Likelihoods with Conditional Normalizing Flows
Normalizing Flows (NFs) are able to model complicated distributions p(y) with strong inter-dimensional correlations and high multimodality by transforming a simple base density p(z) through an invertible neural network under the change of variables formula.
Contrastive Learning of Structured World Models
Our experiments demonstrate that C-SWMs can overcome limitations of models based on pixel reconstruction and outperform typical representatives of this model class in highly structured environments, while learning interpretable object-based representations.
Invert to Learn to Invert
Iterative learning to infer approaches have become popular solvers for inverse problems.
i-RIM applied to the fastMRI challenge
We, team AImsterdam, summarize our submission to the fastMRI challenge (Zbontar et al., 2018).
DP-MAC: The Differentially Private Method of Auxiliary Coordinates for Deep Learning
Developing a differentially private deep learning algorithm is challenging, due to the difficulty in analyzing the sensitivity of objective functions that are typically used to train deep neural networks.
Relational Generalized Few-Shot Learning
Instead, we consider the extended setup of generalized few-shot learning (GFSL), where the model is required to perform classification on the joint label space consisting of both previously seen and novel classes.
Batch-Shaping for Learning Conditional Channel Gated Networks
To achieve this, we introduce a new residual block architecture that gates convolutional channels in a fine-grained manner.
Supervised Uncertainty Quantification for Segmentation with Multiple Annotations
The accurate estimation of predictive uncertainty carries importance in medical scenarios such as lung node segmentation.
The Functional Neural Process
We present a new family of exchangeable stochastic processes, the Functional Neural Processes (FNPs).
Differentiable probabilistic models of scientific imaging with the Fourier slice theorem
Finally, we demonstrate how the reconstruction algorithm can be extended with an amortized inference scheme on unknown attributes such as object pose.
Data-Free Quantization Through Weight Equalization and Bias Correction
This improves quantization accuracy performance, and can be applied to many common computer vision architectures with a straight forward API call.
Covariance in Physics and Convolutional Neural Networks
In this proceeding we give an overview of the idea of covariance (or equivariance) featured in the recent development of convolutional neural networks (CNNs).
Combining Generative and Discriminative Models for Hybrid Inference
A graphical model is a structured representation of the data generating process.
An Introduction to Variational Autoencoders
Variational autoencoders provide a principled framework for learning deep latent-variable models and corresponding inference models.
Deep Scale-spaces: Equivariance Over Scale
We introduce deep scale-spaces (DSS), a generalization of convolutional neural networks, exploiting the scale symmetry structure of conventional image recognition tasks.
DIVA: Domain Invariant Variational Autoencoders
We consider the problem of domain generalization, namely, how to learn representations given data from a set of domains that generalize to data from a previously unseen domain.
Integer Discrete Flows and Lossless Compression
For that reason, we introduce a flow-based generative model for ordinal discrete data called Integer Discrete Flow (IDF): a bijective integer map that can learn rich transformations on high-dimensional data.
Initialized Equilibrium Propagation for Backprop-Free Training
In response to this, Scellier & Bengio (2017) proposed Equilibrium Propagation - a method for gradient-based train- ing of neural networks which uses only local learning rules and, crucially, does not rely on neurons having a mechanism for back-propagating an error gradient.
DIVA: Domain Invariant Variational Autoencoder
We consider the problem of domain generalization, namely, how to learn representations given data from a set of domains that generalize to data from a previously unseen domain.
Buy 4 REINFORCE Samples, Get a Baseline for Free!
REINFORCE can be used to train models in structured prediction settings to directly optimize the test-time objective.
Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement
We show how to implicitly apply this 'Gumbel-Top-$k$' trick on a factorized distribution over sequences, allowing to draw exact samples without replacement using a Stochastic Beam Search.
Gauge Equivariant Convolutional Networks and the Icosahedral CNN
The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design.
Ranked #23 on
Semantic Segmentation
on Stanford2D3D Panoramic
Combinatorial Bayesian Optimization using the Graph Cartesian Product
On this combinatorial graph, we propose an ARD diffusion kernel with which the GP is able to model high-order interactions between variables leading to better performance.
Emerging Convolutions for Generative Normalizing Flows
We generalize the 1 x 1 convolutions proposed in Glow to invertible d x d convolutions, which are more flexible since they operate on both channel and spatial axes.
To Relieve Your Headache of Training an MRF, Take AdVIL
We propose a black-box algorithm called {\it Adversarial Variational Inference and Learning} (AdVIL) to perform inference and learning on a general Markov random field (MRF).
Data-Driven Reconstruction of Gravitationally Lensed Galaxies using Recurrent Inference Machines
We present a machine learning method for the reconstruction of the undistorted images of background sources in strongly lensed systems.
Instrumentation and Methods for Astrophysics Cosmology and Nongalactic Astrophysics Astrophysics of Galaxies
Graph Refinement based Airway Extraction using Mean-Field Networks and Graph Neural Networks
Graph refinement, or the task of obtaining subgraphs of interest from over-complete graphs, can have many varied applications.
The Deep Weight Prior
Bayesian inference is known to provide a general framework for incorporating prior knowledge or specific properties into machine learning models via carefully choosing a prior distribution.
Predictive Uncertainty through Quantization
High-risk domains require reliable confidence estimates from predictive models.
Relaxed Quantization for Discretized Neural Networks
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices.
Sinkhorn AutoEncoders
We show that minimizing the p-Wasserstein distance between the generator and the true data distribution is equivalent to the unconstrained min-min optimization of the p-Wasserstein distance between the encoder aggregated posterior and the prior in latent space, plus a reconstruction error.
Probabilistic Binary Neural Networks
Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks.
3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
We prove that equivariant convolutions are the most general equivariant linear maps between fields over R^3.
Sample Efficient Semantic Segmentation using Rotation Equivariant Convolutional Networks
The group equivariant CNN framework is extended for segmentation by introducing a new equivariant (G->Z2)-convolution that transforms feature maps on a group to planar feature maps.
Rotation Equivariant CNNs for Digital Pathology
We propose a new model for digital pathology segmentation, based on the observation that histopathology images are inherently symmetric under rotation and reflection.
Ranked #7 on
Breast Tumour Classification
on PCam
BOCK : Bayesian Optimization with Cylindrical Kernels
A major challenge in Bayesian Optimization is the boundary issue (Swersky, 2017) where an algorithm spends too many evaluations near the boundary of its search space.
Primal-Dual Wasserstein GAN
We introduce Primal-Dual Wasserstein GAN, a new learning algorithm for building latent variable models of the data distribution based on the primal and the dual formulations of the optimal transport (OT) problem.
Extraction of Airways using Graph Neural Networks
We present extraction of tree structures, such as airways, from image data as a graph refinement task.
Mean Field Network based Graph Refinement with application to Airway Tree Extraction
Performance of the method is compared with two methods: the first uses probability images from a trained voxel classifier with region growing, which is similar to one of the best performing methods at EXACT'09 airway challenge, and the second method is based on Bayesian smoothing on these probability images.
Graphical Generative Adversarial Networks
We propose Graphical Generative Adversarial Networks (Graphical-GAN) to model structured data.
Attention, Learn to Solve Routing Problems!
The recently presented idea to learn heuristics for combinatorial optimization problems is promising as it can save costly development.
Sylvester Normalizing Flows for Variational Inference
Variational inference relies on flexible approximate posterior distributions.
HexaConv
We find that, due to the reduced anisotropy of hexagonal filters, planar HexaConv provides better accuracy than planar convolution with square filters, given a fixed parameter budget.
Attention-based Deep Multiple Instance Learning
Multiple instance learning (MIL) is a variation of supervised learning where a single class label is assigned to a bag of instances.
Ranked #7 on
Aerial Scene Classification
on UCM (50% as trainset)
Neural Relational Inference for Interacting Systems
Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics.
Spherical CNNs
Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images.
Learning Sparse Neural Networks through L_0 Regularization
We further propose the \emph{hard concrete} distribution for the gates, which is obtained by ``stretching'' a binary concrete distribution and then transforming its samples with a hard-sigmoid.
The Mutual Autoencoder: Controlling Information in Latent Code Representations
Thus, we decouple the choice of decoder capacity and the latent code dimensionality from the amount of information stored in the code.
Learning Sparse Neural Networks through $L_0$ Regularization
We further propose the \emph{hard concrete} distribution for the gates, which is obtained by "stretching" a binary concrete distribution and then transforming its samples with a hard-sigmoid.
Deep Learning with Permutation-invariant Operator for Multi-instance Histopathology Classification
The computer-aided analysis of medical scans is a longstanding goal in the medical imaging field.
Improved Bayesian Compression
Compression of Neural Networks (NN) has become a highly studied topic in recent years.
Convolutional Networks for Spherical Signals
Many areas of science and egineering deal with signals with other symmetries, such as rotation invariant data on the sphere.
Temporally Efficient Deep Learning with Spikes
We present a variant on backpropagation for neural networks in which computation scales with the rate of change of the data - not the rate at which we process the data.
Recurrent Inference Machines for Solving Inverse Problems
Much of the recent research on solving iterative inference problems focuses on moving away from hand-chosen inference algorithms and towards learned inference.
Improving Variational Auto-Encoders using convex combination linear Inverse Autoregressive Flow
In this paper, we propose a new volume-preserving flow and show that it performs similarly to the linear general normalizing flow.
Graph Convolutional Matrix Completion
We consider matrix completion for recommender systems from the point of view of link prediction on graphs.
Ranked #4 on
Recommendation Systems
on YahooMusic Monti
(using extra training data)
Causal Effect Inference with Deep Latent-Variable Models
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers.
Ranked #9 on
Causal Inference
on IHDP
Bayesian Compression for Deep Learning
Compression and computational efficiency in deep learning have become a problem of great significance.
VAE with a VampPrior
In this paper, we propose to extend the variational auto-encoder (VAE) framework with a new type of prior which we call "Variational Mixture of Posteriors" prior, or VampPrior for short.
Modeling Relational Data with Graph Convolutional Networks
We demonstrate the effectiveness of R-GCNs as a stand-alone model for entity classification.
Ranked #1 on
Node Classification
on AIFB
Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
We reinterpret multiplicative noise in neural networks as auxiliary random variables that augment the approximate posterior in a variational setting for Bayesian neural networks.
Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
This article presents the prediction difference analysis method for visualizing the response of a deep neural network to a specific input.
Soft Weight-Sharing for Neural Network Compression
The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices.
Steerable CNNs
It has long been recognized that the invariance and equivariance properties of a representation are critically important for success in many vision tasks.
Improved Variational Inference with Inverse Autoregressive Flow
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables.
Ranked #40 on
Image Generation
on CIFAR-10
(bits/dimension metric)
Improving Variational Auto-Encoders using Householder Flow
One fashion of enriching the variational posterior distribution is application of normalizing flows, i. e., a series of invertible transformations to latent variables with a simple posterior.
Variational Graph Auto-Encoders
We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE).
Ranked #1 on
Link Prediction
on Pubmed
(ACC metric)
Accelerating the BSM interpretation of LHC data with machine learning
The interpretation of Large Hadron Collider (LHC) data in the framework of Beyond the Standard Model (BSM) theories is hampered by the need to run computationally expensive event generators and detector simulators.
Sigma Delta Quantized Networks
Thus the amount of computation that the network does scales with the amount of change in the input and layer activations, rather than the size of the network.
Variational Bayes In Private Settings (VIPS)
Many applications of Bayesian data analysis involve sensitive information, motivating methods which ensure that privacy is protected.
Private Topic Modeling
We develop a privatised stochastic variational inference method for Latent Dirichlet Allocation (LDA).
Semi-Supervised Classification with Graph Convolutional Networks
We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs.
Ranked #1 on
Graph Property Prediction
on ogbg-ppa
Automatic Variational ABC
Approximate Bayesian Computation (ABC) is a framework for performing likelihood-free posterior inference for simulation models.
Improving Variational Inference with Inverse Autoregressive Flow
The framework of normalizing flows provides a general strategy for flexible variational inference of posteriors over latent variables.
A note on privacy preserving iteratively reweighted least squares
In particular, IRLS for L1 minimisation under the linear model provides a closed-form solution in each step, which is a simple multiplication between the inverse of the weighted second moment matrix and the weighted first moment vector.
DP-EM: Differentially Private Expectation Maximization
The iterative nature of the expectation maximization (EM) algorithm presents a challenge for privacy-preserving estimation, as each iteration increases the amount of noise needed.
On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis
Bayesian inference has great promise for the privacy-preserving analysis of sensitive data, as posterior sampling automatically preserves differential privacy, an algorithmic notion of data privacy, under certain conditions (Dimitrakakis et al., 2014; Wang et al., 2015).
Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors
We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on random matrices.
A New Method to Visualize Deep Neural Networks
We present a method for visualising the response of a deep neural network to a specific input.
Deep Spiking Networks
Our network is "spiking" in the sense that our neurons accumulate their activation into a potential over time, and only send out a signal (a "spike") when this potential crosses a threshold and the neuron is reset.
Group Equivariant Convolutional Networks
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries.
Ranked #6 on
Breast Tumour Classification
on PCam
Breast Tumour Classification
Colorectal Gland Segmentation:
+2
Herding as a Learning System with Edge-of-Chaos Dynamics
Herding defines a deterministic dynamical system at the edge of chaos.
The Variational Fair Autoencoder
We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible.
Ranked #4 on
Sentiment Analysis
on Multi-Domain Sentiment Dataset
Scalable MCMC for Mixed Membership Stochastic Blockmodels
We propose a stochastic gradient Markov chain Monte Carlo (SG-MCMC) algorithm for scalable inference in mixed-membership stochastic blockmodels (MMSB).
Bayesian Dark Knowledge
We consider the problem of Bayesian parameter estimation for deep neural networks, which is important in problem settings where we may have little data, and/ or where we need accurate posterior predictive densities, e. g., for applications involving bandits or active learning.
Optimization Monte Carlo: Efficient and Embarrassingly Parallel Likelihood-Free Inference
We describe an embarrassingly parallel, anytime Monte Carlo method for likelihood-free models.
Variational Dropout and the Local Reparameterization Trick
Our method allows inference of more flexibly parameterized posteriors; specifically, we propose variational dropout, a generalization of Gaussian dropout where the dropout rates are learned, often leading to better models.
Harmonic Exponential Families on Manifolds
In a range of fields including the geosciences, molecular biology, robotics and computer vision, one encounters problems that involve random variables on manifolds.
Hamiltonian ABC
Approximate Bayesian computation (ABC) is a powerful and elegant framework for performing inference in simulation-based models.
Large-Scale Distributed Bayesian Matrix Factorization using Stochastic Gradient MCMC
Despite having various attractive qualities such as high prediction accuracy and the ability to quantify uncertainty and avoid over-fitting, Bayesian Matrix Factorization has not been widely adopted because of the prohibitive cost of inference.
Transformation Properties of Learned Visual Representations
Starting with the idea that a good visual representation is one that transforms linearly under scene motions, we show, using the theory of group representations, that any such representation is equivalent to a combination of the elementary irreducible representations.
POPE: Post Optimization Posterior Evaluation of Likelihood Free Models
We propose a post optimization posterior analysis that computes and visualizes all the models that can generate equally good or better simulation results, subject to constraints.
MLitB: Machine Learning in the Browser
Beyond an educational resource for ML, the browser has vast potential to not only improve the state-of-the-art in ML research, but also, inexpensively and on a massive scale, to bring sophisticated ML learning and prediction to the public at large.
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Recent advances in stochastic gradient variational inference have made it possible to perform variational Bayesian inference with posterior approximations containing auxiliary random variables.
Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior
In recent years a number of methods have been developed for automatically learning the (sparse) connectivity structure of Markov Random Fields.
Semi-Supervised Learning with Deep Generative Models
The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis.
Ranked #53 on
Image Classification
on SVHN
Exploiting the Statistics of Learning and Inference
When dealing with datasets containing a billion instances or with simulations that require a supercomputer to execute, computational resources become part of the equation.
Learning the Irreducible Representations of Commutative Lie Groups
We present a new probabilistic model of compact commutative Lie groups that produces invariant-equivariant and disentangled representations of data.
Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets
Hierarchical Bayesian networks and neural networks with stochastic hidden units are commonly perceived as two separate types of models.
GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation
Scientists often express their understanding of the world through a computationally demanding simulation program.
Auto-Encoding Variational Bayes
First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods.
Ranked #11 on
Image Clustering
on Tiny-ImageNet
A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration
How many labeled examples are needed to estimate a classifier's performance on a new dataset?
Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation
We propose a stochastic algorithm for collapsed variational Bayesian inference for LDA, which is simpler and more efficient than the state of the art method.
Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget
Can we make Bayesian posterior MCMC sampling more efficient when faced with very large datasets?
The Time-Marginalized Coalescent Prior for Hierarchical Clustering
We introduce a new prior for use in Nonparametric Bayesian Hierarchical Clustering.
On Smoothing and Inference for Topic Models
Latent Dirichlet analysis, or topic modeling, is a flexible latent variable framework for modeling high-dimensional sparse count data.
Super-Samples from Kernel Herding
We extend the herding algorithm to continuous spaces by using the kernel trick.
Statistical Tests for Optimization Efficiency
Learning problems such as logistic regression are typically formulated as pure optimization problems defined on some loss function.
Bayesian Learning via Stochastic Gradient Langevin Dynamics
In this paper we propose a new framework for learning from large scale datasets based on iterative learning from small mini-batches.
On Herding and the Perceptron Cycling Theorem
The paper develops a connection between traditional perceptron algorithms and recently introduced herding algorithms.
Asynchronous Distributed Learning of Topic Models
Distributed learning is a problem of fundamental interest in machine learning and cognitive science.
