Search Results for author:
Found 54 papers, 24 papers with code
Cost-effective Interactive Attention Learning with Neural Attention Process
Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.
Regularizing Towards Soft Equivariance Under Mixed Symmetries
The key component of our method is what we call equivariance regularizer for a given type of symmetries, which measures how much a model is equivariant with respect to the symmetries of the type.
SWAMP: Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruning
Given the ever-increasing size of modern neural networks, the significance of sparse architectures has surged due to their accelerated inference speeds and minimal memory demands.
Martingale Posterior Neural Processes
Based on this result, instead of assuming any form of the latent variables, we equip a NP with a predictive distribution implicitly defined with neural networks and use the corresponding martingale posteriors as the source of uncertainty.
Decoupled Training for Long-Tailed Classification With Stochastic Representations
Decoupling representation learning and classifier learning has been shown to be effective in classification with long-tailed data.
Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning
We consider the optimisation of large and shallow neural networks via gradient flow, where the output of each hidden node is scaled by some positive parameter.
On Divergence Measures for Bayesian Pseudocoresets
Finally, we propose a novel Bayesian pseudocoreset algorithm based on minimizing forward KL divergence.
Exploring The Role of Mean Teachers in Self-supervised Masked Auto-Encoders
Masked image modeling (MIM) has become a popular strategy for self-supervised learning~(SSL) of visual representations with Vision Transformers.
Self-Distillation for Further Pre-training of Transformers
Pre-training a large transformer model on a massive amount of unlabeled data and fine-tuning it on labeled datasets for diverse downstream tasks has proven to be a successful strategy, for a variety of vision and natural language processing tasks.
Scalable Set Encoding with Universal Mini-Batch Consistency and Unbiased Full Set Gradient Approximation
Recent work on mini-batch consistency (MBC) for set functions has brought attention to the need for sequentially processing and aggregating chunks of a partitioned set while guaranteeing the same output for all partitions.
Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation
Ensembles of deep neural networks have demonstrated superior performance, but their heavy computational cost hinders applying them for resource-limited environments.
Set-based Meta-Interpolation for Few-Task Meta-Learning
Recently, several task augmentation methods have been proposed to tackle this issue using domain-specific knowledge to design augmentation techniques to densify the meta-training task distribution.
Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility
Under this model, we show that each layer of the infinite-width neural network can be characterised by two simple quantities: a non-negative scalar parameter and a L\'evy measure on the positive reals.
PolarDenseNet: A Deep Learning Model for CSI Feedback in MIMO Systems
In multiple-input multiple-output (MIMO) systems, the high-resolution channel information (CSI) is required at the base station (BS) to ensure optimal performance, especially in the case of multi-user MIMO (MU-MIMO) systems.
Diversity Matters When Learning From Ensembles
We propose a simple approach for reducing this gap, i. e., making the distilled performance close to the full ensemble.
Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty
Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer.
Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning
Thanks to the gradients aligned between tasks by our method, the model becomes less vulnerable to negative transfer and catastrophic forgetting.
Assumption-Free Survival Analysis Under Local Smoothness Prior
Survival analysis appears in various fields such as medicine, economics, engineering, and business.
Task Conditioned Stochastic Subsampling
Deep Learning algorithms are designed to operate on huge volumes of high dimensional data such as images.
Meta Learning Low Rank Covariance Factors for Energy Based Deterministic Uncertainty
Numerous recent works utilize bi-Lipschitz regularization of neural network layers to preserve relative distances between data instances in the feature spaces of each layer.
Scale Mixtures of Neural Network Gaussian Processes
We show that simply introducing a scale prior on the last-layer parameters can turn infinitely-wide neural networks of any architecture into a richer class of stochastic processes.
Hybrid Generative-Contrastive Representation Learning
On the other hand, the generative pre-training directly estimates the data distribution, so the representations tend to be robust but not optimal for discriminative tasks.
Adversarial purification with Score-based generative models
Recently, an Energy-Based Model (EBM) trained with Markov-Chain Monte-Carlo (MCMC) has been highlighted as a purification model, where an attacked image is purified by running a long Markov-chain using the gradients of the EBM.
Learning to Pool in Graph Neural Networks for Extrapolation
However, according to a recent study, a careful choice of pooling functions, which are used for the aggregation and readout operations in GNNs, is crucial for enabling GNNs to extrapolate.
Learning to Perturb Word Embeddings for Out-of-distribution QA
QA models based on pretrained language mod-els have achieved remarkable performance on various benchmark datasets. However, QA models do not generalize well to unseen data that falls outside the training distribution, due to distributional shifts. Data augmentation (DA) techniques which drop/replace words have shown to be effective in regularizing the model from overfitting to the training data. Yet, they may adversely affect the QA tasks since they incur semantic changes that may lead to wrong answers for the QA task.
SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data
Generative modeling of set-structured data, such as point clouds, requires reasoning over local and global structures at various scales.
Ranked #1 on
Point Cloud Generation
on ShapeNet Car
Mini-Batch Consistent Slot Set Encoder for Scalable Set Encoding
The proposed method adheres to the required symmetries of invariance and equivariance as well as maintaining MBC for any partition of the input set.
Improving Uncertainty Calibration via Prior Augmented Data
Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators.
Improving Neural Network Accuracy and Calibration Under Distributional Shift with Prior Augmented Data
We demonstrate the effectiveness of our method and validate its performance on both classification and regression problems by applying it to the training of recent state-of-the-art neural network models.
A Multi-Mode Modulator for Multi-Domain Few-Shot Classification
Most existing few-shot classification methods only consider generalization on one dataset (i. e., single-domain), failing to transfer across various seen and unseen domains.
Adaptive Strategy for Resetting a Non-stationary Markov Chain during Learning via Joint Stochastic Approximation
The non-stationary kernel problem refers to the degraded performance of the algorithm due to the constant change of the transition kernel of the chain throughout the run of the algorithm.
Amortized Probabilistic Detection of Communities in Graphs
While graph neural networks (GNNs) have been successful in encoding graph structures, existing GNN-based methods for community detection are limited by requiring knowledge of the number of communities in advance, in addition to lacking a proper probabilistic formulation to handle uncertainty.
Neural Complexity Measures
While various complexity measures for deep neural networks exist, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven challenging.
Bootstrapping Neural Processes
While this "data-driven" way of learning stochastic processes has proven to handle various types of data, NPs still rely on an assumption that uncertainty in stochastic processes is modeled by a single latent variable, which potentially limits the flexibility.
Set Based Stochastic Subsampling
Deep models are designed to operate on huge volumes of high dimensional data such as images.
Cost-effective Interactive Attention Learning with Neural Attention Processes
Moreover, it is almost infeasible for the human annotators to examine attentions on tons of instances and features.
Learning with Limited Data for Multilingual Reading Comprehension
This paper studies the problem of supporting question answering in a new language with limited training resources.
Graph Embedding VAE: A Permutation Invariant Model of Graph Structure
Generative models of graph structure have applications in biology and social sciences.
Deep Amortized Clustering
We propose a deep amortized clustering (DAC), a neural architecture which learns to cluster datasets efficiently using a few forward passes.
A unified construction for series representations and finite approximations of completely random measures
In particular, we show that one can get novel series representations for the generalized gamma process and the stable beta process.
Beyond the Chinese Restaurant and Pitman-Yor processes: Statistical Models with Double Power-law Behavior
Bayesian nonparametric approaches, in particular the Pitman-Yor process and the associated two-parameter Chinese Restaurant process, have been successfully used in applications where the data exhibit a power-law behavior.
A Bayesian model for sparse graphs with flexible degree distribution and overlapping community structure
We consider a non-projective class of inhomogeneous random graph models with interpretable parameters and a number of interesting asymptotic properties.
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances.
ADAPTIVE NETWORK SPARSIFICATION VIA DEPENDENT VARIATIONAL BETA-BERNOULLI DROPOUT
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Deep Mixed Effect Model using Gaussian Processes: A Personalized and Reliable Prediction for Healthcare
We present a personalized and reliable prediction model for healthcare, which can provide individually tailored medical services such as diagnosis, disease treatment, and prevention.
Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learning
The goal of few-shot learning is to learn a classifier that generalizes well even when trained with a limited number of training instances per class.
Uncertainty-Aware Attention for Reliable Interpretation and Prediction
Attention mechanism is effective in both focusing the deep learning models on relevant features and interpreting them.
DropMax: Adaptive Stochastic Softmax
Moreover, the learning of dropout probabilities for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.
DropMax: Adaptive Variational Softmax
Moreover, the learning of dropout rates for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes.
Bayesian inference on random simple graphs with power law degree distributions
The BFRY random variables are well approximated by gamma random variables in a variational Bayesian inference routine, which we apply to several network datasets for which power law degree distributions are a natural assumption.
Finite-Dimensional BFRY Priors and Variational Bayesian Inference for Power Law Models
Bayesian nonparametric methods based on the Dirichlet process (DP), gamma process and beta process, have proven effective in capturing aspects of various datasets arising in machine learning.
Tree-Guided MCMC Inference for Normalized Random Measure Mixture Models
Normalized random measures (NRMs) provide a broad class of discrete random measures that are often used as priors for Bayesian nonparametric models.
Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility
Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge.
