Search Results for author:
Found 30 papers, 13 papers with code
Scaling Exponents Across Parameterizations and Optimizers
Robust and effective scaling of models from small to large width typically requires the precise adjustment of many algorithmic and architectural details, such as parameterization and optimizer choices.
Speed Limits for Deep Learning
State-of-the-art neural networks require extreme computational power to train.
Variational Prediction
In this paper, we present variational prediction, a technique for directly learning a variational approximation to the posterior predictive distribution using a variational bound.
Weighted Ensemble Self-Supervised Learning
Ensembling has proven to be a powerful technique for boosting model performance, uncertainty estimation, and robustness in supervised learning.
Bayesian Imitation Learning for End-to-End Mobile Manipulation
In this work we investigate and demonstrate benefits of a Bayesian approach to imitation learning from multiple sensor inputs, as applied to the task of opening office doors with a mobile manipulator.
A Closer Look at the Adversarial Robustness of Information Bottleneck Models
We study the adversarial robustness of information bottleneck models for classification.
Does Knowledge Distillation Really Work?
Knowledge distillation is a popular technique for training a small student network to emulate a larger teacher model, such as an ensemble of networks.
PAC$^m$-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime
The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk".
Density of States Estimation for Out-of-Distribution Detection
Drawing on the statistical physics notion of ``density of states,'' the DoSE decision rule avoids direct comparison of model probabilities, and instead utilizes the ``probability of the model probability,'' or indeed the frequency of any reasonable statistic.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
CEB Improves Model Robustness
We demonstrate that the Conditional Entropy Bottleneck (CEB) can improve model robustness.
Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Neural Tangents is a library designed to enable research into infinite-width neural networks.
Information in Infinite Ensembles of Infinitely-Wide Neural Networks
In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks.
Variational Predictive Information Bottleneck
In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional.
On Predictive Information in RNNs
Certain biological neurons demonstrate a remarkable capability to optimally compress the history of sensory inputs while being maximally informative about the future.
On Predictive Information Sub-optimality of RNNs
Certain biological neurons demonstrate a remarkable capability to optimally compress the history of sensory inputs while being maximally informative about the future.
On Variational Bounds of Mutual Information
Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging.
On the Use of ArXiv as a Dataset
We use this pipeline to extract and analyze a 6. 7 million edge citation graph, with an 11 billion word corpus of full-text research articles.
$β$-VAEs can retain label information even at high compression
In this paper, we investigate the degree to which the encoding of a $\beta$-VAE captures label information across multiple architectures on Binary Static MNIST and Omniglot.
WAIC, but Why? Generative Ensembles for Robust Anomaly Detection
Machine learning models encounter Out-of-Distribution (OoD) errors when the data seen at test time are generated from a different stochastic generator than the one used to generate the training data.
TherML: The Thermodynamics of Machine Learning
In this work we offer an information-theoretic framework for representation learning that connects with a wide class of existing objectives in machine learning.
TherML: Thermodynamics of Machine Learning
In this work we offer a framework for reasoning about a wide class of existing objectives in machine learning.
Uncertainty in the Variational Information Bottleneck
We present a simple case study, demonstrating that Variational Information Bottleneck (VIB) can improve a network's classification calibration as well as its ability to detect out-of-distribution data.
GILBO: One Metric to Measure Them All
We propose a simple, tractable lower bound on the mutual information contained in the joint generative density of any latent variable generative model: the GILBO (Generative Information Lower BOund).
Fixing a Broken ELBO
Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models.
Jeffrey's prior sampling of deep sigmoidal networks
Neural networks have been shown to have a remarkable ability to uncover low dimensional structure in data: the space of possible reconstructed images form a reduced model manifold in image space.
Improved generator objectives for GANs
We present a framework to understand GAN training as alternating density ratio estimation and approximate divergence minimization.
Deep Variational Information Bottleneck
We present a variational approximation to the information bottleneck of Tishby et al. (1999).
Clustering via Content-Augmented Stochastic Blockmodels
Much of the data being created on the web contains interactions between users and items.
Canonical Sectors and Evolution of Firms in the US Stock Markets
A classification of companies into sectors of the economy is important for macroeconomic analysis and for investments into the sector-specific financial indices and exchange traded funds (ETFs).
Text Segmentation based on Semantic Word Embeddings
We explore the use of semantic word embeddings in text segmentation algorithms, including the C99 segmentation algorithm and new algorithms inspired by the distributed word vector representation.
