Search Results for author: Geoff Pleiss

Found 33 papers, 22 papers with code

Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning

no code implementations14 Feb 2024 Jason Yoo, Yunpeng Liu, Frank Wood, Geoff Pleiss

Our solution, Layerwise Proximal Replay (LPR), balances learning from new and replay data while only allowing for gradual changes in the hidden activation of past data.

Continual Learning

MCMC-driven learning

no code implementations14 Feb 2024 Alexandre Bouchard-Côté, Trevor Campbell, Geoff Pleiss, Nikola Surjanovic

This paper is intended to appear as a chapter for the Handbook of Markov Chain Monte Carlo.

Variational Inference

A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?

1 code implementation7 Feb 2024 Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta, Pascal Poupart, Alán Aspuru-Guzik, Geoff Pleiss

Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior domain knowledge into efficient exploration of a large molecular space.

Bayesian Optimization Efficient Exploration

Large-Scale Gaussian Processes via Alternating Projection

1 code implementation26 Oct 2023 Kaiwen Wu, Jonathan Wenger, Haydn Jones, Geoff Pleiss, Jacob R. Gardner

Training and inference in Gaussian processes (GPs) require solving linear systems with $n\times n$ kernel matrices.

Gaussian Processes Hyperparameter Optimization

CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra

1 code implementation NeurIPS 2023 Andres Potapczynski, Marc Finzi, Geoff Pleiss, Andrew Gordon Wilson

In this paper, we propose a simple but general framework for large-scale linear algebra problems in machine learning, named CoLA (Compositional Linear Algebra).

CoLA Gaussian Processes

Pathologies of Predictive Diversity in Deep Ensembles

no code implementations1 Feb 2023 Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, John P. Cunningham

Here we demonstrate that these intuitions do not apply to high-capacity neural network ensembles (deep ensembles), and in fact the opposite is often true.

Posterior and Computational Uncertainty in Gaussian Processes

1 code implementation30 May 2022 Jonathan Wenger, Geoff Pleiss, Marvin Pförtner, Philipp Hennig, John P. Cunningham

For any method in this class, we prove (i) convergence of its posterior mean in the associated RKHS, (ii) decomposability of its combined posterior covariance into mathematical and computational covariances, and (iii) that the combined variance is a tight worst-case bound for the squared error between the method's posterior mean and the latent function.

Gaussian Processes

Deep Ensembles Work, But Are They Necessary?

1 code implementation14 Feb 2022 Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, Richard Zemel, John P. Cunningham

While deep ensembles are a practical way to achieve improvements to predictive power, uncertainty quantification, and robustness, our results show that these improvements can be replicated by a (larger) single model.

Uncertainty Quantification

Variational Nearest Neighbor Gaussian Process

no code implementations3 Feb 2022 Luhuan Wu, Geoff Pleiss, John Cunningham

Variational approximations to Gaussian processes (GPs) typically use a small set of inducing points to form a low-rank approximation to the covariance matrix.

Gaussian Processes Stochastic Optimization

Preconditioning for Scalable Gaussian Process Hyperparameter Optimization

no code implementations1 Jul 2021 Jonathan Wenger, Geoff Pleiss, Philipp Hennig, John P. Cunningham, Jacob R. Gardner

While preconditioning is well understood in the context of CG, we demonstrate that it can also accelerate convergence and reduce variance of the estimates for the log-determinant and its derivative.

Gaussian Processes Hyperparameter Optimization

The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective

1 code implementation NeurIPS 2021 Geoff Pleiss, John P. Cunningham

Our analysis in this paper decouples capacity and width via the generalization of neural networks to Deep Gaussian Processes (Deep GP), a class of nonparametric hierarchical models that subsume neural nets.

Gaussian Processes L2 Regularization

Rectangular Flows for Manifold Learning

1 code implementation NeurIPS 2021 Anthony L. Caterini, Gabriel Loaiza-Ganem, Geoff Pleiss, John P. Cunningham

Normalizing flows are invertible neural networks with tractable change-of-volume terms, which allow optimization of their parameters to be efficiently performed via maximum likelihood.

Density Estimation Out-of-Distribution Detection

Scalable Cross Validation Losses for Gaussian Process Models

no code implementations24 May 2021 Martin Jankowiak, Geoff Pleiss

We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation.

Classification Multi-class Classification +2

Hierarchical Inducing Point Gaussian Process for Inter-domain Observations

1 code implementation28 Feb 2021 Luhuan Wu, Andrew Miller, Lauren Anderson, Geoff Pleiss, David Blei, John Cunningham

In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions.

Gaussian Processes

Bias-Free Scalable Gaussian Processes via Randomized Truncations

1 code implementation12 Feb 2021 Andres Potapczynski, Luhuan Wu, Dan Biderman, Geoff Pleiss, John P. Cunningham

In the case of RFF, we show that the bias-to-variance conversion is indeed a trade-off: the additional variance proves detrimental to optimization.

Gaussian Processes

Uses and Abuses of the Cross-Entropy Loss: Case Studies in Modern Deep Learning

2 code implementations NeurIPS Workshop ICBINB 2020 Elliott Gordon-Rodriguez, Gabriel Loaiza-Ganem, Geoff Pleiss, John P. Cunningham

Modern deep learning is primarily an experimental science, in which empirical advances occasionally come at the expense of probabilistic rigor.

Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization

1 code implementation NeurIPS 2020 Geoff Pleiss, Martin Jankowiak, David Eriksson, Anil Damle, Jacob R. Gardner

Matrix square roots and their inverses arise frequently in machine learning, e. g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$.

Bayesian Optimization Gaussian Processes

Deep Sigma Point Processes

no code implementations21 Feb 2020 Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner

We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs).

Gaussian Processes Point Processes +1

Identifying Mislabeled Data using the Area Under the Margin Ranking

2 code implementations NeurIPS 2020 Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger

Not all data in a typical training set help with generalization; some samples can be overly ambiguous or outrightly mislabeled.

Convolutional Networks with Dense Connectivity

no code implementations8 Jan 2020 Gao Huang, Zhuang Liu, Geoff Pleiss, Laurens van der Maaten, Kilian Q. Weinberger

Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output.

Object Recognition

Parametric Gaussian Process Regressors

no code implementations ICML 2020 Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner

In an extensive empirical comparison with a number of alternative methods for scalable GP regression, we find that the resulting predictive distributions exhibit significantly better calibrated uncertainties and higher log likelihoods--often by as much as half a nat per datapoint.

regression Variational Inference

Neural Network Out-of-Distribution Detection for Regression Tasks

no code implementations25 Sep 2019 Geoff Pleiss, Amauri Souza, Joseph Kim, Boyi Li, Kilian Q. Weinberger

Neural network out-of-distribution (OOD) detection aims to identify when a model is unable to generalize to new inputs, either due to covariate shift or anomalous data.

Out-of-Distribution Detection Out of Distribution (OOD) Detection +1

Detecting Noisy Training Data with Loss Curves

no code implementations25 Sep 2019 Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger

This paper introduces a new method to discover mislabeled training samples and to mitigate their impact on the training process of deep networks.

GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration

4 code implementations NeurIPS 2018 Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, Andrew Gordon Wilson

Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware.

Gaussian Processes

Constant-Time Predictive Distributions for Gaussian Processes

1 code implementation ICML 2018 Geoff Pleiss, Jacob R. Gardner, Kilian Q. Weinberger, Andrew Gordon Wilson

One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions.

Gaussian Processes regression

Product Kernel Interpolation for Scalable Gaussian Processes

1 code implementation24 Feb 2018 Jacob R. Gardner, Geoff Pleiss, Ruihan Wu, Kilian Q. Weinberger, Andrew Gordon Wilson

Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs).

Gaussian Processes

On Fairness and Calibration

1 code implementation NeurIPS 2017 Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, Kilian Q. Weinberger

The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models.

Fairness General Classification

Memory-Efficient Implementation of DenseNets

6 code implementations21 Jul 2017 Geoff Pleiss, Danlu Chen, Gao Huang, Tongcheng Li, Laurens van der Maaten, Kilian Q. Weinberger

A 264-layer DenseNet (73M parameters), which previously would have been infeasible to train, can now be trained on a single workstation with 8 NVIDIA Tesla M40 GPUs.

On Calibration of Modern Neural Networks

17 code implementations ICML 2017 Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger

Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications.

Document Classification General Classification

Snapshot Ensembles: Train 1, get M for free

10 code implementations1 Apr 2017 Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E. Hopcroft, Kilian Q. Weinberger

In this paper, we propose a method to obtain the seemingly contradictory goal of ensembling multiple neural networks at no additional training cost.

Deep Feature Interpolation for Image Content Changes

2 code implementations CVPR 2017 Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, Kilian Weinberger

We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation.

Cannot find the paper you are looking for? You can Submit a new open access paper.