no code implementations • 14 Feb 2024 • Jason Yoo, Yunpeng Liu, Frank Wood, Geoff Pleiss
Our solution, Layerwise Proximal Replay (LPR), balances learning from new and replay data while only allowing for gradual changes in the hidden activation of past data.
no code implementations • 14 Feb 2024 • Alexandre Bouchard-Côté, Trevor Campbell, Geoff Pleiss, Nikola Surjanovic
This paper is intended to appear as a chapter for the Handbook of Markov Chain Monte Carlo.
1 code implementation • 7 Feb 2024 • Agustinus Kristiadi, Felix Strieth-Kalthoff, Marta Skreta, Pascal Poupart, Alán Aspuru-Guzik, Geoff Pleiss
Bayesian optimization (BO) is an essential part of such workflows, enabling scientists to leverage prior domain knowledge into efficient exploration of a large molecular space.
1 code implementation • 26 Oct 2023 • Kaiwen Wu, Jonathan Wenger, Haydn Jones, Geoff Pleiss, Jacob R. Gardner
Training and inference in Gaussian processes (GPs) require solving linear systems with $n\times n$ kernel matrices.
1 code implementation • NeurIPS 2023 • Andres Potapczynski, Marc Finzi, Geoff Pleiss, Andrew Gordon Wilson
In this paper, we propose a simple but general framework for large-scale linear algebra problems in machine learning, named CoLA (Compositional Linear Algebra).
no code implementations • 1 Feb 2023 • Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, John P. Cunningham
Here we demonstrate that these intuitions do not apply to high-capacity neural network ensembles (deep ensembles), and in fact the opposite is often true.
1 code implementation • 30 May 2022 • Jonathan Wenger, Geoff Pleiss, Marvin Pförtner, Philipp Hennig, John P. Cunningham
For any method in this class, we prove (i) convergence of its posterior mean in the associated RKHS, (ii) decomposability of its combined posterior covariance into mathematical and computational covariances, and (iii) that the combined variance is a tight worst-case bound for the squared error between the method's posterior mean and the latent function.
1 code implementation • 14 Feb 2022 • Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, Richard Zemel, John P. Cunningham
While deep ensembles are a practical way to achieve improvements to predictive power, uncertainty quantification, and robustness, our results show that these improvements can be replicated by a (larger) single model.
no code implementations • 3 Feb 2022 • Luhuan Wu, Geoff Pleiss, John Cunningham
Variational approximations to Gaussian processes (GPs) typically use a small set of inducing points to form a low-rank approximation to the covariance matrix.
no code implementations • 1 Jul 2021 • Jonathan Wenger, Geoff Pleiss, Philipp Hennig, John P. Cunningham, Jacob R. Gardner
While preconditioning is well understood in the context of CG, we demonstrate that it can also accelerate convergence and reduce variance of the estimates for the log-determinant and its derivative.
1 code implementation • NeurIPS 2021 • Geoff Pleiss, John P. Cunningham
Our analysis in this paper decouples capacity and width via the generalization of neural networks to Deep Gaussian Processes (Deep GP), a class of nonparametric hierarchical models that subsume neural nets.
1 code implementation • NeurIPS 2021 • Anthony L. Caterini, Gabriel Loaiza-Ganem, Geoff Pleiss, John P. Cunningham
Normalizing flows are invertible neural networks with tractable change-of-volume terms, which allow optimization of their parameters to be efficiently performed via maximum likelihood.
no code implementations • 24 May 2021 • Martin Jankowiak, Geoff Pleiss
We introduce a simple and scalable method for training Gaussian process (GP) models that exploits cross-validation and nearest neighbor truncation.
1 code implementation • 28 Feb 2021 • Luhuan Wu, Andrew Miller, Lauren Anderson, Geoff Pleiss, David Blei, John Cunningham
In this work, we introduce the hierarchical inducing point GP (HIP-GP), a scalable inter-domain GP inference method that enables us to improve the approximation accuracy by increasing the number of inducing points to the millions.
1 code implementation • 12 Feb 2021 • Andres Potapczynski, Luhuan Wu, Dan Biderman, Geoff Pleiss, John P. Cunningham
In the case of RFF, we show that the bias-to-variance conversion is indeed a trade-off: the additional variance proves detrimental to optimization.
2 code implementations • NeurIPS Workshop ICBINB 2020 • Elliott Gordon-Rodriguez, Gabriel Loaiza-Ganem, Geoff Pleiss, John P. Cunningham
Modern deep learning is primarily an experimental science, in which empirical advances occasionally come at the expense of probabilistic rigor.
1 code implementation • NeurIPS 2020 • Geoff Pleiss, Martin Jankowiak, David Eriksson, Anil Damle, Jacob R. Gardner
Matrix square roots and their inverses arise frequently in machine learning, e. g., when sampling from high-dimensional Gaussians $\mathcal{N}(\mathbf 0, \mathbf K)$ or whitening a vector $\mathbf b$ against covariance matrix $\mathbf K$.
no code implementations • 21 Feb 2020 • Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner
We introduce Deep Sigma Point Processes, a class of parametric models inspired by the compositional structure of Deep Gaussian Processes (DGPs).
2 code implementations • NeurIPS 2020 • Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger
Not all data in a typical training set help with generalization; some samples can be overly ambiguous or outrightly mislabeled.
no code implementations • 8 Jan 2020 • Gao Huang, Zhuang Liu, Geoff Pleiss, Laurens van der Maaten, Kilian Q. Weinberger
Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output.
no code implementations • ICML 2020 • Martin Jankowiak, Geoff Pleiss, Jacob R. Gardner
In an extensive empirical comparison with a number of alternative methods for scalable GP regression, we find that the resulting predictive distributions exhibit significantly better calibrated uncertainties and higher log likelihoods--often by as much as half a nat per datapoint.
no code implementations • 25 Sep 2019 • Geoff Pleiss, Amauri Souza, Joseph Kim, Boyi Li, Kilian Q. Weinberger
Neural network out-of-distribution (OOD) detection aims to identify when a model is unable to generalize to new inputs, either due to covariate shift or anomalous data.
Out-of-Distribution Detection Out of Distribution (OOD) Detection +1
no code implementations • 25 Sep 2019 • Geoff Pleiss, Tianyi Zhang, Ethan R. Elenberg, Kilian Q. Weinberger
This paper introduces a new method to discover mislabeled training samples and to mitigate their impact on the training process of deep networks.
1 code implementation • ICLR 2020 • Yurong You, Yan Wang, Wei-Lun Chao, Divyansh Garg, Geoff Pleiss, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
In this paper we provide substantial advances to the pseudo-LiDAR framework through improvements in stereo depth estimation.
3D Object Detection From Stereo Images Autonomous Driving +2
3 code implementations • NeurIPS 2019 • Ke Alexander Wang, Geoff Pleiss, Jacob R. Gardner, Stephen Tyree, Kilian Q. Weinberger, Andrew Gordon Wilson
Gaussian processes (GPs) are flexible non-parametric models, with a capacity that grows with the available data.
4 code implementations • NeurIPS 2018 • Jacob R. Gardner, Geoff Pleiss, David Bindel, Kilian Q. Weinberger, Andrew Gordon Wilson
Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware.
1 code implementation • ICML 2018 • Geoff Pleiss, Jacob R. Gardner, Kilian Q. Weinberger, Andrew Gordon Wilson
One of the most compelling features of Gaussian process (GP) regression is its ability to provide well-calibrated posterior distributions.
1 code implementation • 24 Feb 2018 • Jacob R. Gardner, Geoff Pleiss, Ruihan Wu, Kilian Q. Weinberger, Andrew Gordon Wilson
Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs).
1 code implementation • NeurIPS 2017 • Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, Kilian Q. Weinberger
The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models.
6 code implementations • 21 Jul 2017 • Geoff Pleiss, Danlu Chen, Gao Huang, Tongcheng Li, Laurens van der Maaten, Kilian Q. Weinberger
A 264-layer DenseNet (73M parameters), which previously would have been infeasible to train, can now be trained on a single workstation with 8 NVIDIA Tesla M40 GPUs.
17 code implementations • ICML 2017 • Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger
Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications.
10 code implementations • 1 Apr 2017 • Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E. Hopcroft, Kilian Q. Weinberger
In this paper, we propose a method to obtain the seemingly contradictory goal of ensembling multiple neural networks at no additional training cost.
2 code implementations • CVPR 2017 • Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, Kilian Weinberger
We propose Deep Feature Interpolation (DFI), a new data-driven baseline for automatic high-resolution image transformation.