Search Results for author: Sanae Lotfi

Found 7 papers, 4 papers with code

Stochastic Damped L-BFGS with Controlled Norm of the Hessian Approximation

no code implementations10 Dec 2020 Sanae Lotfi, Tiphaine Bonniot de Ruisselet, Dominique Orban, Andrea Lodi

We propose a new stochastic variance-reduced damped L-BFGS algorithm, where we leverage estimates of bounds on the largest and smallest eigenvalues of the Hessian approximation to balance its quality and conditioning.

regression

Loss Surface Simplexes for Mode Connecting Volumes and Fast Ensembling

1 code implementation25 Feb 2021 Gregory W. Benton, Wesley J. Maddox, Sanae Lotfi, Andrew Gordon Wilson

In this paper, we show that there are mode-connecting simplicial complexes that form multi-dimensional manifolds of low loss, connecting many independently trained models.

Dangers of Bayesian Model Averaging under Covariate Shift

1 code implementation NeurIPS 2021 Pavel Izmailov, Patrick Nicholson, Sanae Lotfi, Andrew Gordon Wilson

Approximate Bayesian inference for neural networks is considered a robust alternative to standard training, often providing good performance on out-of-distribution data.

Bayesian Inference

Adaptive First- and Second-Order Algorithms for Large-Scale Machine Learning

no code implementations29 Nov 2021 Sanae Lotfi, Tiphaine Bonniot de Ruisselet, Dominique Orban, Andrea Lodi

In this paper, we consider both first- and second-order techniques to address continuous optimization problems arising in machine learning.

BIG-bench Machine Learning Stochastic Optimization

Bayesian Model Selection, the Marginal Likelihood, and Generalization

1 code implementation23 Feb 2022 Sanae Lotfi, Pavel Izmailov, Gregory Benton, Micah Goldblum, Andrew Gordon Wilson

We provide a partial remedy through a conditional marginal likelihood, which we show is more aligned with generalization, and practically valuable for large-scale hyperparameter learning, such as in deep kernel learning.

Model Selection Neural Architecture Search

PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization

1 code implementation24 Nov 2022 Sanae Lotfi, Marc Finzi, Sanyam Kapoor, Andres Potapczynski, Micah Goldblum, Andrew Gordon Wilson

While there has been progress in developing non-vacuous generalization bounds for deep neural networks, these bounds tend to be uninformative about why deep learning works.

Generalization Bounds Transfer Learning

Non-Vacuous Generalization Bounds for Large Language Models

no code implementations28 Dec 2023 Sanae Lotfi, Marc Finzi, Yilun Kuang, Tim G. J. Rudner, Micah Goldblum, Andrew Gordon Wilson

Modern language models can contain billions of parameters, raising the question of whether they can generalize beyond the training data or simply regurgitate their training corpora.

Generalization Bounds valid

Cannot find the paper you are looking for? You can Submit a new open access paper.