Search Results for author: Michael Munn

Found 8 papers, 2 papers with code

Training in reverse: How iteration order influences convergence and stability in deep learning

no code implementations3 Feb 2025 Benoit Dherin, Benny Avelin, Anders Karlsson, Hanna Mazzawi, Javier Gonzalvo, Michael Munn

Despite exceptional achievements, training neural networks remains computationally expensive and is often plagued by instabilities that can degrade convergence.

Leveraging free energy in pretraining model selection for improved fine-tuning

no code implementations8 Oct 2024 Michael Munn, Susan Wei

Recent advances in artificial intelligence have been fueled by the development of foundation models such as BERT, GPT, T5, and Vision Transformers.

Model Selection

A Margin-based Multiclass Generalization Bound via Geometric Complexity

no code implementations28 May 2024 Michael Munn, Benoit Dherin, Javier Gonzalvo

We derive a new upper bound on the generalization error which scales with the margin-normalized geometric complexity of the network and which holds for a broad family of data distributions and model classes.

Generalization Bounds

The Impact of Geometric Complexity on Neural Collapse in Transfer Learning

no code implementations24 May 2024 Michael Munn, Benoit Dherin, Javier Gonzalvo

Many of the recent remarkable advances in computer vision and language models can be attributed to the success of transfer learning via the pre-training of large foundation models.

Transfer Learning

Unified Functional Hashing in Automatic Machine Learning

1 code implementation10 Feb 2023 Ryan Gillard, Stephen Jonany, Yingjie Miao, Michael Munn, Connal de Souza, Jonathan Dungay, Chen Liang, David R. So, Quoc V. Le, Esteban Real

In this paper, we show that large efficiency gains can be obtained by employing a fast unified functional hash, especially through the functional equivalence caching technique, which we also present.

Neural Architecture Search

Why neural networks find simple solutions: the many regularizers of geometric complexity

no code implementations27 Sep 2022 Benoit Dherin, Michael Munn, Mihaela Rosca, David G. T. Barrett

Using a combination of theoretical arguments and empirical results, we show that many common training heuristics such as parameter norm regularization, spectral norm regularization, flatness regularization, implicit gradient regularization, noise regularization and the choice of parameter initialization all act to control geometric complexity, providing a unifying framework in which to characterize the behavior of deep learning models.

Deep Learning

The Geometric Occam's Razor Implicit in Deep Learning

no code implementations30 Nov 2021 Benoit Dherin, Michael Munn, David G. T. Barrett

We argue that over-parameterized neural networks trained with stochastic gradient descent are subject to a Geometric Occam's Razor; that is, these networks are implicitly regularized by the geometric model complexity.

ARC Deep Learning

COT-GAN: Generating Sequential Data via Causal Optimal Transport

2 code implementations NeurIPS 2020 Tianlin Xu, Li K. Wenliang, Michael Munn, Beatrice Acciaio

We introduce COT-GAN, an adversarial algorithm to train implicit generative models optimized for producing sequential data.

Time Series Time Series Analysis

Cannot find the paper you are looking for? You can Submit a new open access paper.