Undecidability of Underfitting in Learning Algorithms

no code implementations • 4 Feb 2021 • Sonia Sehra, David Flores, George D. Montanez

Using recent machine learning results that present an information-theoretic perspective on underfitting and overfitting, we prove that deciding whether an encodable learning algorithm will always underfit a dataset, even if given unlimited training time, is undecidable.

BIG-bench Machine Learning

An Information-Theoretic Perspective on Overfitting and Underfitting

no code implementations • 12 Oct 2020 • Daniel Bashir, George D. Montanez, Sonia Sehra, Pedro Sandoval Segura, Julius Lauw

We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.

BIG-bench Machine Learning

The Labeling Distribution Matrix (LDM): A Tool for Estimating Machine Learning Algorithm Capacity

no code implementations • 23 Dec 2019 • Pedro Sandoval Segura, Julius Lauw, Daniel Bashir, Kinjal Shah, Sonia Sehra, Dominique Macias, George Montanez

Algorithm performance in supervised learning is a combination of memorization, generalization, and luck.

BIG-bench Machine Learning Memorization