Search Results for author:
Found 7 papers, 0 papers with code
Undecidability of Underfitting in Learning Algorithms
Using recent machine learning results that present an information-theoretic perspective on underfitting and overfitting, we prove that deciding whether an encodable learning algorithm will always underfit a dataset, even if given unlimited training time, is undecidable.
An Information-Theoretic Perspective on Overfitting and Underfitting
We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.
Limits of Transfer Learning
To address this, we prove several novel results related to transfer learning, showing the need to carefully select which sets of information to transfer and the need for dependence between transferred information and target problems.
The Bias-Expressivity Trade-off
Learning algorithms need bias to generalize and perform better than random guessing.
The Futility of Bias-Free Learning and Search
For a given degree of bias towards a fixed target, we show that the proportion of favorable information resources is strictly bounded from above.
The Famine of Forte: Few Search Problems Greatly Favor Your Algorithm
Casting machine learning as a type of search, we demonstrate that the proportion of problems that are favorable for a fixed algorithm is strictly bounded, such that no single algorithm can perform well over a large fraction of them.
The LICORS Cabinet: Nonparametric Algorithms for Spatio-temporal Prediction
Spatio-temporal data is intrinsically high dimensional, so unsupervised modeling is only feasible if we can exploit structure in the process.
