Systematic Misestimation of Machine Learning Performance in Neuroimaging Studies of Depression

no code implementations • 13 Dec 2019 • Claas Flint, Micah Cearns, Nils Opel, Ronny Redlich, David M. A. Mehler, Daniel Emden, Nils R. Winter, Ramona Leenings, Simon B. Eickhoff, Tilo Kircher, Axel Krug, Igor Nenadic, Volker Arolt, Scott Clark, Bernhard T. Baune, Xiaoyi Jiang, Udo Dannlowski, Tim Hahn

We currently observe a disconcerting phenomenon in machine learning studies in psychiatry: While we would expect larger samples to yield better results due to the availability of more data, larger machine learning studies consistently show much weaker performance than the numerous small-scale studies.

BIG-bench Machine Learning General Classification

Bayesian Optimization for Machine Learning : A Practical Guidebook

no code implementations • 14 Dec 2016 • Ian Dewancker, Michael McCourt, Scott Clark

The engineering of machine learning systems is still a nascent field; relying on a seemingly daunting collection of quickly evolving tools and best practices.

Bayesian Optimization BIG-bench Machine Learning

Evaluation System for a Bayesian Optimization Service

no code implementations • 19 May 2016 • Ian Dewancker, Michael McCourt, Scott Clark, Patrick Hayes, Alexandra Johnson, George Ke

Bayesian optimization is an elegant solution to the hyperparameter optimization problem in machine learning.

Bayesian Optimization BIG-bench Machine Learning +1

A Stratified Analysis of Bayesian Optimization Methods

no code implementations • 31 Mar 2016 • Ian Dewancker, Michael McCourt, Scott Clark, Patrick Hayes, Alexandra Johnson, George Ke

Empirical analysis serves as an important complement to theoretical analysis for studying practical Bayesian optimization.

Bayesian Optimization Hyperparameter Optimization