Hyperparameter Search
Random Search
Random Search replaces the exhaustive enumeration of all combinations by selecting them randomly. This can be simply applied to the discrete setting described above, but also generalizes to continuous and mixed spaces. It can outperform Grid search, especially when only a small number of hyperparameters affects the final performance of the machine learning algorithm. In this case, the optimization problem is said to have a low intrinsic dimensionality. Random Search is also embarrassingly parallel, and additionally allows the inclusion of prior knowledge by specifying the distribution from which to sample.
Extracted from Wikipedia
Source Paper
Image Source: BERGSTRA AND BENGIO
Papers
| Paper | Code | Results | Date | Stars |
|---|
Tasks
| Task | Papers | Share |
|---|---|---|
| Bayesian Optimization | 45 | 15.68% |
| Reinforcement Learning (RL) | 27 | 9.41% |
| BIG-bench Machine Learning | 20 | 6.97% |
| Image Classification | 9 | 3.14% |
| General Classification | 7 | 2.44% |
| Evolutionary Algorithms | 6 | 2.09% |
| Time Series Analysis | 6 | 2.09% |
| Decision Making | 5 | 1.74% |
| Continuous Control | 5 | 1.74% |
Usage Over Time
Components
| Component | Type |
|
|---|---|---|
| 🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |
