Random Search

Random Search replaces the exhaustive enumeration of all combinations by selecting them randomly. This can be simply applied to the discrete setting described above, but also generalizes to continuous and mixed spaces. It can outperform Grid search, especially when only a small number of hyperparameters affects the final performance of the machine learning algorithm. In this case, the optimization problem is said to have a low intrinsic dimensionality. Random Search is also embarrassingly parallel, and additionally allows the inclusion of prior knowledge by specifying the distribution from which to sample.

Extracted from Wikipedia

Source Paper

Latest Papers

PAPER DATE
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians
| Juhan BaeRoger Grosse
2020-10-26
Robot Design With Neural Networks, MILP Solvers and Active Learning
Sanjai NarainEmily MakDana CheeTodd HusterJeremy CohenKishore PochirajuBrendan EnglotNiraj K. JhaKarthik Narayan
2020-10-19
Choosing News Topics to Explain Stock Market Returns
Paul GlassermanKriste KrstovskiPaul LaliberteHarry Mamaysky
2020-10-14
Vulnerability of Face Recognition Systems Against Composite Face Reconstruction Attack
Hadi MansourifarWeidong Shi
2020-08-23
Adversarial Imitation Learning via Random Search
MyungJae ShinJoongheon Kim
2020-08-21
Can weight sharing outperform random architecture search? An investigation with TuNAS
Gabriel BenderHanxiao LiuBo ChenGrace ChuShuyang ChengPieter-Jan KindermansQuoc Le
2020-08-13
Robust Long-Term Object Tracking via Improved Discriminative Model Prediction
| Seokeon ChoiJunhyun LeeYunsung LeeAlexander Hauptmann
2020-08-11
Quantity vs. Quality: On Hyperparameter Optimization for Deep Reinforcement Learning
Lars HertelPierre BaldiDaniel L. Gillen
2020-07-29
A Gradient-based Bilevel Optimization Approach for Tuning Hyperparameters in Machine Learning
Ankur SinhaTanmay KhandaitRaja Mohanty
2020-07-21
Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap
| Eric HeidenDavid MillardErwin CoumansGaurav S. Sukhatme
2020-07-12
Prior-guided Bayesian Optimization
Artur SouzaLuigi NardiLeonardo B. OliveiraKunle OlukotunMarius LindauerFrank Hutter
2020-06-25
Hyperparameter Ensembles for Robustness and Uncertainty Quantification
Florian WenzelJasper SnoekDustin TranRodolphe Jenatton
2020-06-24
Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks
| Francesco CroceMaksym AndriushchenkoNaman D. SinghNicolas FlammarionMatthias Hein
2020-06-23
Learning Causal Models Online
| Khurram JavedMartha WhiteYoshua Bengio
2020-06-12
Bonsai-Net: One-Shot Neural Architecture Search via Differentiable Pruners
Rob GeadaDennis PrangleAndrew Stephen McGough
2020-06-12
AMER: Automatic Behavior Modeling and Interaction Exploration in Recommender System
Pengyu ZhaoKecheng XiaoYuanxing ZhangKaigui BianWei Yan
2020-06-10
Adaptation Strategies for Automated Machine Learning on Evolving Data
| Bilge CelikJoaquin Vanschoren
2020-06-09
AutoHAS: Differentiable Hyper-parameter and Architecture Search
Xuanyi DongMingxing TanAdams Wei YuDaiyi PengBogdan GabrysQuoc V. Le
2020-06-05
MLE-guided parameter search for task loss minimization in neural sequence modeling
Sean WelleckKyunghyun Cho
2020-06-04
A Robust Experimental Evaluation of Automated Multi-Label Classification Methods
Alex G. C. de SáCristiano G. PimentaGisele L. PappaAlex A. Freitas
2020-05-16
Out-of-the-box channel pruned networks
Ragav VenkatesanGurumurthy SwaminathanXiong ZhouAnna Luo
2020-04-30
Genetic programming approaches to learning fair classifiers
| William La CavaJason H. Moore
2020-04-28
Learning to Guide Random Search
| Ozan SenerVladlen Koltun
2020-04-25
Local Search is a Remarkably Strong Baseline for Neural Architecture Search
| T. Den OttelanderA. DushatskiyM. VirgolinP. A. N. Bosman
2020-04-20
Distributed Evolution of Deep Autoencoders
Jeff HajewskiSuely OliveiraXiaoyu Xing
2020-04-16
A reinforcement learning application of guided Monte Carlo Tree Search algorithm for beam orientation selection in radiation therapy
Azar Sadeghnejad-BarkousaraieGyanendra BoharaSteve JiangDan Nguyen
2020-04-14
A Modified Bayesian Optimization based Hyper-Parameter Tuning Approach for Extreme Gradient Boosting
Sayan PutatundaKiran Rama
2020-04-10
Learning Stabilizing Control Policies for a Tensegrity Hopper with Augmented Random Search
| Vladislav KurenkovHany HamedSergei Savin
2020-04-06
Weighted Random Search for Hyperparameter Optimization
Adrian-Catalin FloreaRazvan Andonie
2020-04-03
Weighted Random Search for CNN Hyperparameter Optimization
| Razvan AndonieAdrian-Catalin Florea
2020-03-30
RandomNet: Towards Fully Automatic Neural Architecture Design for Multimodal Learning
Stefano AllettoShenyang HuangVincent Francois-LavetYohei NakataGuillaume Rabusseau
2020-03-02
To Share or Not To Share: A Comprehensive Appraisal of Weight-Sharing
| Aloïs PourchotAlexis DucarougeOlivier Sigaud
2020-02-11
Novelty Producing Synaptic Plasticity
Anil YamanGiovanni IaccaDecebal Constantin MocanuGeorge FletcherMykola Pechenizkiy
2020-02-10
Machine learning enables completely automatic tuning of a quantum device faster than human experts
| H. MoonD. T. LennonJ. KirkpatrickN. M. van EsbroeckL. C. CamenzindLiuqi YuF. VigneauD. M. ZumbühlG. A. D. BriggsM. A OsborneD. SejdinovicE. A. LairdN. Ares
2020-01-08
Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem
Hesameddin MohammadiArmin ZareMahdi SoltanolkotabiMihailo R. Jovanović
2019-12-26
CNN-LSTM models for Multi-Speaker Source Separation using Bayesian Hyper Parameter Optimization
Jeroen ZegersHugo Van hamme
2019-12-19
COEGAN: Evaluating the Coevolution Effect in Generative Adversarial Networks
| Victor CostaNuno LourençoJoão CorreiaPenousal Machado
2019-12-12
Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS
Petro LiashchynskyiPavlo Liashchynskyi
2019-12-12
Augmented Random Search for Quadcopter Control: An alternative to Reinforcement Learning
Ashutosh Kumar TiwariSandeep Varma Nadimpalli
2019-11-28
Efficient Sample-based Neural Architecture Search with Learnable Predictor
Han ShiRenjie PiHang XuZhenguo LiJames T. KwokTong Zhang
2019-11-21
ImmuNeCS: Neural Committee Search by an Artificial Immune System
Luc FrachonWei PangGeorge M. Coghill
2019-11-18
FLO: Fast and Lightweight Hyperparameter Optimization for AutoML
| Chi WangQingyun Wu
2019-11-12
Auptimizer -- an Extensible, Open-Source Framework for Hyperparameter Tuning
| Jiayi LiuSamarth TripathiUnmesh KurupMohak Shah
2019-11-06
Policy Learning for Malaria Control
| Van Bach NguyenBelaid Mohamed KarimBao Long VuJörg SchlöttererMichael Granitzer
2019-10-20
Fully Parallel Hyperparameter Search: Reshaped Space-Filling
M. -L. CauwetC. CouprieJ. DehosP. LucJ. RapinM. RiviereF. TeytaudO. Teytaud
2019-10-18
Derivative-Free Optimization of Neural Networks using Local Search
| Ahmed AlyGianluca GuadagniJoanne Bechta Dugan
2019-10-15
Auto-Sizing the Transformer Network: Improving Speed, Efficiency, and Performance for Low-Resource Machine Translation
| Kenton MurrayJeffery KinnisonToan Q. NguyenWalter ScheirerDavid Chiang
2019-10-01
Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
Valerio PerroneHuibin ShenMatthias SeegerCedric ArchambeauRodolphe Jenatton
2019-09-27
Error Analysis of Elitist Random Search Heuristics
Yu ChenCong WangJun HeChengwang Xie
2019-09-03
MANAS: Multi-Agent Neural Architecture Search
Fabio Maria CarlucciPedro M EsperançaMarco SinghVictor GabillonAntoine YangHang XuZewei ChenJun Wang
2019-09-03
Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems
Mohsen ShahhosseiniGuiping HuHieu Pham
2019-08-14
Large-scale traffic signal control using machine learning: some traffic flow considerations
Jorge A. LavalHao Zhou
2019-08-07
A View on Deep Reinforcement Learning in System Optimization
Ameer Haj-AliNesreen K. AhmedTed WillkeJoseph GonzalezKrste AsanovicIon Stoica
2019-08-04
Hyperparameter Optimisation with Early Termination of Poor Performers
Dobromir MarinovDaniel Karapetyan
2019-07-19
FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control
Longxiang ShiShijian LiLongbing CaoLong YangGang ZhengGang Pan
2019-07-01
Max-Affine Regression: Provable, Tractable, and Near-Optimal Statistical Estimation
Avishek GhoshAshwin PananjadyAdityanand GuntuboyinaKannan Ramchandran
2019-06-21
Symbolic regression by uniform random global search
Sohrab Towfighi
2019-06-18
Derivative-Free Global Optimization Algorithms: Bayesian Method and Lipschitzian Approaches
Jiawei Zhang
2019-04-19
Understanding Neural Architecture Search Techniques
George AdamJonathan Lorraine
2019-03-31
AlphaX: eXploring Neural Architectures with Deep Neural Networks and Monte Carlo Tree Search
| Linnan WangYiyang ZhaoYuu JinnaiYuandong TianRodrigo Fonseca
2019-03-26
Quantifying error contributions of computational steps, algorithms and hyperparameter choices in image classification pipelines
Aritra ChowdhuryMalik Magdin-IsmailBulent Yener
2019-02-25
Bayes Optimal Early Stopping Policies for Black-Box Optimization
Matthew Streeter
2019-02-21
Quantifying contribution and propagation of error from computational steps, algorithms and hyperparameter choices in image classification pipelines
| Aritra ChowdhuryMalik Magdon-IsmailBulent Yener
2019-02-21
Random Search and Reproducibility for Neural Architecture Search
| Liam LiAmeet Talwalkar
2019-02-20
Learnable Embedding Space for Efficient Neural Architecture Compression
| Shengcao CaoXiaofang WangKris M. Kitani
2019-02-01
Universal Rules for Fooling Deep Neural Networks based Text Classification
Di LiDanilo Vasconcellos VargasSakurai Kouichi
2019-01-22
Multi-level CNN for lung nodule classification with Gaussian Process assisted hyperparameter optimization
Miao ZhangHuiqi LiJuan LyuSai Ho LingSteven Su
2019-01-02
Simple random search of static linear policies is competitive for reinforcement learning
Horia ManiaAurelia GuyBenjamin Recht
2018-12-01
Learning Multiple Defaults for Machine Learning Algorithms
Florian PfistererJan N. van RijnPhilipp ProbstAndreas MüllerBernd Bischl
2018-11-23
Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems
Miguel de PradoNuria PazosLuca Benini
2018-11-18
Learning data augmentation policies using augmented random search
Mingyang GengKele XuBo DingHuaimin WangLei Zhang
2018-11-12
EA-LSTM: Evolutionary Attention-based LSTM for Time Series Prediction
Youru LiZhenfeng ZhuDeqiang KongHua HanYao Zhao
2018-11-09
Graph HyperNetworks for Neural Architecture Search
Chris ZhangMengye RenRaquel Urtasun
2018-10-12
Deep Recurrent Electricity Theft Detection in AMI Networks with Random Tuning of Hyper-parameters
Mahmoud NabilMuhammad IsmailMohamed MahmoudMostafa ShahinKhalid QaraqeErchin Serpedin
2018-09-06
Did you take the pill? - Detecting Personal Intake of Medicine from Twitter
Debanjan MahataJasper FriedrichsRajiv Ratn ShahJing Jiang
2018-08-03
Neural Architecture Search using Deep Neural Networks and Monte Carlo Tree Search
| Linnan WangYiyang ZhaoYuu JinnaiYuandong TianRodrigo Fonseca
2018-05-18
#phramacovigilance - Exploring Deep Learning Techniques for Identifying Mentions of Medication Intake from Twitter
Debanjan MahataJasper FriedrichsHitkulRajiv Ratn Shah
2018-05-16
An LP-based hyperparameter optimization model for language modeling
Amir Hossein Akhavan RahnamaMehdi TolooNezer Jacob Zaidenberg
2018-03-29
InfyNLP at SMM4H Task 2: Stacked Ensemble of Shallow Convolutional Neural Networks for Identifying Personal Medication Intake from Twitter
Jasper FriedrichsDebanjan MahataShubham Gupta
2018-03-21
Simple random search provides a competitive approach to reinforcement learning
| Horia ManiaAurelia GuyBenjamin Recht
2018-03-19
Continuous-fidelity Bayesian Optimization with Knowledge Gradient
Jian WuPeter I. Frazier
2018-01-01
A Flexible Approach to Automated RNN Architecture Generation
Martin SchrimpfStephen MerityJames BradburyRichard Socher
2017-12-20
Hyperparameters Optimization in Deep Convolutional Neural Network / Bayesian Approach with Gaussian Process Prior
Pushparaja Murugan
2017-12-19
HyperPower: Power- and Memory-Constrained Hyper-Parameter Optimization for Neural Networks
Dimitrios StamoulisErmao CaiDa-Cheng JuanDiana Marculescu
2017-12-06
Computer-aided diagnosis of lung nodule using gradient tree boosting and Bayesian optimization
Mizuho NishioMitsuo NishizawaOsamu SugiyamaRyosuke KojimaMasahiro YakamiTomohiro KurodaKaori Togashi
2017-08-19
Critical Hyper-Parameters: No Random, No Cry
Olivier BousquetSylvain GellyKarol KurachOlivier TeytaudDamien Vincent
2017-06-10
Open Loop Hyperparameter Optimization and Determinantal Point Processes
Jesse DodgeKevin JamiesonNoah A. Smith
2017-06-06
Parallel and Distributed Thompson Sampling for Large-scale Accelerated Exploration of Chemical Space
José Miguel Hernández-LobatoJames RequeimaEdward O. Pyzer-KnappAlán Aspuru-Guzik
2017-06-06
Hyperparameter Optimization: A Spectral Approach
| Elad HazanAdam KlivansYang Yuan
2017-06-02
Diffusion geometry unravels the emergence of functional clusters in collective phenomena
Manlio De Domenico
2017-04-24
Black-Box Optimization in Machine Learning with Trust Region Based Derivative Free Algorithm
Hiva GhanbariKatya Scheinberg
2017-03-20
Drift Analysis and Evolutionary Algorithms Revisited
Johannes LenglerAngelika Steger
2016-08-10
Efficient Coarse-To-Fine PatchMatch for Large Displacement Optical Flow
| Yinlin HuRui SongYunsong Li
2016-06-01
apsis - Framework for Automated Optimization of Machine Learning Hyper Parameters
Frederik DiehlAndreas Jauch
2015-03-10
A Critical Reassessment of Evolutionary Algorithms on the cryptanalysis of the simplified data encryption standard algorithm
Fabien TeytaudCyril Fonlupt
2014-07-08
Kaggle LSHTC4 Winning Solution
Antti PuurulaJesse ReadAlbert Bifet
2014-05-03
State Transition Algorithm
Xiaojun ZhouChunhua YangWeihua Gui
2012-05-30
Algorithms for Hyper-Parameter Optimization
James S. BergstraRémi BardenetYoshua BengioBalázs Kégl
2011-12-01

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories