Early Stopping

Early Stopping is a regularization technique for deep neural networks that stops training when parameter updates no longer begin to yield improves on a validation set. In essence we store and update the current best parameters during training, and when parameter updates no longer yield an improvement (after a set number of iterations) we stop training and use the last best parameters. It works as a regularizer by restricting the optimization procedure to a smaller volume of parameter space.

Image Source: Ramazan Gençay

Latest Papers

PAPER DATE
Conditioning Trick for Training Stable GANs
Mohammad EsmaeilpourRaymel Alfonso SalloOlivier St-GeorgesPatrick CardinalAlessandro Lameiras Koerich
2020-10-12
PANDA -- Adapting Pretrained Features for Anomaly Detection
Tal ReissNiv CohenLiron BergmanYedid Hoshen
2020-10-12
SMYRF: Efficient Attention using Asymmetric Clustering
| Giannis DarasNikita KitaevAugustus OdenaAlexandros G. Dimakis
2020-10-11
LOGAN: Local Group Bias Detection by Clustering
Jieyu ZhaoKai-Wei Chang
2020-10-06
Bag of Tricks for Adversarial Training
| Tianyu PangXiao YangYinpeng DongHang SuJun Zhu
2020-10-01
TinyGAN: Distilling BigGAN for Conditional Image Generation
Ting-Yun ChangChi-Jen Lu
2020-09-29
not-so-BigGAN: Generating High-Fidelity Images on a Small Compute Budget
Seungwook HanAkash SrivastavaCole HurwitzPrasanna SattigeriDavid D. Cox
2020-09-09
Neural Crossbreed: Neural Based Image Metamorphosis
Sanghun ParkKwanggyoon SeoJunyong Noh
2020-09-02
Minimum discrepancy principle strategy for choosing $k$ in $k$-NN regression
Yaroslav AveryanovAlain Celisse
2020-08-20
A Functional Model for Structure Learning and Parameter Estimation in Continuous Time Bayesian Network: An Application in Identifying Patterns of Multiple Chronic Conditions
Syed Hasib Akhter FaruquiAdel AlaeddiniJing WangCarlos A. Jaramillo
2020-07-31
Instance Selection for GANs
Terrance DeVriesMichal DrozdzalGraham W. Taylor
2020-07-30
Interpolating GANs to Scaffold Autotelic Creativity
Ziv EpsteinOcéane BoulaisSkylar GordonMatt Groh
2020-07-21
Early Stopping in Deep Networks: Double Descent and How to Eliminate it
Reinhard HeckelFatih Furkan Yilmaz
2020-07-20
Early stopping and polynomial smoothing in regression with reproducing kernels
Yaroslav AveryanovAlain Celisse
2020-07-14
Automated Synthetic-to-Real Generalization
Wuyang ChenZhiding YuZhangyang WangAnima Anandkumar
2020-07-14
Estimating Generalization under Distribution Shifts via Domain-Invariant Representations
Ching-Yao ChuangAntonio TorralbaStefanie Jegelka
2020-07-06
On Dropout, Overfitting, and Interaction Effects in Deep Neural Networks
Benjamin LengerichEric P. XingRich Caruana
2020-07-02
Particle Swarm Optimization for Energy Disaggregation in Industrial and Commercial Buildings
Karoline BruckeStefan ArensJan-Simon TelleSunke~SchlütersBenedikt HankeKarsten von MaydellCarsten Agert
2020-06-23
Differentiable Augmentation for Data-Efficient GAN Training
| Shengyu ZhaoZhijian LiuJi LinJun-Yan ZhuSong Han
2020-06-18
Training Generative Adversarial Networks with Limited Data
| Tero KarrasMiika AittalaJanne HellstenSamuli LaineJaakko LehtinenTimo Aila
2020-06-11
Revisiting the Train Loss: an Efficient Performance Estimator for Neural Architecture Search
Binxin RuClare LyleLisa SchutMark van der WilkYarin Gal
2020-06-08
Learning disconnected manifolds: a no GANs land
Ugo TanielianThibaut IssenhuthElvis DohmatobJeremie Mary
2020-06-08
Big GANs Are Watching You: Towards Unsupervised Object Segmentation with Off-the-Shelf Generative Models
| Andrey VoynovStanislav MorozovArtem Babenko
2020-06-08
A U-Net Based Discriminator for Generative Adversarial Networks
Edgar Schonfeld Bernt Schiele Anna Khoreva
2020-06-01
Consistent Second-Order Conic Integer Programming for Learning Bayesian Networks
Simge KucukyavuzAli ShojaieHasan ManzourLinchuan Wei
2020-05-29
Network Fusion for Content Creation with Conditional INNs
Robin RombachPatrick EsserBjörn Ommer
2020-05-27
Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation
| Reinhard HeckelMahdi Soltanolkotabi
2020-05-07
Drawing Early-Bird Tickets: Toward More Efficient Training of Deep Networks
| Haoran YouChaojian LiPengfei XuYonggan FuYue WangXiaohan ChenRichard G. BaraniukZhangyang WangYingyan Lin
2020-05-01
SciREX: A Challenge Dataset for Document-Level Information Extraction
| Sarthak JainMadeleine van ZuylenHannaneh HajishirziIz Beltagy
2020-05-01
Query-level Early Exit for Additive Learning-to-Rank Ensembles
Claudio LuccheseFranco Maria NardiniSalvatore OrlandoRaffaele PeregoSalvatore Trani
2020-04-30
Analyzing the discrepancy principle for kernelized spectral filter learning algorithms
Alain CelisseMartin Wahl
2020-04-17
GANSpace: Discovering Interpretable GAN Controls
| Erik HärkönenAaron HertzmannJaakko LehtinenSylvain Paris
2020-04-06
Evolving Normalization-Activation Layers
| Hanxiao LiuAndrew BrockKaren SimonyanQuoc V. Le
2020-04-06
Feature Quantization Improves GAN Training
| Yang ZhaoChunyuan LiPing YuJianfeng GaoChangyou Chen
2020-04-05
Fully-Corrective Gradient Boosting with Squared Hinge: Fast Learning Rates and Early Stopping
Jinshan ZengMin ZhangShao-Bo Lin
2020-04-01
MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation
| Chaoyang HeHaishan YeLi ShenTong Zhang
2020-03-27
Improving Adversarial Robustness Through Progressive Hardening
Chawin SitawarinSupriyo ChakrabortyDavid Wagner
2020-03-18
BigGAN-based Bayesian reconstruction of natural images from human brain activity
Kai QiaoJian ChenLinyuan WangChi ZhangLi TongBin Yan
2020-03-13
Transformation-based Adversarial Video Prediction on Large-Scale Data
Pauline LucAidan ClarkSander DielemanDiego de Las CasasYotam DoronAlbin CassirerKaren Simonyan
2020-03-09
Time-varying neural network for stock return prediction
Steven Y. K. WongJennifer ChanLamiae AziziRichard Y. D. Xu
2020-03-05
A U-Net Based Discriminator for Generative Adversarial Networks
| Edgar SchönfeldBernt SchieleAnna Khoreva
2020-02-28
Bounding the expected run-time of nonconvex optimization with early stopping
Thomas FlynnKwang Min YuAbid MalikNicolas D'ImperioShinjae Yoo
2020-02-20
Learning Not to Learn in the Presence of Noisy Labels
Liu ZiyinBlair ChenRu WangPaul Pu LiangRuslan SalakhutdinovLouis-Philippe MorencyMasahito Ueda
2020-02-16
The Differentially Private Lottery Ticket Mechanism
Lovedeep GondaraKe WangRicardo Silva Carvalho
2020-02-16
Improved Consistency Regularization for GANs
Zhengli ZhaoSameer SinghHonglak LeeZizhao ZhangAugustus OdenaHan Zhang
2020-02-11
Reconstructing Natural Scenes from fMRI Patterns using BigBiGAN
Milad MozafariLeila ReddyRufin VanRullen
2020-01-31
Stochastic Optimization of Plain Convolutional Neural Networks with Simple methods
Yahia Assiri
2020-01-24
Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures
Mohamed El Amine SeddikCosme LouartMohamed TamaazoustiRomain Couillet
2020-01-21
Adaptive Stopping Rule for Kernel-based Gradient Descent Algorithms
Xiangyu ChangShao-Bo Lin
2020-01-09
A GOODNESS OF FIT MEASURE FOR GENERATIVE NETWORKS
Anonymous
2020-01-01
Drawing Early-Bird Tickets: Toward More Efficient Training of Deep Networks
| Anonymous
2020-01-01
Under what circumstances do local codes emerge in feed-forward neural networks
Anonymous
2020-01-01
GENERALIZATION GUARANTEES FOR NEURAL NETS VIA HARNESSING THE LOW-RANKNESS OF JACOBIAN
Anonymous
2020-01-01
A Simple Approach to the Noisy Label Problem Through the Gambler's Loss
Anonymous
2020-01-01
On the expected running time of nonconvex optimization with early stopping
Anonymous
2020-01-01
Distillation $\approx$ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized NN
Anonymous
2020-01-01
Winning Privately: The Differentially Private Lottery Ticket Mechanism
Anonymous
2020-01-01
Leveraging inductive bias of neural networks for learning without explicit human annotations
Anonymous
2020-01-01
CNN-generated images are surprisingly easy to spot... for now
| Sheng-Yu WangOliver WangRichard ZhangAndrew OwensAlexei A. Efros
2019-12-23
The Spectral Bias of the Deep Image Prior
| Prithvijit ChakrabartySubhransu Maji
2019-12-18
Detecting GAN generated errors
Xiru ZhuFengdi CheTianzi YangTzuyang YuDavid MegerGregory Dudek
2019-12-02
LOGAN: Latent Optimisation for Generative Adversarial Networks
| Yan WuJeff DonahueDavid BalduzziKaren SimonyanTimothy Lillicrap
2019-12-02
Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
| Ceyuan YangYujun ShenBolei Zhou
2019-11-21
Improving singing voice separation with the Wave-U-Net using Minimum Hyperspherical Energy
Joaquin Perez-LapilloOleksandr GalkinTillman Weyde
2019-10-22
Image recognition from raw labels collected without annotators
Fatih Furkan YilmazReinhard Heckel
2019-10-20
Improving sample diversity of a pre-trained, class-conditional GAN by changing its class embeddings
| Qi LiLong MaiMichael A. AlcornAnh Nguyen
2019-10-10
Distillation $\approx$ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Network
| Bin DongJikai HouYiping LuZhihua Zhang
2019-10-02
Drawing early-bird tickets: Towards more efficient training of deep networks
| Haoran YouChaojian LiPengfei XuYonggan FuYue WangXiaohan ChenRichard G. BaraniukZhangyang WangYingyan Lin
2019-09-26
k-Relevance Vectors for Pattern Classification
Peyman Hosseinzadeh KassaniSara Hosseinzadeh Kassani
2019-09-18
Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising
Haokui ZhangYing LiHao ChenChunhua Shen
2019-09-18
DARTS+: Improved Differentiable Architecture Search with Early Stopping
Hanwen LiangShifeng ZhangJiacheng SunXingqiu HeWeiran HuangKechen ZhuangZhenguo Li
2019-09-13
Towards Realistic Practices In Low-Resource Natural Language Processing: The Development Set
Katharina KannKyunghyun ChoSamuel R. Bowman
2019-09-04
Adversarial Video Generation on Complex Datasets
Aidan ClarkJeff DonahueKaren Simonyan
2019-07-15
Large Scale Adversarial Representation Learning
| Jeff DonahueKaren Simonyan
2019-07-04
Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian
Samet OymakZalan FabianMingchen LiMahdi Soltanolkotabi
2019-06-12
Off-Policy Evaluation via Off-Policy Classification
Alex IrpanKanishka RaoKonstantinos BousmalisChris HarrisJulian IbarzSergey Levine
2019-06-04
The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorial
Benyamin GhojoghMark Crowley
2019-05-28
Style transfer-based image synthesis as an efficient regularization technique in deep learning
Agnieszka MikołajczykMichał Grochowski
2019-05-27
Optimizing Interim Analysis Timing for Bayesian Adaptive Commensurate Designs
Xiao WuYi XuBradley P. Carlin
2019-05-17
Moving Target Defense for Deep Visual Sensing against Adversarial Examples
Qun SongZhenyu YanRui Tan
2019-05-11
Improved Precision and Recall Metric for Assessing Generative Models
| Tuomas KynkäänniemiTero KarrasSamuli LaineJaakko LehtinenTimo Aila
2019-04-15
Bayesian Neural Networks at Finite Temperature
| Robert J. N. BaldockNicola Marzari
2019-04-08
Sound source ranging using a feed-forward neural network with fitting-based early stopping
Jing ChiXiaolei LiHaozhong WangDazhi GaoPeter Gerstoft
2019-04-01
Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks
| Mingchen LiMahdi SoltanolkotabiSamet Oymak
2019-03-27
Implicit Regularization via Hadamard Product Over-Parametrization in High-Dimensional Linear Regression
Peng ZhaoYun YangQiao-Chu He
2019-03-22
High-Fidelity Image Generation With Fewer Labels
| Mario LucicMichael TschannenMarvin RitterXiaohua ZhaiOlivier BachemSylvain Gelly
2019-03-06
Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
| Bastian RieckMatteo TogninalliChristian BockMichael MoorMax HornThomas GumbschKarsten Borgwardt
2018-12-23
DeepCalib: a deep learning approach for automatic intrinsic calibration of wide field-of-view cameras
| Oleksandr BogdanViktor EcksteinFrancois RameauJean-Charles Bazin
2018-12-15
Pitfalls of Graph Neural Network Evaluation
| Oleksandr ShchurMaximilian MummeAleksandar BojchevskiStephan Günnemann
2018-11-14
Fast Hyperparameter Optimization of Deep Neural Networks via Ensembling Multiple Surrogates
Yang LiJiawei JiangYingxia ShaoBin Cui
2018-11-06
Metropolis-Hastings view on variational inference and adversarial training
Kirill NeklyudovEvgenii EgorovPavel ShvechikovDmitry Vetrov
2018-10-16
A Unified Dynamic Approach to Sparse Model Selection
Chendi HuangYuan Yao
2018-10-08
Large Scale GAN Training for High Fidelity Natural Image Synthesis
| Andrew BrockJeff DonahueKaren Simonyan
2018-09-28
An analytic theory of generalization dynamics and transfer learning in deep linear networks
Andrew K. LampinenSurya Ganguli
2018-09-27
A Collaborative Approach to Angel and Venture Capital Investment Recommendations
Xinyi LiuArtit Wangperawong
2018-07-26
Generalization Bounds for Unsupervised Cross-Domain Mapping with WGANs
Tomer GalantiSagie BenaimLior Wolf
2018-07-23
Minnorm training: an algorithm for training over-parameterized deep neural networks
Yamini BansalMadhu AdvaniDavid D CoxAndrew M Saxe
2018-06-03
The Dynamics of Learning: A Random Matrix Approach
Zhenyu LiaoRomain Couillet
2018-05-30
Early Stopping for Nonparametric Testing
Meimei LiuGuang Cheng
2018-05-25
The Importance of Norm Regularization in Linear Graph Embedding: Theoretical Analysis and Empirical Demonstration
Yihan GaoChao ZhangJian PengAditya Parameswaran
2018-02-10
TESLA: Task-wise Early Stopping and Loss Aggregation for Dynamic Neural Network Inference
Chun-Min ChangChia-Ching LinHung-Yi Ou YangChin-Laung LeiKuan-Ta Chen
2018-01-01
Theory of Deep Learning III: explaining the non-overfitting puzzle
Tomaso PoggioKenji KawaguchiQianli LiaoBrando MirandaLorenzo RosascoXavier BoixJack HidaryHrushikesh Mhaskar
2017-12-30
Building Robust Deep Neural Networks for Road Sign Detection
Arkar Min AungYousef FadilaRadian GondokaryonoLuis Gonzalez
2017-12-26
Stochastic Particle Gradient Descent for Infinite Ensembles
Atsushi NitandaTaiji Suzuki
2017-12-14
High-dimensional dynamics of generalization error in neural networks
Madhu S. AdvaniAndrew M. Saxe
2017-10-10
Massively-Parallel Feature Selection for Big Data
Ioannis TsamardinosGiorgos BorboudakisPavlos KatsogridakisPolyvios PratikakisVassilis Christophides
2017-08-23
Optimization by gradient boosting
Gérard BiauBenoît Cadre
2017-07-17
Object Detection Using Deep CNNs Trained on Synthetic Images
Param S. RajpuraHristo BojinovRavi S. Hegde
2017-06-21
Toward Optimal Run Racing: Application to Deep Learning Calibration
Olivier BousquetSylvain GellyKarol KurachMarc SchoenauerMichele SebagOlivier TeytaudDamien Vincent
2017-06-10
Accelerating Neural Architecture Search using Performance Prediction
| Bowen BakerOtkrist GuptaRamesh RaskarNikhil Naik
2017-05-30
Regularizing Model Complexity and Label Structure for Multi-Label Text Classification
Bingyu WangCheng LiVirgil PavluJaved Aslam
2017-05-01
Google Vizier: A Service for Black-Box Optimization
| Daniel GolovinBenjamin SolnikSubhodeep MoitraGreg KochanskiJohn KarroD. Sculley
2017-01-01
Boosted Sparse Non-linear Distance Metric Learning
Yuting MaTian Zheng
2015-12-10
NYTRO: When Subsampling Meets Early Stopping
Tomas AnglesRaffaello CamorianoAlessandro RudiLorenzo Rosasco
2015-10-19
How to Generate a Good Word Embedding?
| Siwei LaiKang LiuLiheng XuJun Zhao
2015-07-20
Totally Corrective Boosting with Cardinality Penalization
Vasil S. DenchevNan DingShin MatsushimaS. V. N. VishwanathanHartmut Neven
2015-04-07
Early Stopping is Nonparametric Variational Inference
| Dougal MaclaurinDavid DuvenaudRyan P. Adams
2015-04-06
Compute Less to Get More: Using ORC to Improve Sparse Filtering
Johannes LedererSergio Guadarrama
2014-09-16
Nonconvex Statistical Optimization: Minimax-Optimal Sparse PCA in Polynomial Time
Zhaoran WangHuanran LuHan Liu
2014-08-22
Approximated Infomax Early Stopping: Revisiting Gaussian RBMs on Natural Images
Taichi KiwakiTakaki MakinoKazuyuki Aihara
2013-12-19
Early stopping and non-parametric regression: An optimal data-dependent stopping rule
Garvesh RaskuttiMartin J. WainwrightBin Yu
2013-06-15

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories