Exponential Decay Explained | Papers With Code

Exponential Decay

Exponential Decay is a learning rate schedule where we decay the learning rate with more iterations using an exponential function:

$$ \text{lr} = \text{lr}_{0}\exp\left(-kt\right) $$

Image Credit: Suki Lau

Latest Papers

PAPER	DATE
Noise-Induced Barren Plateaus in Variational Quantum Algorithms Samson Wang • Enrico Fontana • M. Cerezo • Kunal Sharma • Akira Sone • Lukasz Cincio • Patrick J. Coles	2020-07-28
Context-Aware Attentive Knowledge Tracing \| Aritra Ghosh • Neil Heffernan • Andrew S. Lan	2020-07-24
Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward Guannan Qu • Yiheng Lin • Adam Wierman • Na Li	2020-06-11
Revisiting Regex Generation for Modeling Industrial Applications by Incorporating Byte Pair Encoder Desheng Wang • Jiawei Liu • Xiang Qi • Baolin Sun • Peng Zhang	2020-05-06
Supervised Contrastive Learning \| Prannay Khosla • Piotr Teterwak • Chen Wang • Aaron Sarna • Yonglong Tian • Phillip Isola • Aaron Maschinot • Ce Liu • Dilip Krishnan # 55 IMAGE CLASSIFICATION ON IMAGENET	2020-04-23
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases DongHyun Choi • Myeong Cheol Shin • EungGyun Kim • Dong Ryeol Shin # 1 TEXT-TO-SQL ON SPIDER	2020-04-07
Scalable Gradients for Stochastic Differential Equations \| Xuechen Li • Ting-Kam Leonard Wong • Ricky T. Q. Chen • David Duvenaud # 1 VIDEO PREDICTION ON CMU MOCAP-2	2020-01-05
Momentum Contrast for Unsupervised Visual Representation Learning \| Kaiming He • Haoqi Fan • Yuxin Wu • Saining Xie • Ross Girshick # 22 SELF-SUPERVISED IMAGE CLASSIFICATION ON IMAGENET	2019-11-13
Learning pairwise Markov network structures using correlation neighborhoods Juri Kuronen • Jukka Corander • Johan Pensar	2019-10-30
Quasiclassical Modeling of Cavity Quantum Electrodynamics Tao E. Li • Hsing-Ta Chen • Abraham Nitzan • Joseph E. Subotnik	2019-10-05
Global exponential stability of primal-dual gradient flow dynamics based on the proximal augmented Lagrangian: A Lyapunov-based approach Dongsheng Ding • Mihailo R. Jovanović	2019-10-02
Augmented Memory for Correlation Filters in Real-Time UAV Tracking \| Yiming Li • Changhong Fu • Fangqiang Ding • Ziyuan Huang • Jia Pan	2019-09-24
Asymptotically Optimal One- and Two-Sample Testing with Kernels Shengyu Zhu • Biao Chen • Zhitang Chen • Pengfei Yang	2019-08-27
Learning the Model Update for Siamese Trackers \| Lichao Zhang • Abel Gonzalez-Garcia • Joost van de Weijer • Martin Danelljan • Fahad Shahbaz Khan	2019-08-02
Supervised Symbolic Music Style Translation Using Synthetic Data \| Ondřej Cífka • Umut Şimşekli • Gaël Richard	2019-07-04
Measurement-driven entanglement transition in hybrid quantum circuits Yaodong Li • Xiao Chen • Matthew P. A. Fisher	2019-01-23
Concentration bounds for CVaR estimation: The cases of light-tailed and heavy-tailed distributions Prashanth L. A. • Krishna Jagannathan • Ravi Kumar Kolla	2019-01-04
K-nearest Neighbor Search by Random Projection Forests Donghui Yan • Yingjie Wang • Jin Wang • Honggang Wang • Zhenpeng Li	2018-12-31
Shortcut Matrix Product States and its applications Zhuan Li • Pan Zhang	2018-12-13
Predicting Recall Probability to Adaptively Prioritize Study Shane Mooney • Karen Sun • Eric Bomgardner	2018-02-28
Exponentially Consistent Kernel Two-Sample Tests Shengyu Zhu • Biao Chen • Zhitang Chen	2018-02-23
Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit Shengyu Zhu • Biao Chen • Pengfei Yang • Zhitang Chen	2018-02-21
Approximation beats concentration? An approximation view on inference with smooth radial kernels Mikhail Belkin	2018-01-10
Scale-invariant temporal history (SITH): optimal slicing of the past in an uncertain world Tyler A. Spears • Brandon G. Jacques • Marc W. Howard • Per B. Sederberg	2017-12-19
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions \| Jonathan Shen • Ruoming Pang • Ron J. Weiss • Mike Schuster • Navdeep Jaitly • Zongheng Yang • Zhifeng Chen • Yu Zhang • Yuxuan Wang • RJ Skerry-Ryan • Rif A. Saurous • Yannis Agiomyrgiannakis • Yonghui Wu # 1 SPEECH SYNTHESIS ON NORTH AMERICAN ENGLISH	2017-12-16
Learning Transferable Architectures for Scalable Image Recognition \| Barret Zoph • Vijay Vasudevan • Jonathon Shlens • Quoc V. Le # 35 IMAGE CLASSIFICATION ON IMAGENET	2017-07-21
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications \| Andrew G. Howard • Menglong Zhu • Bo Chen • Dmitry Kalenichenko • Weijun Wang • Tobias Weyand • Marco Andreetto • Hartwig Adam # 89 OBJECT DETECTION ON COCO TEST-DEV	2017-04-17
A generalized Bayesian framework for the analysis of subscription based businesses Rahul Madhavan • Ankit Baraskar	2017-04-12
Double spend races Cyril Grunspan • Ricardo Pérez-Marco	2017-02-06
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning \| Christian Szegedy • Sergey Ioffe • Vincent Vanhoucke • Alex Alemi # 60 IMAGE CLASSIFICATION ON IMAGENET	2016-02-23
Uniform {\varepsilon}-Stability of Distributed Nonlinear Filtering over DNAs: Gaussian-Finite HMMs Dionysios S. Kalogerias • Athina P. Petropulu	2016-02-16
Rethinking the Inception Architecture for Computer Vision \| Christian Szegedy • Vincent Vanhoucke • Sergey Ioffe • Jonathon Shlens • Zbigniew Wojna # 79 IMAGE CLASSIFICATION ON IMAGENET	2015-12-02
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift \| Sergey Ioffe • Christian Szegedy # 127 IMAGE CLASSIFICATION ON IMAGENET	2015-02-11
Bayesian Inference for NMR Spectroscopy with Applications to Chemical Quantification Andrew Gordon Wilson • Yuting Wu • Daniel J. Holland • Sebastian Nowozin • Mick D. Mantle • Lynn F. Gladden • Andrew Blake	2014-02-14

Tasks

TASK	PAPERS	SHARE
Image Classification	6	26.09%
Knowledge Tracing	1	4.35%
Multi-agent Reinforcement Learning	1	4.35%
Self-Supervised Learning	1	4.35%
Slot Filling	1	4.35%
Text-To-Sql	1	4.35%
Motion Capture	1	4.35%
Video Prediction	1	4.35%
Self-Supervised Image Classification	1	4.35%

Usage Over Time

This feature is experimental; we are continuously improving our matching algorithm.

Components

COMPONENT	TYPE	Add Remove
🤖 No Components Found	You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories

Add Remove

Learning Rate Schedules