L1 Regularization

$L_{1}$ Regularization is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L_{1}$ Norm of the weights:

$$L_{new}\left(w\right) = L_{original}\left(w\right) + \lambda{||w||}_{1}$$

where $\lambda$ is a value determining the strength of the penalty. In contrast to weight decay, $L_{1}$ regularization promotes sparsity; i.e. some parameters have an optimal value of zero.

Image Source: Wikipedia

Latest Papers

PAPER DATE
Learning from a Complementary-label Source Domain: Theory and Algorithms
Yiyang ZhangFeng LiuZhen FangBo YuanGuangquan ZhangJie Lu
2020-08-04
Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation
| Yiyang ZhangFeng LiuZhen FangBo YuanGuangquan ZhangJie Lu
2020-07-29
Graph Neural Networks Including Sparse Interpretability
Chris LinGerald J. SunKrishna C. BulusuJonathan R. DryMarylens Hernandez
2020-06-30
Deep Double-Side Learning Ensemble Model for Few-Shot Parkinson Speech Recognition
Yongming LiLang ZhouLingyun QinYuwei ZengYuchuan LiuYan LeiPin WangFan Li
2020-06-20
CLARINET: A RISC-V Based Framework for Posit Arithmetic Empiricism
Riya JainNiraj SharmaFarhad MerchantSachin PatkarRainer Leupers
2020-05-30
Hybrid Embedded Deep Stacked Sparse Autoencoder with w_LPPD SVM Ensemble
Yongming LiYan LeiPin WangYuchuan Liu
2020-02-17
Lossless Compression of Deep Neural Networks
Thiago SerraAbhinav KumarSrikumar Ramalingam
2020-01-01
Parallel Neural Text-to-Speech
Kainan PengWei PingZhao SongKexin Zhao
2020-01-01
Relating lp regularization and reweighted l1 regularization
Hao WangHao ZengJiashan Wang
2019-12-02
Improving Transformer Models by Reordering their Sublayers
| Ofir PressNoah A. SmithOmer Levy
2019-11-10
Identifying Epigenetic Signature of Breast Cancer with Machine Learning
Maxim Vaysburd
2019-10-12
Mish: A Self Regularized Non-Monotonic Activation Function
| Diganta Misra
2019-08-23
SlimYOLOv3: Narrower, Faster and Better for Real-Time UAV Applications
| Pengyi ZhangYunxin ZhongXiaoqiong Li
2019-07-25
Multi-Speaker End-to-End Speech Synthesis
Jihyun ParkKexin ZhaoKainan PengWei Ping
2019-07-09
Augmenting Self-attention with Persistent Memory
| Sainbayar SukhbaatarEdouard GraveGuillaume LampleHerve JegouArmand Joulin
2019-07-02
Predicting epileptic seizures using nonnegative matrix factorization
| Olivera StojanovićGordon Pipa
2019-06-25
Non-Autoregressive Neural Text-to-Speech
| Kainan PengWei PingZhao SongKexin Zhao
2019-05-21
Adaptive Attention Span in Transformers
| Sainbayar SukhbaatarEdouard GravePiotr BojanowskiArmand Joulin
2019-05-19
Neural source-filter waveform models for statistical parametric speech synthesis
Xin WangShinji TakakiJunichi Yamagishi
2019-04-27
Automatic Target Recognition Using Discrimination Based on Optimal Transport
Ali SadeghianDeoksu LimJohan KarlssonJian Li
2019-04-06
PLIT: An alignment-free computational tool for identification of long non-coding RNAs in plant transcriptomic datasets
S. DeshpandeJ. ShuttleworthJ. YangS. TaramonliM. England
2019-02-12
FloWaveNet : A Generative Flow for Raw Audio
Sungwon KimSang-gil LeeJongyoon SongSungroh Yoon
2018-11-06
ET-Lasso: A New Efficient Tuning of Lasso-type Regularization for High-Dimensional Data
Songshan YangJiawei WenXiang ZhanDaniel Kifer
2018-10-10
Wasserstein Autoencoders for Collaborative Filtering
Jingbin ZhongXiaofeng Zhang
2018-09-15
ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
| Wei PingKainan PengJitong Chen
2018-07-19
De-Biased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers
Victor ChernozhukovWhitney NeweyRahul Singh
2018-02-23
Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning
| Wei PingKainan PengAndrew GibianskySercan O. ArikAjay KannanSharan NarangJonathan RaimanJohn Miller
2017-10-20
On parameters transformations for emulating sparse priors using variational-Laplace inference
Jean Daunizeau
2017-03-06
Convex Optimization For Non-Convex Problems via Column Generation
Julian YarkonyKamalika Chaudhuri
2016-02-14
Lass-0: sparse non-convex regression by local search
William HerlandsMaria De-ArteagaDaniel NeillArtur Dubrawski
2015-11-13
A concave regularization technique for sparse mixture models
Martin O. LarssonJohan Ugander
2011-12-01
Linear Complementarity for Regularized Policy Evaluation and Improvement
Jeffrey JohnsChristopher Painter-WakefieldRonald Parr
2010-12-01

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories