A Simple Sparse Denoising Layer for Robust Deep Learning
Deep models have achieved great success in many applications. However, vanilla deep models are not well-designed against the input perturbation. In this work, we take an initial step to designing a simple robust layer as a lightweight plug-in for vanilla deep models. To achieve this goal, we first propose a fast sparse coding and dictionary learning algorithm for sparse coding problem with an exact $k$-sparse constraint or $l_0$ norm regularization. Our method comes with a closed-form solution for the sparse coding phase by taking advantage of a novel structured dictionary. With this handy solution, we propose a simple sparse denoising layer (SDL) as a lightweight robust plug-in. Extensive experiments on both classification and reinforcement learning tasks manifest the effectiveness of our methods.
PDF Abstract
