Search Results for author:
Found 12 papers, 2 papers with code
Fast Deterministic CUR Matrix Decomposition with Accuracy Assurance
This is because coordinate descent iteratively updates all the parameters in the objective until convergence.
Fast Regularized Discrete Optimal Transport with Group-Sparse Regularizers
Regularized discrete optimal transport (OT) is a powerful tool to measure the distance between two discrete distributions that have been constructed from data samples on two different domains.
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks
We prove that the gradient vanishing of the gate function can be mitigated by accelerating the convergence of the saturating function, i. e., making the output of the function converge to 0 or 1 faster.
One-vs-the-Rest Loss to Focus on Important Samples in Adversarial Training
This paper proposes a new loss function for adversarial training.
Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks
Few-shot learning for neural networks (NNs) is an important problem that aims to train NNs with a few data.
Pruning Randomly Initialized Neural Networks with Iterative Randomization
Pruning the weights of randomly initialized neural networks plays an important role in the context of lottery ticket hypothesis.
Smoothness Analysis of Adversarial Training
We reveal that the constraint of adversarial attacks is one cause of the non-smoothness and that the smoothness depends on the types of the constraints.
Constraining Logits by Bounded Function for Adversarial Robustness
We theoretically and empirically reveal that small logits by addition of a common activation function, e. g., hyperbolic tangent, do not improve adversarial robustness since input vectors of the function (pre-logit vectors) can have large norms.
Fast Sparse Group Lasso
Block Coordinate Descent is a standard approach to obtain the parameters of Sparse Group Lasso, and iteratively updates the parameters for each parameter group.
Absum: Simple Regularization Method for Reducing Structural Sensitivity of Convolutional Neural Networks
Furthermore, we reveal that robust CNNs with Absum are more robust against transferred attacks due to decreasing the common sensitivity and against high-frequency noise than standard regularization methods.
Network Implosion: Effective Model Compression for ResNets via Static Layer Pruning and Retraining
Our key idea is to introduce a priority term that identifies the importance of a layer; we can select unimportant layers according to the priority and erase them after the training.
Adaptive Learning Rate via Covariance Matrix Based Preconditioning for Deep Neural Networks
Adaptive learning rate algorithms such as RMSProp are widely used for training deep neural networks.
