Search Results for author:
Found 5 papers, 1 papers with code
Adaptively Weighted Data Augmentation Consistency Regularization for Robust Optimization under Concept Shift
At the saddle point of the underlying objective, the weights assign label-dense samples to the supervised loss and label-sparse samples to the unsupervised consistency regularization.
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning
Inspired by the success of the iterative magnitude pruning technique in finding lottery tickets of neural networks, we propose a new method -- Sparser Random Feature Models via IMP (ShRIMP) -- to efficiently fit high-dimensional data with inherent low-dimensional structure in the form of sparse variable dependencies.
AdaLoss: A computationally-efficient and provably convergent adaptive gradient method
We propose a computationally-friendly adaptive learning rate schedule, "AdaLoss", which directly uses the information of the loss function to adjust the stepsize in gradient descent methods.
Overparameterization and generalization error: weighted trigonometric interpolation
Motivated by surprisingly good generalization properties of learned deep neural networks in overparameterized scenarios and by the related double descent phenomenon, this paper analyzes the relation between smoothness and low generalization error in an overparameterized linear learning problem.
Linear Convergence of Adaptive Stochastic Gradient Descent
We prove that the norm version of the adaptive stochastic gradient method (AdaGrad-Norm) achieves a linear convergence rate for a subset of either strongly convex functions or non-convex functions that satisfy the Polyak Lojasiewicz (PL) inequality.
