Search Results for author:
Found 10 papers, 6 papers with code
Understanding Hyperbolic Metric Learning through Hard Negative Sampling
In recent years, there has been a growing trend of incorporating hyperbolic geometry methods into computer vision.
AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix
Additionally, we introduce an auto-switching function that enables the preconditioning matrix to switch dynamically between Stochastic Gradient Descent (SGD) and the adaptive optimizer.
Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term
Deep Neural Networks (DNNs) generalization is known to be closely related to the flatness of minima, leading to the development of Sharpness-Aware Minimization (SAM) for seeking flatter minima and better generalization.
Hyperbolic Contrastive Learning
Learning good image representations that are beneficial to downstream tasks is a challenging task in computer vision.
Hyperbolic Chamfer Distance for Point Cloud Completion
Chamfer distance (CD) is a standard metric to measure the shape dissimilarity between point clouds in point cloud completion, as well as a loss function for (deep) learning.
SBO-RNN: Reformulating Recurrent Neural Networks via Stochastic Bilevel Optimization
In this paper we consider the training stability of recurrent neural networks (RNNs) and propose a family of RNNs, namely SBO-RNN, that can be formulated using stochastic bilevel optimization (SBO).
Learning Lightweight Neural Networks via Channel-Split Recurrent Convolution
Lightweight neural networks refer to deep networks with small numbers of parameters, which are allowed to be implemented in resource-limited hardware such as embedded systems.
Adaptive Optimizers with Sparse Group Lasso for Neural Networks in CTR Prediction
We develop a novel framework that adds the regularizers of the sparse group lasso to a family of adaptive optimizers in deep learning, such as Momentum, Adagrad, Adam, AMSGrad, AdaHessian, and create a new class of optimizers, which are named Group Momentum, Group Adagrad, Group Adam, Group AMSGrad and Group AdaHessian, etc., accordingly.
Adaptive Optimizers with Sparse Group Lasso
We develop a novel framework that adds the regularizers to a family of adaptive optimizers in deep learning, such as MOMENTUM, ADAGRAD, ADAM, AMSGRAD, ADAHESSIAN, and create a new class of optimizers, which are named GROUP MOMENTUM, GROUP ADAGRAD, GROUP ADAM, GROUP AMSGRAD and GROUP ADAHESSIAN, etc., accordingly.
RNN Training along Locally Optimal Trajectories via Frank-Wolfe Algorithm
We propose to utilize the Frank-Wolfe (FW) algorithm in this context.
