Search Results for author:
Found 20 papers, 5 papers with code
A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
Pruning is a critical strategy for compressing trained large language models (LLMs), aiming at substantial memory conservation and computational acceleration without compromising performance.
Adapprox: Adaptive Approximation in Adam Optimization via Randomized Low-Rank Matrices
As deep learning models exponentially increase in size, optimizers such as Adam encounter significant memory consumption challenges due to the storage of first and second moment data.
Coverage Axis++: Efficient Inner Point Selection for 3D Shape Skeletonization
We introduce Coverage Axis++, a novel and efficient approach to 3D shape skeletonization.
The Hard-Constraint PINNs for Interface Optimal Control Problems
We show that the physics-informed neural networks (PINNs), in combination with some recently developed discontinuity capturing neural networks, can be applied to solve optimal control problems subject to partial differential equations (PDEs) with interfaces and some control constraints.
Accelerated primal-dual methods with enlarged step sizes and operator learning for nonsmooth optimal control problems
The accelerated primal-dual method with operator learning is mesh-free, numerically efficient, and scalable to different types of PDEs.
The ADMM-PINNs Algorithmic Framework for Nonsmooth PDE-Constrained Optimization: A Deep Learning Approach
We study the combination of the alternating direction method of multipliers (ADMM) with physics-informed neural networks (PINNs) for a general class of nonsmooth partial differential equation (PDE)-constrained optimization problems, where additional regularization can be employed for constraints on the control or design variables.
Towards Tailored Models on Private AIoT Devices: Federated Direct Neural Architecture Search
To further adapt to both various data distributions and different types of devices with heterogeneous embedded hardware platforms, inspired by meta-learning, a Cluster Federated Direct Neural Architecture Search (CFDNAS) framework is proposed to achieve device-aware NAS, in the sense that each device can learn a tailored deep learning model for its particular data distribution and hardware constraint.
Enhanced total variation minimization for stable image reconstruction
The total variation (TV) regularization has phenomenally boosted various variational models for image processing tasks.
A Value-Function-based Interior-point Method for Non-convex Bi-level Optimization
Bi-level optimization model is able to capture a wide range of complex learning tasks with practical interest.
A General Descent Aggregation Framework for Gradient-based Bi-level Optimization
In this work, we formulate BLOs from an optimistic bi-level viewpoint and establish a new gradient-based algorithmic framework, named Bi-level Descent Aggregation (BDA), to partially address the above issues.
Bilinear Optimal Control of an Advection-reaction-diffusion System
However, due to the additional divergence-free constraint on the control variable and the nonlinear relation between the state and control variables, it is challenging to compute the gradient and the optimal stepsize at each CG iteration, and thus nontrivial to implement the CG method.
Optimization and Control 49M41, 35Q93, 49J20
FDNAS: Improving Data Privacy and Model Diversity in AutoML
To further adapt for various data distributions of clients, inspired by meta-learning, a cluster Federated Direct Neural Architecture Search (CFDNAS) framework is proposed to achieve client-aware NAS, in the sense that each client can learn a tailored deep learning model for its particular data distribution.
The flare Package for High Dimensional Linear Regression and Precision Matrix Estimation in R
This paper describes an R package named flare, which implements a family of new high dimensional regression methods (LAD Lasso, SQRT Lasso, $\ell_q$ Lasso, and Dantzig selector) and their extensions to sparse precision matrix estimation (TIGER and CLIME).
A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton
In recent years, a variety of gradient-based first-order methods have been developed to solve bi-level optimization problems for learning applications.
Task-Oriented Convex Bilevel Optimization with Latent Feasibility
This paper firstly proposes a convex bilevel optimization paradigm to formulate and optimize popular learning and vision problems in real-world scenarios.
Stability and Optimization Error of Stochastic Gradient Descent for Pairwise Learning
From this fundamental trade-off, we obtain lower bounds for the optimization error of SGD algorithms and the excess expected risk over a class of pairwise losses.
Adaptive Consensus ADMM for Distributed Optimization
The alternating direction method of multipliers (ADMM) is commonly used for distributed model fitting problems, but its performance and reliability depend strongly on user-defined penalty parameters.
Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation
Relaxed ADMM is a generalization of ADMM that often achieves better performance, but its efficiency depends strongly on algorithm parameters that must be chosen by an expert user.
Adaptive Primal-Dual Splitting Methods for Statistical Learning and Image Processing
The alternating direction method of multipliers (ADMM) is an important tool for solving complex optimization problems, but it involves minimization sub-steps that are often difficult to solve efficiently.
Adaptive Primal-Dual Hybrid Gradient Methods for Saddle-Point Problems
The Primal-Dual hybrid gradient (PDHG) method is a powerful optimization scheme that breaks complex problems into simple sub-steps.
Numerical Analysis 65K15 G.1.6
