Search Results for author: Defeng Sun

Found 17 papers, 2 papers with code

Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise

no code implementations12 Jun 2025 Chuan He, Zhaosong Lu, Defeng Sun, Zhanwang Deng

In this paper, we propose practical normalized stochastic first-order methods with Polyak momentum, multi-extrapolated momentum, and recursive momentum for solving unconstrained optimization problems.

Approximation Bounds for Transformer Networks with Application to Regression

no code implementations16 Apr 2025 Yuling Jiao, Yanming Lai, Defeng Sun, Yang Wang, Bokai Yan

First, we establish novel upper bounds for standard Transformer networks approximating sequence-to-sequence mappings whose component functions are H\"older continuous with smoothness index $\gamma \in (0, 1]$.

regression

Distribution Matching for Self-Supervised Transfer Learning

1 code implementation20 Feb 2025 Yuling Jiao, Wensen Ma, Defeng Sun, Hansheng Wang, Yang Wang

In this paper, we propose a novel self-supervised transfer learning method called Distribution Matching (DM), which drives the representation distribution toward a predefined reference distribution while preserving augmentation invariance.

Classification Self-Supervised Learning +1

LAMBDA: A Large Model Based Data Agent

1 code implementation24 Jul 2024 Maojun Sun, Ruijian Han, Binyan Jiang, HouDuo Qi, Defeng Sun, Yancheng Yuan, Jian Huang

The strong performance of LAMBDA in solving data analysis problems is demonstrated using real-world data examples.

model

Machine Learning Insides OptVerse AI Solver: Design Principles and Applications

no code implementations11 Jan 2024 Xijun Li, Fangzhou Zhu, Hui-Ling Zhen, Weilin Luo, Meng Lu, Yimin Huang, Zhenan Fan, Zirui Zhou, Yufei Kuang, Zhihai Wang, Zijie Geng, Yang Li, Haoyang Liu, Zhiwu An, Muming Yang, Jianshu Li, Jie Wang, Junchi Yan, Defeng Sun, Tao Zhong, Yong Zhang, Jia Zeng, Mingxuan Yuan, Jianye Hao, Jun Yao, Kun Mao

To this end, we present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI Solver, which aims to mitigate the scarcity of real-world mathematical programming instances, and to surpass the capabilities of traditional optimization techniques.

Decision Making Management

Randomly Projected Convex Clustering Model: Motivation, Realization, and Cluster Recovery Guarantees

no code implementations29 Mar 2023 Ziwen Wang, Yancheng Yuan, Jiaming Ma, Tieyong Zeng, Defeng Sun

In this paper, we propose a randomly projected convex clustering model for clustering a collection of $n$ high dimensional data points in $\mathbb{R}^d$ with $K$ hidden clusters.

Clustering

Learning Graph Laplacian with MCP

no code implementations22 Oct 2020 Yangjing Zhang, Kim-Chuan Toh, Defeng Sun

We consider the problem of learning a graph under the Laplacian constraint with a non-convex penalty: minimax concave penalty (MCP).

Estimation of sparse Gaussian graphical models with hidden clustering structure

no code implementations17 Apr 2020 Meixia Lin, Defeng Sun, Kim-Chuan Toh, Chengjing Wang

The sparsity and clustering structure of the concentration matrix is enforced to reduce model complexity and describe inherent regularities.

Clustering

Efficient algorithms for multivariate shape-constrained convex regression problems

no code implementations26 Feb 2020 Meixia Lin, Defeng Sun, Kim-Chuan Toh

We prove that the least squares estimator is computable via solving a constrained convex quadratic programming (QP) problem with $(n+1)d$ variables and at least $n(n-1)$ linear inequality constraints, where $n$ is the number of data points.

regression

A sparse semismooth Newton based proximal majorization-minimization algorithm for nonconvex square-root-loss regression problems

no code implementations27 Mar 2019 Peipei Tang, Chengjing Wang, Defeng Sun, Kim-Chuan Toh

In this paper, we consider high-dimensional nonconvex square-root-loss regression problems and introduce a proximal majorization-minimization (PMM) algorithm for these problems.

regression

A dual Newton based preconditioned proximal point algorithm for exclusive lasso models

no code implementations1 Feb 2019 Meixia Lin, Defeng Sun, Kim-Chuan Toh, Yancheng Yuan

In addition, we derive the corresponding HS-Jacobian to the proximal mapping and analyze its structure --- which plays an essential role in the efficient computation of the PPA subproblem via applying a semismooth Newton method on its dual.

Convex Clustering: Model, Theoretical Guarantee and Efficient Algorithm

no code implementations4 Oct 2018 Defeng Sun, Kim-Chuan Toh, Yancheng Yuan

The perfect recovery properties of the convex clustering model with uniformly weighted all pairwise-differences regularization have been proved by Zhu et al. (2014) and Panahi et al. (2017).

Clustering

A Fast Globally Linearly Convergent Algorithm for the Computation of Wasserstein Barycenters

no code implementations12 Sep 2018 Lei Yang, Jia Li, Defeng Sun, Kim-Chuan Toh

When the support points of the barycenter are pre-specified, this problem can be modeled as a linear programming (LP) problem whose size can be extremely large.

Efficient sparse semismooth Newton methods for the clustered lasso problem

no code implementations22 Aug 2018 Meixia Lin, Yong-Jin Liu, Defeng Sun, Kim-Chuan Toh

Based on the new formulation, we derive an efficient procedure for its computation.

An Efficient Semismooth Newton Based Algorithm for Convex Clustering

no code implementations ICML 2018 Yancheng Yuan, Defeng Sun, Kim-Chuan Toh

Clustering may be the most fundamental problem in unsupervised learning which is still active in machine learning research because its importance in many applications.

Clustering

A Rank-Corrected Procedure for Matrix Completion with Fixed Basis Coefficients

no code implementations13 Oct 2012 Weimin Miao, Shaohua Pan, Defeng Sun

To seek a solution of high recovery quality beyond the reach of the nuclear norm, in this paper, we propose a rank-corrected procedure using a nuclear semi-norm to generate a new estimator.

Low-Rank Matrix Completion Quantum State Tomography

Cannot find the paper you are looking for? You can Submit a new open access paper.