Search Results for author:
Found 38 papers, 5 papers with code
Gradient Descent Finds Over-Parameterized Neural Networks with Sharp Generalization for Nonparametric Regression: A Distribution-Free Analysis
We show that, if the neural network is trained by GD with early stopping, then the trained network renders a sharp rate of the nonparametric regression risk of $\cO(\eps_n^2)$, which is the same rate as that for the classical kernel regression trained by GD with early stopping, where $\eps_n$ is the critical population rate of the Neural Tangent Kernel (NTK) associated with the network and $n$ is the size of the training data.
Locally Regularized Sparse Graph by Fast Proximal Gradient Descent
In order to obtain a sparse graph aligned with the local geometric structure of data, we propose a novel Support Regularized Sparse Graph, abbreviated as SRSG, for data clustering.
Deep Geometric Moments Promote Shape Consistency in Text-to-3D Generation
To address the data scarcity associated with 3D assets, 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines.
IDNet: A Novel Dataset for Identity Document Analysis and Fraud Detection
Effective fraud detection and analysis of government-issued identity documents, such as passports, driver's licenses, and identity cards, are essential in thwarting identity theft and bolstering security on online platforms.
Efficient Visual Transformer by Learnable Token Merging
In this paper, we propose a novel and compact transformer block, Transformer with Learnable Token Merging (LTM), or LTM-Transformer.
Preconditioned Gradient Descent Finds Over-Parameterized Neural Networks with Sharp Generalization for Nonparametric Regression
We show that, if the neural network is trained with a novel Preconditioned Gradient Descent (PGD) with early stopping and the target function has spectral bias widely studied in the deep learning literature, the trained network renders a particularly sharp generalization bound with a minimax optimal rate of $\cO({1}/{n^{4\alpha/(4\alpha+1)}})$, which is sharper the current standard rate of $\cO({1}/{n^{2\alpha/(2\alpha+1)}})$ with $2\alpha = d/(d-1)$ when the data is distributed uniformly on the unit sphere in $\RR^d$ and $n$ is the size of the training data.
Learning Low-Rank Feature for Thorax Disease Classification
To address this challenge, we propose a novel Low-Rank Feature Learning (LRFL) method in this paper, which is universally applicable to the training of all neural networks.
Low-Rank Graph Contrastive Learning for Node Classification
To the best of our knowledge, our theoretical result is among the first to theoretically demonstrate the advantage of low-rank learning in graph contrastive learning supported by strong empirical performance.
A Learning-based Declarative Privacy-Preserving Framework for Federated Data Management
It is challenging to select the right privacy-preserving mechanism for federated query processing over multiple private data silos.
Sketching for Convex and Nonconvex Regularized Least Squares with Sharp Guarantees
We present general theoretical result for the approximation error between the optimization results of the original problem and the sketched problem for regularized least square problems which can be convex or nonconvex.
Sharp Generalization of Transductive Learning: A Transductive Local Rademacher Complexity Approach
Using a peeling strategy and a new surrogate variance operator, we derive the a novel excess risk bound in the transductive setting which is consistent with the classical LRC-based excess risk bound in the inductive setting.
RecMind: Large Language Model Powered Agent For Recommendation
While the recommendation system (RS) has advanced significantly through deep learning, current RS approaches usually train and fine-tune models on task-specific datasets, limiting their generalizability to new recommendation tasks and their ability to leverage external knowledge due to model scale and data size constraints.
Projective Proximal Gradient Descent for A Class of Nonconvex Nonsmooth Optimization Problems: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property
It is proved that PPGD achieves a fast convergence rate of $\cO(1/k^2)$ when the iteration number $k \ge k_0$ for a finite $k_0$ on a class of nonconvex and nonsmooth problems under mild assumptions, which is locally Nesterov's optimal convergence rate of first-order methods on smooth and convex objective function with Lipschitz continuous gradient.
RNAS-CL: Robust Neural Architecture Search by Cross-Layer Knowledge Distillation
In this paper, we propose Robust Neural Architecture Search by Cross-Layer Knowledge Distillation (RNAS-CL), a novel NAS algorithm that improves the robustness of NAS by learning from a robust teacher through cross-layer knowledge distillation.
Noisy $\ell^{0}$-Sparse Subspace Clustering on Dimensionality Reduced Data
Our results provide theoretical guarantee on the correctness of noisy $\ell^{0}$-SSC in terms of SDP on noisy data for the first time, which reveals the advantage of noisy $\ell^{0}$-SSC in terms of much less restrictive condition on subspace affinity.
Benchmark of DNN Model Search at Deployment Time
Deep learning has become the most popular direction in machine learning and artificial intelligence.
Bayesian Robust Graph Contrastive Learning
In this work, we propose a novel and robust method, Bayesian Robust Graph Contrastive Learning (BRGCL), which trains a GNN encoder to learn robust node representations.
Adaptive Cross-Layer Attention for Image Restoration
Non-local attention module has been proven to be crucial for image restoration.
Eliciting Structural and Semantic Global Knowledge in Unsupervised Graph Contrastive Learning
In general, the contrastive learning process in GCL is performed on top of the representations learned by a graph neural network (GNN) backbone, which transforms and propagates the node contextual information based on its local neighborhoods.
Discriminative Similarity for Data Clustering
Similarity-based clustering methods separate data into clusters according to the pairwise similarity between the data, and the pairwise similarity is crucial for their performance.
Adjoined Networks: A Training Paradigm with Applications to Network Compression
In this paper, we introduce Adjoined Networks, or AN, a learning paradigm that trains both the original base network and the smaller compressed network together.
FSNet: Compression of Deep Convolutional Neural Networks by Filter Summary
FSNet has the same architecture as that of the baseline CNN to be compressed, and each convolution layer of FSNet has the same number of filters from FS as that of the basline CNN in the forward process.
An Empirical Study on Regularization of Deep Neural Networks by Local Rademacher Complexity
In this paper, we investigate the role of Rademacher complexity in improving generalization of DNNs and propose a novel regularizer rooted in Local Rademacher Complexity (LRC).
Learning $3$D-FilterMap for Deep Convolutional Neural Networks
Due to the weight sharing scheme, the parameter size of the $3$D-FilterMap is much smaller than that of the filters to be learned in the conventional convolution layer when $3$D-FilterMap generates the same number of filters.
WSNet: Learning Compact and Efficient Networks with Weight Sampling
We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks.
WSNet: Compact and Efficient Networks Through Weight Sampling
We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks.
On the Suboptimality of Proximal Gradient Descent for $\ell^{0}$ Sparse Approximation
We study the proximal gradient descent (PGD) method for $\ell^{0}$ sparse approximation problem as well as its accelerated optimization with randomized algorithms in this paper.
Discriminative Similarity for Clustering and Semi-Supervised Learning
By generalization analysis via Rademacher complexity, the generalization error bound for the kernel classifier learned from hypothetical labeling is expressed as the sum of pairwise similarity between the data from different classes, parameterized by the weights of the kernel classifier.
D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain (D3) based fast restoration model to remove artifacts of JPEG compressed images.
Learning A Deep $\ell_\infty$ Encoder for Hashing
We investigate the $\ell_\infty$-constrained representation which demonstrates robustness to quantization errors, utilizing the tool of deep learning.
$\mathbf{D^3}$: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images
In this paper, we design a Deep Dual-Domain ($\mathbf{D^3}$) based fast restoration model to remove artifacts of JPEG compressed images.
Studying Very Low Resolution Recognition Using Deep Networks
Visual recognition research often assumes a sufficient resolution of the region of interest (ROI).
Learning with $\ell^{0}$-Graph: $\ell^{0}$-Induced Sparse Subspace Clustering
Sparse subspace clustering methods, such as Sparse Subspace Clustering (SSC) \cite{ElhamifarV13} and $\ell^{1}$-graph \cite{YanW09, ChengYYFH10}, are effective in partitioning the data that lie in a union of subspaces.
Self-Tuned Deep Super Resolution
Deep learning has been successfully applied to image super resolution (SR).
Designing A Composite Dictionary Adaptively From Joint Examples
We study the complementary behaviors of external and internal examples in image restoration, and are motivated to formulate a composite dictionary design framework.
Learning Super-Resolution Jointly from External and Internal Examples
Single image super-resolution (SR) aims to estimate a high-resolution (HR) image from a lowresolution (LR) input.
On a Theory of Nonparametric Pairwise Similarity for Clustering: Connecting Clustering to Classification
Modeling the underlying data distribution by nonparametric kernel density estimation, the generalization error bounds for both unsupervised nonparametric classifiers are the sum of nonparametric pairwise similarity terms between the data points for the purpose of clustering.
Nonparametric Unsupervised Classification
Unsupervised classification methods learn a discriminative classifier from unlabeled data, which has been proven to be an effective way of simultaneously clustering the data and training a classifier from the data.
