Search Results for author:
Found 79 papers, 15 papers with code
Mutual Transfer Learning for Massive Data
In this article, we study a new paradigm called mutual transfer learning where among many heterogeneous data domains, every data domain could potentially be the target of interest, and it could also be a useful source to help the learning in other data domains.
Minimax Optimal Fair Classification with Bounded Demographic Disparity
Mitigating the disparate impact of statistical machine learning methods is crucial for ensuring fairness.
Approximation of RKHS Functionals by Neural Networks
Motivated by the abundance of functional data such as time series and images, there has been a growing interest in integrating such data into neural networks and learning maps from function spaces to R (i. e., functionals).
FairRR: Pre-Processing for Group Fairness through Randomized Response
The increasing usage of machine learning models in consequential decision-making processes has spurred research into the fairness of these systems.
Rate-Optimal Rank Aggregation with Private Pairwise Rankings
Motivated from this, we propose a debiased randomized response mechanism to protect the raw pairwise rankings, ensuring consistent estimation of true preferences and rankings in downstream rank aggregation.
Bayes-Optimal Fair Classification with Linear Disparity Constraints via Pre-, In-, and Post-processing
To address this, we develop methods for Bayes-optimal fair classification, aiming to minimize classification error subject to given group fairness constraints.
Benefits of Transformer: In-Context Learning in Linear Regression Tasks with Unstructured Data
In practice, it is observed that transformer-based models can learn concepts in context in the inference stage.
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective
Pre-training is known to generate universal representations for downstream tasks in large-scale deep learning such as large language models.
A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models
In this article, we review the literature on statistical theories of neural networks from three perspectives.
Downstream Task-Oriented Generative Model Selections on Synthetic Data Training for Fraud Detection Models
Devising procedures for downstream task-oriented generative model selections is an unresolved problem of practical importance.
Improve Fidelity and Utility of Synthetic Credit Card Transaction Time Series from Data-centric Perspective
Exploring generative model training for synthetic tabular data, specifically in sequential contexts such as credit card transaction data, presents significant challenges.
MalPurifier: Enhancing Android Malware Detection with Adversarial Purification against Evasion Attacks
Experimental results on two Android malware datasets demonstrate that MalPurifier outperforms the state-of-the-art defenses, and it significantly strengthens the vulnerable malware detector against 37 evasion attacks, achieving accuracies over 90. 91%.
AutoDiff: combining Auto-encoder and Diffusion model for tabular data synthesizing
Diffusion model has become a main paradigm for synthetic data generation in many subfields of modern machine learning, including computer vision, language model, or speech synthesis.
Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule
As a modern ensemble technique, Deep Forest (DF) employs a cascading structure to construct deep models, providing stronger representational power compared to traditional decision forests.
MissDiff: Training Diffusion Models on Tabular Data with Missing Values
The diffusion model has shown remarkable performance in modeling data distributions and synthesizing data.
Utility Theory of Synthetic Data Generation
The former is defined as the generalization difference between models trained on synthetic and on real data.
Tight Non-asymptotic Inference via Sub-Gaussian Intrinsic Moment Norm
In non-asymptotic learning, variance-type parameters of sub-Gaussian distributions are of paramount importance.
Double Matching Under Complementary Preferences
In this paper, we propose a new algorithm for addressing the problem of matching markets with complementary preferences, where agents' preferences are unknown a priori and must be learned from data.
Statistical Theory of Differentially Private Marginal-based Data Synthesis Algorithms
To complete the picture, we establish a lower bound for TV accuracy that holds for every $\epsilon$-DP synthetic data generator.
Ranking Differential Privacy
This allows us to develop a multistage ranking algorithm to generate synthetic rankings while satisfying the developed $\epsilon$-ranking differential privacy.
On the Utility Recovery Incapability of Neural Net-based Differential Private Tabular Training Data Synthesizer under Privacy Deregulation
Devising procedures for auditing generative model privacy-utility tradeoff is an important yet unresolved problem in practice.
Improving Adversarial Robustness by Contrastive Guided Diffusion Process
Among various deep generative models, the diffusion model has been shown to produce high-quality synthetic images and has achieved good performance in improving the adversarial robustness.
Differentially Private Bootstrap: New Privacy Analysis and Inference Strategies
Differentially private (DP) mechanisms protect individual-level information by introducing randomness into the statistical analysis procedure.
Fair Bayes-Optimal Classifiers Under Predictive Parity
This paper considers predictive parity, which requires equalizing the probability of success given a positive prediction among different protected groups.
Rate-Optimal Contextual Online Matching Bandit
Existing works focus on multi-armed bandit with static preference, but this is insufficient: the two-sided preference changes as along as one-side's contextual information updates, resulting in non-static matching.
Federated Online Sparse Decision Making
This paper presents a novel federated linear contextual bandits model, where individual clients face different K-armed stochastic bandits with high-dimensional decision context and coupled through common global parameters.
Enhanced Nearest Neighbor Classification for Crowdsourcing
In machine learning, crowdsourcing is an economical way to label a large amount of data.
Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions
Convolutional neural networks have shown impressive abilities in many applications, especially those related to the classification tasks.
Attention Enables Zero Approximation Error
In this work, we show that with suitable adaptations, the single-head self-attention transformer with a fixed number of transformer encoder blocks and free parameters is able to generate any desired polynomial of the input with no error.
Residual Bootstrap Exploration for Stochastic Linear Bandit
We propose a new bootstrap-based online algorithm for stochastic linear bandit problems.
Benefit of Interpolation in Nearest Neighbor Algorithms
In some studies \citep[e. g.,][]{zhang2016understanding} of deep learning, it is observed that over-parametrized deep neural networks achieve a small testing error even when the training error is almost zero.
Bayes-Optimal Classifiers under Group Fairness
Machine learning algorithms are becoming integrated into more and more high-stakes decision-making processes, such as in social welfare issues.
Unlabeled Data Help: Minimax Analysis and Adversarial Robustness
The recent proposed self-supervised learning (SSL) approaches successfully demonstrate the great potential of supplementing learning algorithms with additional unlabeled data.
High-Dimensional Inference over Networks: Linear Convergence and Statistical Guarantees
Our theory shows that, under standard notions of restricted strong convexity and smoothness of the loss functions, suitable conditions on the network connectivity and algorithm tuning, the distributed algorithm converges globally at a {\it linear} rate to an estimate that is within the centralized {\it statistical precision} of the model, $O(s\log d/N)$.
On the Algorithmic Stability of Adversarial Training
In contrast, this paper studies the algorithmic stability of a generic adversarial training algorithm, which can further help to establish an upper bound for generalization error.
Online Bootstrap Inference For Policy Evaluation in Reinforcement Learning
The recent emergence of reinforcement learning has created a demand for robust statistical inference methods for the parameter estimates computed using these algorithms.
Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization
In this paper, we make the key delineation on the roles of resolution and statistical uncertainty in hierarchical bandits-based black-box optimization algorithms, guiding a more general analysis and a more efficient algorithm design.
Distributed Bootstrap for Simultaneous Inference Under High Dimensionality
We propose a distributed bootstrap method for simultaneous inference on high-dimensional massive data that are stored and processed with many machines.
On the Marginal Regret Bound Minimization of Adaptive Methods
Numerous adaptive algorithms such as AMSGrad and Radam have been proposed and applied to deep learning recently.
Variance Reduction on General Adaptive Stochastic Mirror Descent
In this work, we investigate the idea of variance reduction by studying its properties with general adaptive mirror descent algorithms in nonsmooth nonconvex finite-sum optimization problems.
Adversarially Robust Estimate and Risk Analysis in Linear Regression
Further, we reveal an explicit connection of adversarial and standard estimates, and propose a straightforward two-stage adversarial learning framework, which facilitates to utilize model structure information to improve adversarial robustness.
Online Forgetting Process for Linear Regression Models
Motivated by the EU's "Right To Be Forgotten" regulation, we initiate a study of statistical data deletion problems where users' data are accessible only for a limited period of time.
Statistical Guarantees of Distributed Nearest Neighbor Classification
It is interesting to note that the weighted voting scheme allows a larger number of subsamples than the majority voting one.
Efficient Variational Inference for Sparse Deep Learning with Theoretical Guarantee
Sparse deep learning aims to address the challenge of huge storage consumption by deep neural networks, and to recover the sparse structure of target functions.
Nearly Optimal Variational Inference for High Dimensional Regression with Shrinkage Priors
We propose a variational Bayesian (VB) procedure for high-dimensional linear model inferences with heavy tail shrinkage priors, such as student-t prior.
On the Generalization Properties of Adversarial Training
Modern machine learning and deep learning models are shown to be vulnerable when testing data are slightly perturbed.
Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network
Overparametrized neural networks trained by gradient descent (GD) can provably overfit any training data.
Online Regularization towards Always-Valid High-Dimensional Dynamic Pricing
In this paper, we propose a novel approach for designing dynamic pricing policy based regularized online statistical learning with theoretical guarantees.
Directional Pruning of Deep Neural Networks
In the light of the fact that the stochastic gradient descent (SGD) often finds a flat minimum valley in the training loss, we propose a novel directional pruning method which searches for a sparse minimizer in or close to that flat region.
On Deep Instrumental Variables Estimate
The endogeneity issue is fundamentally important as many empirical applications may suffer from the omission of explanatory variables, measurement error, or simultaneous causality.
Online Batch Decision-Making with High-Dimensional Covariates
In such a scenario, our goal is to allocate a batch of treatments to maximize treatment efficacy based on observed high-dimensional user covariates.
Residual Bootstrap Exploration for Bandit Algorithms
In this paper, we propose a novel perturbation-based exploration method in bandit algorithms with bounded or unbounded rewards, called residual bootstrap exploration (\texttt{ReBoot}).
Simultaneous Inference for Massive Data: Distributed Bootstrap
In this paper, we propose a bootstrap method applied to massive data processed distributedly in a large number of machines.
Predictive Power of Nearest Neighbors Algorithm under Random Perturbation
We consider a data corruption scenario in the classical $k$ Nearest Neighbors ($k$-NN) algorithm, that is, the testing data are randomly perturbed.
Sharp Rate of Convergence for Deep Neural Network Classifiers under the Teacher-Student Setting
In this paper, we attempt to understand this empirical success in high dimensional classification by deriving the convergence rates of excess risk.
Benefit of Interpolation in Nearest Neighbor Algorithms
The over-parameterized models attract much attention in the era of data science and deep learning.
A generalization of regularized dual averaging and its dynamics
Preliminary empirical analysis of modern image data shows that learning very sparse deep neural networks by gRDA does not necessarily sacrifice testing accuracy.
Machine Learning in/for Blockchain: Future and Challenges
Machine learning and blockchain are two of the most noticeable technologies in recent years.
Rates of Convergence for Large-scale Nearest Neighbor Classification
Nearest neighbor is a popular class of classification methods with many desirable properties.
Bootstrapping Upper Confidence Bound
Upper Confidence Bound (UCB) method is arguably the most celebrated one used in online decision making with partial information feedback.
Building an Efficient Intrusion Detection System Based on Feature Selection and Ensemble Classifier
Intrusion detection system (IDS) is one of extensively used techniques in a network topology to safeguard the integrity and availability of sensitive assets in the protected systems.
Stein Neural Sampler
We propose two novel samplers to generate high-quality samples from a given (un-normalized) probability density.
Statistical Optimality of Interpolated Nearest Neighbor Algorithms
In the era of deep learning, understanding over-fitting phenomenon becomes increasingly important.
Statistically and Computationally Efficient Variance Estimator for Kernel Ridge Regression
In this paper, we propose a random projection approach to estimate variance in kernel ridge regression.
Optimal Tuning for Divide-and-conquer Kernel Ridge Regression with Massive Data
Divide-and-conquer is a powerful approach for large and massive data analysis.
How Many Machines Can We Use in Parallel Computing for Kernel Ridge Regression?
It is worth noting that the upper bounds of the number of machines are proven to be un-improvable (upto a logarithmic factor) in two important cases: smoothing spline regression and Gaussian RKHS regression.
Early Stopping for Nonparametric Testing
Early stopping of iterative algorithms is an algorithmic regularization method to avoid over-fitting in estimation and classification.
Nonparametric Testing under Random Projection
A common challenge in nonparametric inference is its high computational complexity when data volume is large.
Sparse and Low-rank Tensor Estimation via Cubic Sketchings
In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings.
Stability Enhanced Large-Margin Classifier Selection
Stability is an important aspect of a classification procedure because unstable predictions can potentially reduce users' trust in a classification system and also harm the reproducibility of scientific conclusions.
Distributed Generalized Cross-Validation for Divide-and-Conquer Kernel Ridge Regression and its Asymptotic Optimality
Tuning parameter selection is of critical importance for kernel ridge regression.
Simultaneous Clustering and Estimation of Heterogeneous Graphical Models
We consider joint estimation of multiple graphical models arising from heterogeneous and high-dimensional observations.
Tensor Graphical Model: Non-convex Optimization and Statistical Inference
We consider the estimation and inference of graphical models that characterize the dependency structure of high-dimensional tensor-valued data.
Computational Limits of A Distributed Algorithm For Smoothing Spline
In this paper, we explore statistical versus computational trade-off to address a basic question in the application of a distributed algorithm: what is the minimal computational cost in obtaining statistical optimality?
Statistics Theory Statistics Theory
Non-convex Statistical Optimization for Sparse Tensor Graphical Model
We consider the estimation of sparse graphical models that characterize the dependency structure of high-dimensional tensor-valued data.
Provable Sparse Tensor Decomposition
We propose a novel sparse tensor decomposition method, namely Tensor Truncated Power (TTP) method, that incorporates variable selection into the estimation of decomposition components.
Tracking on the Product Manifold of Shape and Orientation for Tractography from Diffusion MRI
In this paper, we propose a new intrinsic recursive filter on the product manifold of shape and orientation.
Stabilized Nearest Neighbor Classifier and Its Statistical Properties
In this paper, we introduce a general measure of classification instability (CIS) to quantify the sampling variability of the prediction made by a classification method.
Local and global asymptotic inference in smoothing spline models
In particular, our confidence intervals are proved to be asymptotically valid at any point in the support, and they are shorter on average than the Bayesian confidence intervals proposed by Wahba [J. R. Stat.
