Search Results for author:
Found 28 papers, 3 papers with code
Robust NAS under adversarial training: benchmark, theory, and beyond
Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data.
Generalization of Scaled Deep ResNets in the Mean-Field Regime
To derive the generalization bounds under this setting, our analysis necessitates a shift from the conventional time-invariant Gram matrix employed in the lazy training regime to a time-variant, distribution-dependent version.
Can overfitted deep neural networks in adversarial training generalize? -- An approximation viewpoint
Adversarial training is a widely used method to improve the robustness of deep neural networks (DNNs) over adversarial perturbations.
Efficient local linearity regularization to overcome catastrophic overfitting
Our regularization term can be theoretically linked to curvature of the loss function and is computationally cheaper than previous methods by avoiding Double Backpropagation.
Federated Learning under Covariate Shifts with Generalization Guarantees
This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance.
Benign Overfitting in Deep Neural Networks under Lazy Training
This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification while obtaining (nearly) zero-training error under the lazy training regime.
What can online reinforcement learning with function approximation benefit from general coverage conditions?
In online reinforcement learning (RL), instead of employing standard structural assumptions on Markov decision processes (MDPs), using a certain coverage condition (original from offline RL) is enough to ensure sample-efficient guarantees (Xie et al. 2023).
Random Fourier Features for Asymmetric Kernels
The theoretical foundation of RFFs is based on the Bochner theorem that relates symmetric, positive definite (PD) functions to probability measures.
Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study
Neural tangent kernel (NTK) is a powerful tool to analyze training dynamics of neural networks and their generalization bounds.
Generalization Properties of NAS under Activation and Skip Connection Search
To this end, we derive the lower (and upper) bounds of the minimum eigenvalue of the Neural Tangent Kernel (NTK) under the (in)finite-width regime using a certain search space including mixed activation functions, fully connected, and residual neural networks.
Understanding Deep Neural Function Approximation in Reinforcement Learning via $ε$-Greedy Exploration
To be specific, we focus on the value based algorithm with the $\epsilon$-greedy exploration via deep (and two-layer) neural networks endowed by Besov (and Barron) function spaces, respectively, which aims at approximating an $\alpha$-smooth Q-function in a $d$-dimensional feature space.
Sound and Complete Verification of Polynomial Networks
Polynomial Networks (PNs) have demonstrated promising performance on face and image recognition recently.
Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)
In particular, when initialized with LeCun initialization, depth helps robustness with the lazy training regime.
On the Double Descent of Random Features Models Trained with SGD
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime.
Towards a Unified Quadrature Framework for Large-Scale Kernel Machines
In this paper, we develop a quadrature framework for large-scale kernel machines via a numerical integration representation.
Kernel regression in high dimensions: Refined analysis beyond double descent
In this paper, we provide a precise characterization of generalization properties of high dimensional kernel ridge regression across the under- and over-parameterized regimes, depending on whether the number of training data n exceeds the feature dimension d. By establishing a bias-variance decomposition of the expected excess risk, we show that, while the bias is (almost) independent of d and monotonically decreases with n, the variance depends on n, d and can be unimodal or monotonically decreasing under different regularization schemes.
End-to-end Kernel Learning via Generative Random Fourier Features
In the second-stage process, a linear learner is conducted with respect to the mapped random features.
Analysis of Regularized Least Squares in Reproducing Kernel Krein Spaces
In this paper, we study the asymptotic properties of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS).
Fast Learning in Reproducing Kernel Krein Spaces via Signed Measures
In this paper, we attempt to solve a long-lasting open question for non-positive definite (non-PD) kernels in machine learning community: can a given non-PD kernel be decomposed into the difference of two PD kernels (termed as positive decomposition)?
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond
This survey may serve as a gentle introduction to this topic, and as a users' guide for practitioners interested in applying the representative algorithms and understanding theoretical results under various technical assumptions.
Random Fourier Features via Fast Surrogate Leverage Weighted Sampling
In this paper, we propose a fast surrogate leverage weighted sampling strategy to generate refined random Fourier features for kernel approximation.
Deep Kernel Learning via Random Fourier Features
On small datasets (less than 1000 samples), for which deep learning is generally not suitable due to overfitting, our method achieves superior performance compared to advanced kernel methods.
Robust Visual Tracking Revisited: From Correlation Filter to Template Matching
In this paper, we propose a novel matching based tracker by investigating the relationship between template matching and the recent popular correlation filter based trackers (CFTs).
Generalization Properties of hyper-RKHS and its Applications
This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view.
Learning Data-adaptive Nonparametric Kernels
Learning this data-adaptive matrix in a formulation-free strategy enlarges the margin between classes and thus improves the model flexibility.
Indefinite Kernel Logistic Regression with Concave-inexact-convex Procedure
Since the concave-convex procedure has to solve a sub-problem in each iteration, we propose a concave-inexact-convex procedure (CCICP) algorithm with an inexact solving scheme to accelerate the solving process.
Visual Tracking via Nonnegative Regularization Multiple Locality Coding
Weights of these various dictionaries are also learned from approximated LLC in the similar framework.
Robust Visual Tracking via Inverse Nonnegative Matrix Factorization
It utilizes both the foreground and background information, and imposes a local coordinate constraint, where the basis matrix is sparse matrix from the linear combination of candidates with corresponding nonnegative coefficient vectors.
