Search Results for author:
Found 24 papers, 5 papers with code
A Mean Field Analysis Of Deep ResNet And Beyond: Towards Provably Optimization Via Overparameterization From Depth
Specifically, we propose a \textbf{new continuum limit} of deep residual networks, which enjoys a good landscape in the sense that \textbf{every local minimizer is global}.
Generalization of Scaled Deep ResNets in the Mean-Field Regime
To derive the generalization bounds under this setting, our analysis necessitates a shift from the conventional time-invariant Gram matrix employed in the lazy training regime to a time-variant, distribution-dependent version.
Statistical Spatially Inhomogeneous Diffusion Inference
Inferring a diffusion equation from discretely-observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments.
When can Regression-Adjusted Control Variates Help? Rare Events, Sobolev Embedding and Minimax Optimality
We demonstrate that this kind of quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under a sufficient smoothness assumption.
Synthetic Principal Component Design: Fast Covariate Balancing with Synthetic Controls
The optimal design of experiments typically involves solving an NP-hard combinatorial optimization problem.
Minimax Optimal Kernel Operator Learning via Multilevel Training
Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional data analysis, causal inference, and multi-agent reinforcement learning.
Importance Tempering: Group Robustness for Overparameterized Models
In this paper, we propose importance tempering to improve the decision boundary and achieve consistently better results for overparameterized models.
Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks
Adversarial examples, which are usually generated for specific inputs with a specific model, are ubiquitous for neural networks.
Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent
In this paper, we study the statistical limits in terms of Sobolev norms of gradient descent for solving inverse problem from randomly sampled noisy observations using a general class of objective functions.
Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality
In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).
An Unconstrained Layer-Peeled Perspective on Neural Collapse
We prove that gradient flow on this model converges to critical points of a minimum-norm separation problem exhibiting neural collapse in its global minimizer.
Statistical Numerical PDE : Fast Rate, Neural Scaling Law and When it’s Optimal
In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).
How Gradient Descent Separates Data with Neural Collapse: A Layer-Peeled Perspective
In this paper, we derive a landscape analysis to the surrogate model to study the inductive bias of the neural features and parameters from neural networks with cross-entropy.
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth
Specifically, we propose a new continuum limit of deep residual networks, which enjoys a good landscape in the sense that every local minimizer is global.
A Mean-field Analysis of Deep ResNet and Beyond:Towards Provable Optimization Via Overparameterization From Depth
Specifically, we propose a \textbf{new continuum limit} of deep residual networks, which enjoys a good landscape in the sense that \textbf{every local minimizer is global}.
Distillation $\approx$ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized Neural Network
Assuming that the teacher network is overparameterized, we argue that the teacher network is essentially harvesting dark knowledge from the data via early stopping.
Distillation $\approx$ Early Stopping? Harvesting Dark Knowledge Utilizing Anisotropic Information Retrieval For Overparameterized NN
Assuming that the teacher network is overparameterized, we argue that the teacher network is essentially harvesting dark knowledge from the data via early stopping.
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
In this paper, we provide a novel perspective towards understanding the architecture: we show that the Transformer can be mathematically interpreted as a numerical Ordinary Differential Equation (ODE) solver for a convection-diffusion equation in a multi-particle dynamic system.
You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle
Adversarial training, typically formulated as a robust optimization problem, is an effective way of improving the robustness of deep networks.
CURE: Curvature Regularization For Missing Data Recovery
For that, we introduce a new regularization by combining the low dimension manifold regularization with a higher order Curvature Regularization, and we call this new regularization CURE for short.
PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
Numerical experiments show that the PDE-Net 2. 0 has the potential to uncover the hidden PDE of the observed dynamics, and predict the dynamical behavior for a relatively long time, even in a noisy environment.
Dynamically Unfolding Recurrent Restorer: A Moving Endpoint Control Method for Image Restoration
In this paper, we propose a new control framework called the moving endpoint control to restore images corrupted by different degradation levels in one model.
Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations
We show that many effective networks, such as ResNet, PolyNet, FractalNet and RevNet, can be interpreted as different numerical discretizations of differential equations.
PDE-Net: Learning PDEs from Data
In this paper, we present an initial attempt to learn evolution PDEs from data.
