Search Results for author:
Found 39 papers, 9 papers with code
Convergence Analysis of the Deep Galerkin Method for Weak Solutions
This paper analyzes the convergence rate of a deep Galerkin method for the weak solution (DGMW) of second-order elliptic partial differential equations on $\mathbb{R}^d$ with Dirichlet, Neumann, and Robin boundary conditions, respectively.
Deep Operator Learning Lessens the Curse of Dimensionality for PDEs
Deep neural networks (DNNs) have seen tremendous success in many fields and their developments in PDE-related problems are rapidly growing.
A Distributed Block Chebyshev-Davidson Algorithm for Parallel Spectral Clustering
We develop a distributed Block Chebyshev-Davidson algorithm to solve large-scale leading eigenvalue problems for spectral analysis in spectral clustering.
What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?
Additionally, we propose a Robust Multi-Agent Adversarial Actor-Critic (RMA3C) algorithm to learn robust policies for MARL agents under state uncertainties.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Accelerating Numerical Solvers for Large-Scale Simulation of Dynamical System via NeurVec
Conventional numerical solvers used in the simulation are significantly limited by the step size for time integration, which hampers efficiency and feasibility especially when high accuracy is desired.
The Lottery Ticket Hypothesis for Self-attention in Convolutional Neural Network
Recently many plug-and-play self-attention modules (SAMs) are proposed to enhance the model generalization by exploiting the internal information of deep convolutional neural networks (CNNs).
Finite Expression Method for Solving High-Dimensional Partial Differential Equations
Designing efficient and accurate numerical solvers for high-dimensional partial differential equations (PDEs) remains a challenging and important topic in computational science and engineering, mainly due to the "curse of dimensionality" in designing numerical schemes that scale in dimension.
Reinforced Inverse Scattering
Inverse wave scattering aims at determining the properties of an object using data on how the object scatters incoming waves.
Neural Network Architecture Beyond Width and Depth
It is proved by construction that height-$s$ ReLU NestNets with $\mathcal{O}(n)$ parameters can approximate $1$-Lipschitz continuous functions on $[0, 1]^d$ with an error $\mathcal{O}(n^{-(s+1)/d})$, while the optimal approximation error of standard ReLU networks with $\mathcal{O}(n)$ parameters is $\mathcal{O}(n^{-2/d})$.
IAE-Net: Integral Autoencoders for Discretization-Invariant Learning
Discretization invariant learning aims at learning in the infinite-dimensional function spaces with the capacity to process heterogeneous discrete representations of functions as inputs and/or outputs of a learning model.
From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality
Optimization and generalization are two essential aspects of statistical machine learning.
Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces
Learning operators between infinitely dimensional spaces is an important learning task arising in wide applications in machine learning, imaging science, mathematical modeling and simulations, etc.
Deep Network Approximation in Terms of Intrinsic Parameters
Furthermore, we show that the idea of learning a small number of parameters to achieve a good approximation can be numerically observed.
Short optimization paths lead to good generalization
Through our approach, we show that, with a proper initialization, gradient flow converges following a short path with an explicit length estimate.
Stationary Density Estimation of Itô Diffusions Using Deep Learning
In this paper, we consider the density estimation problem associated with the stationary measure of ergodic It\^o diffusions from a discrete-time series that approximate the solutions of the stochastic differential equations.
Blending Pruning Criteria for Convolutional Neural Networks
One filter could be important according to a certain criterion, while it is unnecessary according to another one, which indicates that each criterion is only a partial view of the comprehensive "importance".
Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons
This paper develops simple feed-forward neural networks that achieve the universal approximation property for all continuous functions with a fixed finite number of neurons.
Solving PDEs on Unknown Manifolds with Machine Learning
The resulting numerical method is to solve a highly non-convex empirical risk minimization problem subjected to a solution from a hypothesis space of neural networks.
The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation
We put forward error estimates for these methods using the approximation property of deep networks.
Optimal Approximation Rate of ReLU Networks in terms of Width and Depth
This paper concentrates on the approximation power of deep feed-forward neural networks in terms of width and depth.
Reproducing Activation Function for Deep Learning
We propose reproducing activation functions (RAFs) to improve deep learning accuracy for various applications ranging from computer vision to scientific computing.
Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning
This paper proposes Friedrichs learning as a novel deep learning methodology that can learn the weak solutions of PDEs via a minmax formulation, which transforms the PDE problem into a minimax optimization problem to identify weak solutions.
Efficient Attention Network: Accelerate Attention by Searching Where to Plug
Recently, many plug-and-play self-attention modules are proposed to enhance the model generalization by exploiting the internal information of deep convolutional neural networks (CNNs).
Neural Network Approximation: Three Hidden Layers Are Enough
A three-hidden-layer neural network with super approximation power is introduced.
Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
The problem of solving partial differential equations (PDEs) can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions.
Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth
More generally for an arbitrary continuous function $f$ on $[0, 1]^d$ with a modulus of continuity $\omega_f(\cdot)$, the constructive approximation rate is $\omega_f(\sqrt{d}\, N^{-\sqrt{L}})+2\omega_f(\sqrt{d}){N^{-\sqrt{L}}}$.
Deep Network Approximation for Smooth Functions
This paper establishes the (nearly) optimal approximation error characterization of deep rectified linear unit (ReLU) networks for smooth functions in terms of both width and depth simultaneously.
Machine Learning for Prediction with Missing Dynamics
This article presents a general framework for recovering missing dynamical systems using available data and machine learning techniques.
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise
Batch Normalization (BN)(Ioffe and Szegedy 2015) normalizes the features of an input image via statistics of a batch of images and hence BN will bring the noise to the gradient of the training loss.
Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theorem
We prove a theorem concerning the approximation of multivariate functions by deep ReLU networks, for which the curse of the dimensionality is lessened.
Deep Network Approximation Characterized by Number of Neurons
This paper quantitatively characterizes the approximation power of deep feed-forward neural networks (FNNs) in terms of the number of neurons.
DIANet: Dense-and-Implicit Attention Network
Attention networks have successfully boosted the performance in various vision problems.
Ranked #132 on
Image Classification
on CIFAR-100
SelectNet: Learning to Sample from the Wild for Imbalanced Data Training
Supervised learning from training data with imbalanced class sizes, a commonly encountered scenario in real applications such as anomaly/fraud detection, has long been considered a significant challenge in machine learning.
Generative Imaging and Image Processing via Generative Encoder
This paper introduces a novel generative encoder (GE) model for generative imaging and image processing with applications in compressed sensing and imaging, image compression, denoising, inpainting, deblurring, and super-resolution.
CASS: Cross Adversarial Source Separation via Autoencoder
This paper introduces a cross adversarial source separation (CASS) framework via autoencoder, a new model that aims at separating an input signal consisting of a mixture of multiple components into individual components defined via adversarial learning and autoencoder fitting.
Nonlinear Approximation via Compositions
In particular, for any function $f$ on $[0, 1]$, regardless of its smoothness and even the continuity, if $f$ can be approximated using a dictionary when $L=1$ with the best $N$-term approximation rate $\varepsilon_{L, f}={\cal O}(N^{-\eta})$, we show that dictionaries with $L=2$ can improve the best $N$-term approximation rate to $\varepsilon_{L, f}={\cal O}(N^{-2\eta})$.
Drop-Activation: Implicit Parameter Reduction and Harmonic Regularization
Overfitting frequently occurs in deep learning.
Ranked #10 on
Image Classification
on SVHN
PiPs: a Kernel-based Optimization Scheme for Analyzing Non-Stationary 1D Signals
This paper proposes a novel kernel-based optimization scheme to handle tasks in the analysis, e. g., signal spectral estimation and single-channel source separation of 1D non-stationary oscillatory data.
Recursive Diffeomorphism-Based Regression for Shape Functions
This paper proposes a recursive diffeomorphism based regression method for one-dimensional generalized mode decomposition problem that aims at extracting generalized modes $\alpha_k(t)s_k(2\pi N_k\phi_k(t))$ from their superposition $\sum_{k=1}^K \alpha_k(t)s_k(2\pi N_k\phi_k(t))$.
