Search Results for author:
Found 21 papers, 3 papers with code
Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation
Additionally, through an exploration of the representation power of deep ReLU$^k$ networks for shallow networks, we reveal that deep ReLU$^k$ networks can approximate functions from a range of variation spaces, extending beyond those generated solely by the ReLU$^k$ activation function.
Deep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions
In this study, we establish that deep neural networks employing ReLU and ReLU$^2$ activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary dimensions.
MgNO: Efficient Parameterization of Linear Operators via Multigrid
In this work, we propose a concise neural operator architecture for operator learning.
AceGPT, Localizing Large Language Models in Arabic
This paper is devoted to the development of a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models.
DualFL: A Duality-based Federated Learning Algorithm with Communication Acceleration in the General Convex Regime
We propose a new training algorithm, named DualFL (Dualized Federated Learning), for solving distributed optimization problems in federated learning.
FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecasting
By investigating iterative methods for a constrained linear model, we propose a new class of fully connected V-cycle MgNet for long-term time series forecasting, which is one of the most difficult tasks in forecasting.
On the Activation Function Dependence of the Spectral Bias of Neural Networks
Our empirical studies also show that neural networks with the Hat activation function are trained significantly faster using stochastic gradient descent and ADAM.
An Interpretive Constrained Linear Model for ResNet and MgNet
We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN).
Approximation Properties of Deep ReLU CNNs
This paper focuses on establishing $L^2$ approximation properties for deep ReLU convolutional neural networks (CNNs) in two-dimensional space.
Sharp Lower Bounds on the Approximation Rate of Shallow Neural Networks
In this article, we provide a solution to this problem by proving sharp lower bounds on the approximation rates for shallow neural networks, which are obtained by lower bounding the $L^2$-metric entropy of the convex hull of the neural network basis functions.
Characterization of the Variation Spaces Corresponding to Shallow Neural Networks
We study the variation space corresponding to a dictionary of functions in $L^2(\Omega)$ for a bounded domain $\Omega\subset \mathbb{R}^d$.
ReLU Deep Neural Networks from the Hierarchical Basis Perspective
We study ReLU deep neural networks (DNNs) by investigating their connections with the hierarchical basis method in finite element methods.
Sharp Bounds on the Approximation Rates, Metric Entropy, and $n$-widths of Shallow Neural Networks
This result gives sharp lower bounds on the $L^2$-approximation rates, metric entropy, and $n$-widths for variation spaces corresponding to neural networks with a range of important activation functions, including ReLU$^k$ activation functions and sigmoidal activation functions with bounded variation.
High-Order Approximation Rates for Shallow Neural Networks with Cosine and ReLU$^k$ Activation Functions
We show that as the smoothness index $s$ of $f$ increases, shallow neural networks with ReLU$^k$ activation function obtain an improved approximation rate up to a best possible rate of $O(n^{-(k+1)}\log(n))$ in $L^2$, independent of the dimension $d$.
Numerical Analysis Numerical Analysis 41A25
Training Sparse Neural Networks using Compressed Sensing
The adaptive weighting we introduce corresponds to a novel regularizer based on the logarithm of the absolute value of the weights.
Constrained Linear Data-feature Mapping for Image Classification
In this paper, we propose a constrained linear data-feature mapping model as an interpretable mathematical model for image classification using convolutional neural network (CNN) such as the ResNet.
A machine learning method correlating pulse pressure wave data with pregnancy
Pulse feeling, representing the tactile arterial palpation of the heartbeat, has been widely used in traditional Chinese medicine (TCM) to diagnose various diseases.
iRDA Method for Sparse Convolutional Neural Networks
We propose a new approach, known as the iterative regularized dual averaging (iRDA), to improve the efficiency of convolutional neural networks (CNN) by significantly reducing the redundancy of the model without reducing its accuracy.
Approximation Rates for Neural Networks with General Activation Functions
Our first result concerns the rate of approximation of a two layer neural network with a polynomially-decaying non-sigmoidal activation function.
MgNet: A Unified Framework of Multigrid and Convolutional Neural Network
We develop a unified model, known as MgNet, that simultaneously recovers some convolutional neural networks (CNN) for image classification and multigrid (MG) methods for solving discretized partial differential equations (PDEs).
Make $\ell_1$ Regularization Effective in Training Sparse CNN
Compressed Sensing using $\ell_1$ regularization is among the most powerful and popular sparsification technique in many applications, but why has it not been used to obtain sparse deep learning model such as convolutional neural network (CNN)?
