Search Results for author: Haizhao Yang

Found 26 papers, 5 papers with code

ReLU Network Approximation in Terms of Intrinsic Parameters

no code implementations15 Nov 2021 Zuowei Shen, Haizhao Yang, Shijun Zhang

First, we prove by construction that, for any Lipschitz continuous function $f$ on $[0, 1]^d$ with a Lipschitz constant $\lambda>0$, a ReLU network with $n+2$ intrinsic parameters can approximate $f$ with an exponentially small error $5\lambda \sqrt{d}\, 2^{-n}$ measured in the $L^p$-norm for $p\in [1,\infty)$.

Stationary Density Estimation of Itô Diffusions Using Deep Learning

no code implementations9 Sep 2021 Yiqi Gu, John Harlim, Senwei Liang, Haizhao Yang

In this paper, we consider the density estimation problem associated with the stationary measure of ergodic It\^o diffusions from a discrete-time series that approximate the solutions of the stochastic differential equations.

Density Estimation Time Series

Blending Pruning Criteria for Convolutional Neural Networks

no code implementations11 Jul 2021 wei he, Zhongzhan Huang, Mingfu Liang, Senwei Liang, Haizhao Yang

One filter could be important according to a certain criterion, while it is unnecessary according to another one, which indicates that each criterion is only a partial view of the comprehensive "importance".

Network Pruning

Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons

no code implementations6 Jul 2021 Zuowei Shen, Haizhao Yang, Shijun Zhang

This paper develops simple feed-forward neural networks that achieve the universal approximation property for all continuous functions with a fixed finite number of neurons.

Solving PDEs on Unknown Manifolds with Machine Learning

no code implementations12 Jun 2021 Senwei Liang, Shixiao W. Jiang, John Harlim, Haizhao Yang

The resulting numerical method is to solve a highly non-convex empirical risk minimization problem subjected to a solution from a hypothesis space of neural-network type functions.

Learning Theory

The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation

no code implementations21 Mar 2021 Qiang Du, Yiqi Gu, Haizhao Yang, Chao Zhou

We put forward error estimates for these methods using the approximation property of deep networks.

Optimal Approximation Rate of ReLU Networks in terms of Width and Depth

no code implementations28 Feb 2021 Zuowei Shen, Haizhao Yang, Shijun Zhang

This paper concentrates on the approximation power of deep feed-forward neural networks in terms of width and depth.

Reproducing Activation Function for Deep Learning

no code implementations13 Jan 2021 Senwei Liang, Liyao Lyu, Chunmei Wang, Haizhao Yang

We propose reproducing activation functions (RAFs) to improve deep learning accuracy for various applications ranging from computer vision to scientific computing.

Image Reconstruction Video Reconstruction

Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning

no code implementations15 Dec 2020 Fan Chen, Jianguo Huang, Chunmei Wang, Haizhao Yang

This paper proposes Friedrichs learning as a novel deep learning methodology that can learn the weak solutions of PDEs via a minmax formulation, which transforms the PDE problem into a minimax optimization problem to identify weak solutions.

Efficient Attention Network: Accelerate Attention by Searching Where to Plug

1 code implementation28 Nov 2020 Zhongzhan Huang, Senwei Liang, Mingfu Liang, wei he, Haizhao Yang

Recently, many plug-and-play self-attention modules are proposed to enhance the model generalization by exploiting the internal information of deep convolutional neural networks (CNNs).

Neural Network Approximation: Three Hidden Layers Are Enough

no code implementations25 Oct 2020 Zuowei Shen, Haizhao Yang, Shijun Zhang

A three-hidden-layer neural network with super approximation power is introduced.

Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory

no code implementations28 Jun 2020 Tao Luo, Haizhao Yang

The problem of solving partial differential equations (PDEs) can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions.

Deep Network with Approximation Error Being Reciprocal of Width to Power of Square Root of Depth

no code implementations22 Jun 2020 Zuowei Shen, Haizhao Yang, Shijun Zhang

More generally for an arbitrary continuous function $f$ on $[0, 1]^d$ with a modulus of continuity $\omega_f(\cdot)$, the constructive approximation rate is $\omega_f(\sqrt{d}\, N^{-\sqrt{L}})+2\omega_f(\sqrt{d}){N^{-\sqrt{L}}}$.

Deep Network Approximation for Smooth Functions

no code implementations9 Jan 2020 Jianfeng Lu, Zuowei Shen, Haizhao Yang, Shijun Zhang

This paper establishes the (nearly) optimal approximation error characterization of deep rectified linear unit (ReLU) networks for smooth functions in terms of both width and depth simultaneously.

Machine Learning for Prediction with Missing Dynamics

no code implementations13 Oct 2019 John Harlim, Shixiao W. Jiang, Senwei Liang, Haizhao Yang

This article presents a general framework for recovering missing dynamical systems using available data and machine learning techniques.

Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise

2 code implementations12 Aug 2019 Senwei Liang, Zhongzhan Huang, Mingfu Liang, Haizhao Yang

Batch Normalization (BN)(Ioffe and Szegedy 2015) normalizes the features of an input image via statistics of a batch of images and hence BN will bring the noise to the gradient of the training loss.

Image Classification

Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theorem

no code implementations27 Jun 2019 Hadrien Montanelli, Haizhao Yang

We prove a theorem concerning the approximation of multivariate functions by deep ReLU networks, for which the curse of the dimensionality is lessened.

Deep Network Approximation Characterized by Number of Neurons

no code implementations13 Jun 2019 Zuowei Shen, Haizhao Yang, Shijun Zhang

This paper quantitatively characterizes the approximation power of deep feed-forward neural networks (FNNs) in terms of the number of neurons.

DIANet: Dense-and-Implicit Attention Network

3 code implementations25 May 2019 Zhongzhan Huang, Senwei Liang, Mingfu Liang, Haizhao Yang

Attention networks have successfully boosted the performance in various vision problems.

Image Classification

Generative Imaging and Image Processing via Generative Encoder

no code implementations23 May 2019 Lin Chen, Haizhao Yang

This paper introduces a novel generative encoder (GE) model for generative imaging and image processing with applications in compressed sensing and imaging, image compression, denoising, inpainting, deblurring, and super-resolution.

Deblurring Denoising +2

SelectNet: Learning to Sample from the Wild for Imbalanced Data Training

no code implementations23 May 2019 Yunru Liu, Tingran Gao, Haizhao Yang

Supervised learning from training data with imbalanced class sizes, a commonly encountered scenario in real applications such as anomaly/fraud detection, has long been considered a significant challenge in machine learning.

Fraud Detection

CASS: Cross Adversarial Source Separation via Autoencoder

no code implementations23 May 2019 Yong Zheng Ong, Charles K. Chui, Haizhao Yang

This paper introduces a cross adversarial source separation (CASS) framework via autoencoder, a new model that aims at separating an input signal consisting of a mixture of multiple components into individual components defined via adversarial learning and autoencoder fitting.

Dimensionality Reduction

Nonlinear Approximation via Compositions

no code implementations26 Feb 2019 Zuowei Shen, Haizhao Yang, Shijun Zhang

In particular, for any function $f$ on $[0, 1]$, regardless of its smoothness and even the continuity, if $f$ can be approximated using a dictionary when $L=1$ with the best $N$-term approximation rate $\varepsilon_{L, f}={\cal O}(N^{-\eta})$, we show that dictionaries with $L=2$ can improve the best $N$-term approximation rate to $\varepsilon_{L, f}={\cal O}(N^{-2\eta})$.

Non-Oscillatory Pattern Learning for Non-Stationary Signals

no code implementations21 May 2018 Jieren Xu, Yitong Li, David Dunson, Ingrid Daubechies, Haizhao Yang

This paper proposes a novel non-oscillatory pattern (NOP) learning scheme for several oscillatory data analysis problems including signal decomposition, super-resolution, and signal sub-sampling.


Recursive Diffeomorphism-Based Regression for Shape Functions

1 code implementation12 Oct 2016 Jieren Xu, Haizhao Yang, Ingrid Daubechies

This paper proposes a recursive diffeomorphism based regression method for one-dimensional generalized mode decomposition problem that aims at extracting generalized modes $\alpha_k(t)s_k(2\pi N_k\phi_k(t))$ from their superposition $\sum_{k=1}^K \alpha_k(t)s_k(2\pi N_k\phi_k(t))$.

Cannot find the paper you are looking for? You can Submit a new open access paper.