Search Results for author: Jianfeng Lu

Found 55 papers, 12 papers with code

A Mean Field Analysis Of Deep ResNet And Beyond: Towards Provably Optimization Via Overparameterization From Depth

no code implementations ICML 2020 Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying

Specifically, we propose a \textbf{new continuum limit} of deep residual networks, which enjoys a good landscape in the sense that \textbf{every local minimizer is global}.

Gait Recognition from a Single Image using a Phase-Aware Gait Cycle Reconstruction Network

no code implementations ECCV 2020 Chi Xu, Yasushi Makihara, Xiang Li, Yasushi Yagi, Jianfeng Lu

Specifically, a phase estimation network is introduced for the input single image, and the gait cycle reconstruction network exploits the estimated phase to mitigate the dependence of an encoded feature on the phase of that single image.

Gait Recognition

Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees

no code implementations31 Jan 2022 Mo Zhou, Jianfeng Lu

We propose a single time-scale actor-critic algorithm to solve the linear quadratic regulator (LQR) problem.

Bilevel Optimization

A Regularity Theory for Static Schrödinger Equations on $\mathbb{R}^d$ in Spectral Barron Spaces

no code implementations25 Jan 2022 Ziang Chen, Jianfeng Lu, Yulong Lu, Shengxuan Zhou

Spectral Barron spaces have received considerable interest recently as it is the natural function space for approximation theory of two-layer neural networks with a dimension-free convergence rate.

Machine Learning For Elliptic PDEs: Fast Rate Generalization Bound, Neural Scaling Law and Minimax Optimality

no code implementations ICLR 2022 Yiping Lu, Haoxuan Chen, Jianfeng Lu, Lexing Ying, Jose Blanchet

In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).

Statistical Numerical PDE : Fast Rate, Neural Scaling Law and When it’s Optimal

no code implementations NeurIPS Workshop DLDE 2021 Yiping Lu, Haoxuan Chen, Jianfeng Lu, Lexing Ying, Jose Blanchet

In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).

On the Representation of Solutions to Elliptic PDEs in Barron Spaces

no code implementations NeurIPS 2021 Ziang Chen, Jianfeng Lu, Yulong Lu

Numerical solutions to high-dimensional partial differential equations (PDEs) based on neural networks have seen exciting developments.

A Priori Generalization Error Analysis of Two-Layer Neural Networks for Solving High Dimensional Schrödinger Eigenvalue Problems

no code implementations4 May 2021 Jianfeng Lu, Yulong Lu

We prove that the convergence rate of the generalization error is independent of the dimension $d$, under the a priori assumption that the ground state lies in a spectral Barron space.

Incorporating Orientations into End-to-end Driving Model for Steering Control

no code implementations10 Mar 2021 Peng Wan, Zhenbo Song, Jianfeng Lu

In this paper, we present a novel end-to-end deep neural network model for autonomous driving that takes monocular image sequence as input, and directly generates the steering control angle.

Autonomous Driving Steering Control

A Grid-free Approach for Simulating Sweep and Cyclic Voltammetry

1 code implementation9 Feb 2021 Alec J. Coffman, Jianfeng Lu, Joseph E. Subotnik

We present a new computational approach to simulate linear sweep and cyclic voltammetry experiments that does not require a discretized grid in space to quantify diffusion.

Chemical Physics

Algebraic localization implies exponential localization in non-periodic insulators

no code implementations7 Jan 2021 Jianfeng Lu, Kevin D. Stubbs

In two and three spatial dimensions, it is well understood for periodic insulators that exponentially-localized Wannier functions exist if and only if there exists an orthonormal basis for the Fermi projection with finite second moment (i. e. all basis elements satisfy $\int |\boldsymbol{x}|^2 |w(\boldsymbol{x})|^2 \,\text{d}{\boldsymbol{x}} < \infty$).

Mathematical Physics Mesoscale and Nanoscale Physics Mathematical Physics

A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations

no code implementations5 Jan 2021 Jianfeng Lu, Yulong Lu, Min Wang

This paper concerns the a priori generalization analysis of the Deep Ritz Method (DRM) [W. E and B. Yu, 2017], a popular neural-network-based method for solving high dimensional partial differential equations.

Complexity of zigzag sampling algorithm for strongly log-concave distributions

no code implementations21 Dec 2020 Jianfeng Lu, Lihan Wang

We study the computational complexity of zigzag sampling algorithm for strongly log-concave distributions.

Neural Collapse with Cross-Entropy Loss

no code implementations15 Dec 2020 Jianfeng Lu, Stefan Steinerberger

We consider the variational problem of cross-entropy loss with $n$ feature vectors on a unit hypersphere in $\mathbb{R}^d$.

Frame

SMDS-Net: Model Guided Spectral-Spatial Network for Hyperspectral Image Denoising

no code implementations3 Dec 2020 Fengchao Xiong, Shuyin Tao, Jun Zhou, Jianfeng Lu, Jiantao Zhou, Yuntao Qian

This model first projects the observed HSIs into a low-dimensional orthogonal subspace, and then represents the projected image with a multidimensional dictionary.

Hyperspectral Image Denoising Image Denoising

Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime

no code implementations ICLR 2021 Andrea Agazzi, Jianfeng Lu

We study the problem of policy optimization for infinite-horizon discounted Markov Decision Processes with softmax policy and nonlinear function approximation trained with policy gradient algorithms.

Random Coordinate Underdamped Langevin Monte Carlo

no code implementations22 Oct 2020 Zhiyan Ding, Qin Li, Jianfeng Lu, Stephen J. Wright

We investigate the computational complexity of RC-ULMC and compare it with the classical ULMC for strongly log-concave probability distributions.

Random Coordinate Langevin Monte Carlo

no code implementations3 Oct 2020 Zhiyan Ding, Qin Li, Jianfeng Lu, Stephen J. Wright

We investigate the total complexity of RC-LMC and compare it with the classical LMC for log-concave probability distributions.

Efficient sampling from the Bingham distribution

no code implementations30 Sep 2020 Rong Ge, Holden Lee, Jianfeng Lu, Andrej Risteski

We give a algorithm for exact sampling from the Bingham distribution $p(x)\propto \exp(x^\top A x)$ on the sphere $\mathcal S^{d-1}$ with expected runtime of $\operatorname{poly}(d, \lambda_{\max}(A)-\lambda_{\min}(A))$.

Neural Machine Translation with Error Correction

1 code implementation21 Jul 2020 Kaitao Song, Xu Tan, Jianfeng Lu

Neural machine translation (NMT) generates the next target token given as input the previous ground truth target tokens during training while the previous generated target tokens during inference, which causes discrepancy between training and inference as well as error propagation, and affects the translation accuracy.

Machine Translation Translation

End-to-end Learning for Inter-Vehicle Distance and Relative Velocity Estimation in ADAS with a Monocular Camera

1 code implementation7 Jun 2020 Zhenbo Song, Jianfeng Lu, Tong Zhang, Hongdong Li

In this paper, we propose a monocular camera-based inter-vehicle distance and relative velocity estimation method based on end-to-end training of a deep neural network.

Optical Flow Estimation

LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning

no code implementations27 Apr 2020 Kaitao Song, Hao Sun, Xu Tan, Tao Qin, Jianfeng Lu, Hongzhi Liu, Tie-Yan Liu

While pre-training and fine-tuning, e. g., BERT~\citep{devlin2018bert}, GPT-2~\citep{radford2019language}, have achieved great success in language understanding and generation tasks, the pre-trained models are usually too big for online deployment in terms of both memory cost and inference speed, which hinders them from practical online usage.

Knowledge Distillation Language Modelling

MPNet: Masked and Permuted Pre-training for Language Understanding

6 code implementations NeurIPS 2020 Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu

Since BERT neglects dependency among predicted tokens, XLNet introduces permuted language modeling (PLM) for pre-training to address this problem.

Language Modelling Masked Language Modeling

A Universal Approximation Theorem of Deep Neural Networks for Expressing Probability Distributions

no code implementations NeurIPS 2020 Yulong Lu, Jianfeng Lu

In particular, the size of neural network can grow exponentially in $d$ when $1$-Wasserstein distance is used as the discrepancy, whereas for both MMD and KSD the size of neural network only depends on $d$ at most polynomially.

A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth

no code implementations11 Mar 2020 Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying

Specifically, we propose a new continuum limit of deep residual networks, which enjoys a good landscape in the sense that every local minimizer is global.

A Mean-field Analysis of Deep ResNet and Beyond:Towards Provable Optimization Via Overparameterization From Depth

no code implementations ICLR Workshop DeepDiffEq 2019 Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying

Specifically, we propose a \textbf{new continuum limit} of deep residual networks, which enjoys a good landscape in the sense that \textbf{every local minimizer is global}.

Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach

no code implementations7 Feb 2020 Jiequn Han, Jianfeng Lu, Mo Zhou

We propose a new method to solve eigenvalue problems for linear and semilinear second order differential operators in high dimensions based on deep neural networks.

Deep Network Approximation for Smooth Functions

no code implementations9 Jan 2020 Jianfeng Lu, Zuowei Shen, Haizhao Yang, Shijun Zhang

This paper establishes the (nearly) optimal approximation error characterization of deep rectified linear unit (ReLU) networks for smooth functions in terms of both width and depth simultaneously.

Part-based Multi-stream Model for Vehicle Searching

no code implementations11 Nov 2019 Ya Sun, Minxian Li, Jianfeng Lu

We can easily measure the similarity of two vehicle images by computing the Euclidean distance of the features from FC layer.

Metric Learning

Estimating Normalizing Constants for Log-Concave Distributions: Algorithms and Lower Bounds

no code implementations8 Nov 2019 Rong Ge, Holden Lee, Jianfeng Lu

Estimating the normalizing constant of an unnormalized probability distribution has important applications in computer science, statistical physics, machine learning, and statistics.

Temporal-difference learning for nonlinear value function approximation in the lazy training regime

no code implementations25 Sep 2019 Andrea Agazzi, Jianfeng Lu

We then give examples of such convergence results in the case of models that diverge if trained with non-lazy TD learning, and in the case of neural networks.

Temporal-difference learning with nonlinear function approximation: lazy training and mean field regimes

no code implementations27 May 2019 Andrea Agazzi, Jianfeng Lu

We finally give examples of our convergence results in the case of models that diverge if trained with non-lazy TD learning, and in the case of neural networks.

Accelerating Langevin Sampling with Birth-death

no code implementations23 May 2019 Yulong Lu, Jianfeng Lu, James Nolen

A fundamental problem in Bayesian inference and statistical machine learning is to efficiently sample from multimodal distributions.

Bayesian Inference

Variational training of neural network approximations of solution maps for physical models

no code implementations7 May 2019 Yingzhou Li, Jianfeng Lu, Anqi Mao

A novel solve-training framework is proposed to train neural network in representing low dimensional solution maps of physical models.

MASS: Masked Sequence to Sequence Pre-training for Language Generation

5 code implementations7 May 2019 Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, Tie-Yan Liu

Pre-training and fine-tuning, e. g., BERT, have achieved great success in language understanding by transferring knowledge from rich-resource pre-training task to the low/zero-resource downstream tasks.

Conversational Response Generation Response Generation +4

Generating Adversarial Examples With Conditional Generative Adversarial Net

no code implementations18 Mar 2019 Ping Yu, Kaitao Song, Jianfeng Lu

Recently, deep neural networks have significant progress and successful application in various fields, but they are found vulnerable to attack instances, e. g., adversarial examples.

Coordinate descent full configuration interaction

1 code implementation12 Feb 2019 Zhe Wang, Yingzhou Li, Jianfeng Lu

We develop an efficient algorithm, coordinate descent FCI (CDFCI), for the electronic structure ground state calculation in the configuration interaction framework.

Chemical Physics Computational Physics

A stochastic version of Stein Variational Gradient Descent for efficient sampling

no code implementations9 Feb 2019 Lei Li, Yingzhou Li, Jian-Guo Liu, Zibu Liu, Jianfeng Lu

We propose in this work RBM-SVGD, a stochastic version of Stein Variational Gradient Descent (SVGD) method for efficiently sampling from a given probability measure and thus useful for Bayesian inference.

Bayesian Inference

Content-Based Brain Tumor Retrieval for MR Images Using Transfer Learning

no code implementations journal 2019 Zar Nawab Khan Swati, Qinghua Zhao3, Muhammad Kabir, Farman Ali, Ali Zakir, Saeed Ahmad, Jianfeng Lu

It is necessary to design a feature extraction framework to reduce this gap without using handcrafted features by encoding/combining low-level and high-level features.

Content-Based Image Retrieval Metric Learning +2

Weakly supervised segment annotation via expectation kernel density estimation

no code implementations15 Dec 2018 Lian-Tao Wang, Qingwu Li, Jianfeng Lu

In this paper, we propose a voting scheme involving not only the definite negative instances but also the ambiguous positive instances to make use of the extra useful information in the weakly labelled positive bags.

Density Estimation

Hybrid Self-Attention Network for Machine Translation

no code implementations1 Nov 2018 Kaitao Song, Xu Tan, Furong Peng, Jianfeng Lu

The encoder-decoder is the typical framework for Neural Machine Translation (NMT), and different structures have been developed for improving the translation performance.

Machine Translation Translation

Goal-Oriented Visual Question Generation via Intermediate Rewards

no code implementations ECCV 2018 Jun-Jie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu, Anton Van Den Hengel

Despite significant progress in a variety of vision-and-language problems, developing a method capable of asking intelligent, goal-oriented questions about images is proven to be an inscrutable challenge.

Informativeness Question Generation +1

Single Image Water Hazard Detection using FCN with Reflection Attention Units

1 code implementation ECCV 2018 Xiaofeng Han, Chuong Nguyen, ShaoDi You, Jianfeng Lu

Water bodies, such as puddles and flooded areas, on and off road pose significant risks to autonomous cars.

Double Path Networks for Sequence to Sequence Learning

1 code implementation COLING 2018 Kaitao Song, Xu Tan, Di He, Jianfeng Lu, Tao Qin, Tie-Yan Liu

In this work we propose Double Path Networks for Sequence to Sequence learning (DPN-S2S), which leverage the advantages of both models by using double path information fusion.

Stochastic modified equations for the asynchronous stochastic gradient descent

no code implementations21 May 2018 Jing An, Jianfeng Lu, Lexing Ying

The resulting SME of Langevin type extracts more information about the ASGD dynamics and elucidates the relationship between different types of stochastic gradient algorithms.

Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks

1 code implementation18 May 2018 Yingzhou Li, Xiuyuan Cheng, Jianfeng Lu

Theoretical analysis of the approximation power of Butterfly-Net to the Fourier representation of input data shows that the error decays exponentially as the depth increases.

Stop memorizing: A data-dependent regularization framework for intrinsic pattern learning

no code implementations ICLR 2019 Wei Zhu, Qiang Qiu, Bao Wang, Jianfeng Lu, Guillermo Sapiro, Ingrid Daubechies

Deep neural networks (DNNs) typically have enough capacity to fit random data by brute force even when conventional data-dependent regularizations focusing on the geometry of the features are imposed.

Scaling limit of the Stein variational gradient descent: the mean field regime

no code implementations10 May 2018 Jianfeng Lu, Yulong Lu, James Nolen

We study an interacting particle system in $\mathbf{R}^d$ motivated by Stein variational gradient descent [Q. Liu and D. Wang, NIPS 2016], a deterministic algorithm for sampling from a given probability density with unknown normalization.

Solving for high dimensional committor functions using artificial neural networks

no code implementations28 Feb 2018 Yuehaw Khoo, Jianfeng Lu, Lexing Ying

In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes.

Asking the Difficult Questions: Goal-Oriented Visual Question Generation via Intermediate Rewards

no code implementations21 Nov 2017 Jun-Jie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu, Anton Van Den Hengel

Despite significant progress in a variety of vision-and-language problems, developing a method capable of asking intelligent, goal-oriented questions about images is proven to be an inscrutable challenge.

Informativeness Question Generation +1

Kill Two Birds with One Stone: Weakly-Supervised Neural Network for Image Annotation and Tag Refinement

no code implementations19 Nov 2017 Jun-Jie Zhang, Qi Wu, Jian Zhang, Chunhua Shen, Jianfeng Lu

These comments can be a description of the image, or some objects, attributes, scenes in it, which are normally used as the user-provided tags.

TAG

Solving parametric PDE problems with artificial neural networks

1 code implementation11 Jul 2017 Yuehaw Khoo, Jianfeng Lu, Lexing Ying

The representability of such quantity using a neural-network can be justified by viewing the neural-network as performing time evolution to find the solutions to the PDE.

Numerical Analysis 65Nxx

Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods

1 code implementation23 May 2017 Akihiko Nishimura, David Dunson, Jianfeng Lu

Hamiltonian Monte Carlo has emerged as a standard tool for posterior computation.

Computation

Multi-Label Image Classification with Regional Latent Semantic Dependencies

no code implementations4 Dec 2016 Jun-Jie Zhang, Qi Wu, Chunhua Shen, Jian Zhang, Jianfeng Lu

Recent state-of-the-art approaches to multi-label image classification exploit the label dependencies in an image, at global level, largely improving the labeling capacity.

Classification General Classification +1

Numerical scheme for a spatially inhomogeneous matrix-valued quantum Boltzmann equation

1 code implementation8 Aug 2014 Jianfeng Lu, Christian B. Mendl

We develop an efficient algorithm for a spatially inhomogeneous matrix-valued quantum Boltzmann equation derived from the Hubbard model.

Computational Physics Mesoscale and Nanoscale Physics

Cannot find the paper you are looking for? You can Submit a new open access paper.