no code implementations • ICML 2020 • Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying
Specifically, we propose a \textbf{new continuum limit} of deep residual networks, which enjoys a good landscape in the sense that \textbf{every local minimizer is global}.
no code implementations • 29 Apr 2024 • Kaizhao Liu, Jose Blanchet, Lexing Ying, Yiping Lu
Bootstrap is a popular methodology for simulating input uncertainty.
no code implementations • 8 Mar 2024 • Xun Tang, Holakou Rahmanian, Michael Shavlovsky, Kiran Koshy Thekumparampil, Tesi Xiao, Lexing Ying
We derive the corresponding entropy regularization formulation and introduce a Sinkhorn-type algorithm for such constrained OT problems supported by theoretical guarantees.
no code implementations • 27 Feb 2024 • Lexing Ying
This note considers the multidimensional unstructured sparse recovery problems.
no code implementations • 12 Feb 2024 • Hongrui Chen, Lexing Ying
Diffusion models have achieved huge empirical success in data generation tasks.
no code implementations • 28 Jan 2024 • Haoxuan Chen, Lexing Ying
We discuss how the proposed algorithm can be implemented and derive a partial differential equation governing the evolution of the ensemble under the continuous time and mean-field limit.
no code implementations • 21 Jan 2024 • Yinuo Ren, Chao Ma, Lexing Ying
Why do neural networks trained with large learning rates for a longer time often lead to better generalization?
no code implementations • 20 Jan 2024 • Xun Tang, Michael Shavlovsky, Holakou Rahmanian, Elisa Tardini, Kiran Koshy Thekumparampil, Tesi Xiao, Lexing Ying
To achieve possibly super-exponential convergence, we present Sinkhorn-Newton-Sparse (SNS), an extension to the Sinkhorn algorithm, by introducing early stopping for the matrix scaling steps and a second stage featuring a Newton-type subroutine.
no code implementations • 10 Dec 2023 • Yinuo Ren, Yiping Lu, Lexing Ying, Grant M. Rotskoff
Inferring a diffusion equation from discretely-observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments.
no code implementations • 28 Nov 2023 • Lexing Ying
This note considers the unstructured sparse recovery problems in a general form.
no code implementations • 22 Nov 2023 • Yinuo Ren, Tesi Xiao, Tanmay Gangwani, Anshuka Rangi, Holakou Rahmanian, Lexing Ying, Subhajit Sanyal
Multi-objective optimization (MOO) aims to optimize multiple, possibly conflicting objectives with widespread applications.
no code implementations • 25 May 2023 • Jose Blanchet, Haoxuan Chen, Yiping Lu, Lexing Ying
We demonstrate that this kind of quadrature rule can improve the Monte Carlo rate and achieve the minimax optimal rate under a sufficient smoothness assumption.
no code implementations • 1 Dec 2022 • Yinuo Ren, Hongli Zhao, Yuehaw Khoo, Lexing Ying
We propose the tensorizing flow method for estimating high-dimensional probability density functions from the observed data.
no code implementations • 28 Nov 2022 • Yiping Lu, Jiajin Li, Lexing Ying, Jose Blanchet
The optimal design of experiments typically involves solving an NP-hard combinatorial optimization problem.
no code implementations • 14 Oct 2022 • Yuhua Zhu, Zachary Izzo, Lexing Ying
The optimal policy for the limiting HJB equation can be explicitly obtained for several common bandit problems, and we give numerical methods to solve the HJB equation when an explicit solution is not available.
no code implementations • 13 Oct 2022 • Chao Ma, Lexing Ying
The knowledge consists of a set of vectors in the same embedding space as the input sequence, containing the information of the language used to process the input sequence.
no code implementations • 28 Sep 2022 • Jikai Jin, Yiping Lu, Jose Blanchet, Lexing Ying
Learning mappings between infinite-dimensional function spaces has achieved empirical success in many disciplines of machine learning, including generative modeling, functional data analysis, causal inference, and multi-agent reinforcement learning.
no code implementations • 19 Sep 2022 • Yiping Lu, Wenlong Ji, Zachary Izzo, Lexing Ying
In this paper, we propose importance tempering to improve the decision boundary and achieve consistently better results for overparameterized models.
no code implementations • 3 Sep 2022 • Xun Tang, YoonHaeng Hur, Yuehaw Khoo, Lexing Ying
In this paper, we present a density estimation framework based on tree tensor-network states.
no code implementations • 3 Aug 2022 • Samarth Gupta, Daniel N. Hill, Lexing Ying, Inderjit Dhillon
Due to noise, the policy learnedfrom the estimated model is often far from the optimal policy of the underlying model.
no code implementations • 15 May 2022 • Yiping Lu, Jose Blanchet, Lexing Ying
In this paper, we study the statistical limits in terms of Sobolev norms of gradient descent for solving inverse problem from randomly sampled noisy observations using a general class of objective functions.
no code implementations • 24 Apr 2022 • Chao Ma, Daniel Kunin, Lei Wu, Lexing Ying
Numerically, we observe that neural network loss functions possesses a multiscale structure, manifested in two ways: (1) in a neighborhood of minima, the loss mixes a continuum of scales and grows subquadratically, and (2) in a larger region, the loss shows several separate scales clearly.
no code implementations • 30 Mar 2022 • Jiahao Yao, Haoya Li, Marin Bukov, Lin Lin, Lexing Ying
Variational quantum algorithms stand at the forefront of simulations on near-term and future fault-tolerant quantum devices.
no code implementations • 21 Feb 2022 • Haoya Li, Hsiang-Fu Yu, Lexing Ying, Inderjit Dhillon
Entropy regularized Markov decision processes have been widely used in reinforcement learning.
no code implementations • ICLR 2022 • Chao Ma, Lexing Ying
In this paper, we study the problem of finding mixed Nash equilibrium for mean-field two-player zero-sum games.
no code implementations • 13 Dec 2021 • Zachary Izzo, James Zou, Lexing Ying
A recent line of work has focused on training machine learning (ML) models in the performative setting, i. e. when the data distribution reacts to the deployed model.
no code implementations • 25 Oct 2021 • Xun Tang, Lexing Ying, Yuhua Zhu
When the error is in the residual norm, we prove that the shifting factor is always positive and upper bounded by $1+O\left(1/n\right)$, where $n$ is the number of samples used in learning each row of the transition matrix.
Model-based Reinforcement Learning reinforcement-learning +1
no code implementations • 17 Oct 2021 • Chao Ma, Lexing Ying
Later, the infinite-width limit of the two-layer neural networks with BN is considered, and a mean-field formulation is derived for the training dynamics.
no code implementations • ICLR 2022 • Yiping Lu, Haoxuan Chen, Jianfeng Lu, Lexing Ying, Jose Blanchet
In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).
no code implementations • 5 Oct 2021 • Haoya Li, Samarth Gupta, HsiangFu Yu, Lexing Ying, Inderjit Dhillon
This paper proposes an approximate Newton method for the policy gradient algorithm with entropy regularization.
no code implementations • NeurIPS Workshop DLDE 2021 • Yiping Lu, Haoxuan Chen, Jianfeng Lu, Lexing Ying, Jose Blanchet
In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations (PDEs) from random samples using the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).
no code implementations • 3 Aug 2021 • Yuhua Zhu, Lexing Ying
The objective function of the variational formulation consists of two parts: one for maximizing the value function and the other for minimizing the Bellman residual.
1 code implementation • 4 Jun 2021 • Philip A. Etter, Kai Zhong, Hsiang-Fu Yu, Lexing Ying, Inderjit Dhillon
In industrial applications, these models operate at extreme scales, where every bit of performance is critical.
no code implementations • 31 May 2021 • Jing An, Lexing Ying
When the loss function is a sum of multiple terms, a popular method is the stochastic gradient descent.
1 code implementation • NeurIPS 2021 • Chao Ma, Lexing Ying
The multiplicative structure of parameters and input data in the first layer of neural networks is explored to build connection between the landscape of the loss function with respect to parameters and the landscape of the model function with respect to input data.
no code implementations • 7 May 2021 • Haoya Li, Lexing Ying
In this paper, we propose a semigroup method for solving high-dimensional elliptic partial differential equations (PDEs) and the associated eigenvalue problems based on neural networks.
1 code implementation • 15 Feb 2021 • Rajat Sen, Alexander Rakhlin, Lexing Ying, Rahul Kidambi, Dean Foster, Daniel Hill, Inderjit Dhillon
We show that our algorithm has a regret guarantee of $O(k\sqrt{(A-k+1)T \log (|\mathcal{F}|T)})$, where $A$ is the total number of arms and $\mathcal{F}$ is the class containing the regression function, while only requiring $\tilde{O}(A)$ computation per time step.
Computational Efficiency Extreme Multi-Label Classification +2
1 code implementation • 15 Feb 2021 • Zachary Izzo, Lexing Ying, James Zou
Performative distribution shift captures the setting where the choice of which ML model is deployed changes the data distribution.
no code implementations • 18 Jan 2021 • Lukas Einkemmer, Jingwei Hu, Lexing Ying
In this paper, we propose an efficient dynamical low-rank integrator that can capture the fluid limit -- the Navier-Stokes equations -- of the Boltzmann-BGK model even in the compressible regime.
Numerical Analysis Numerical Analysis Computational Physics
no code implementations • 17 Dec 2020 • Lexing Ying, Yuhua Zhu
This note summarizes the optimization formulations used in the study of Markov decision processes.
Optimization and Control
no code implementations • 14 Dec 2020 • Chao Ma, Lexing Ying
A new understanding of adversarial examples and adversarial robustness is proposed by decoupling the data generator and the label generator (which we call the teacher).
no code implementations • 12 Dec 2020 • Haoya Li, Yuehaw Khoo, Yinuo Ren, Lexing Ying
This paper proposes a new method based on neural networks for computing the high-dimensional committor functions that satisfy Fokker-Planck equations.
1 code implementation • 11 Oct 2020 • Yifan Peng, Lin Lin, Lexing Ying, Leonardo Zepeda-Núñez
We showcase this framework by introducing a neural network architecture that combines LRC-layers with short-range convolutional layers to accurately learn the energy and force associated with a $N$-body potential.
no code implementations • ICLR 2021 • Jing An, Lexing Ying, Yuhua Zhu
We consider two commonly-used techniques, resampling and reweighting, that rebalance the proportions of the subgroups to maintain the desired objective function.
1 code implementation • 28 Aug 2020 • Yingzhou Li, Jack Poulson, Lexing Ying
We introduce a data distribution scheme for $\mathcal{H}$-matrices and a distributed-memory algorithm for $\mathcal{H}$-matrix-vector multiplication.
Numerical Analysis Distributed, Parallel, and Cluster Computing Numerical Analysis 65F99, 65Y05
no code implementations • 29 Jun 2020 • Lexing Ying
Natural gradients have been widely used in optimization of loss functionals over probability space, with important examples such as Fisher-Rao gradient descent for Kullback-Leibler divergence, Wasserstein gradient descent for transport-related functionals, and Mahalanobis gradient descent for quadratic loss functionals.
no code implementations • 11 Jun 2020 • Yuhua Zhu, Zach Izzo, Lexing Ying
The main idea is to borrow extra randomness from the future to approximately re-sample the next state when the underlying dynamics of the problem are sufficiently smooth.
no code implementations • 8 Apr 2020 • Lexing Ying
This note considers the problem of minimizing interacting free energy.
no code implementations • 11 Mar 2020 • Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying
Specifically, we propose a new continuum limit of deep residual networks, which enjoys a good landscape in the sense that every local minimizer is global.
no code implementations • ICLR Workshop DeepDiffEq 2019 • Yiping Lu, Chao Ma, Yulong Lu, Jianfeng Lu, Lexing Ying
Specifically, we propose a \textbf{new continuum limit} of deep residual networks, which enjoys a good landscape in the sense that \textbf{every local minimizer is global}.
no code implementations • 27 Nov 2019 • Yuwei Fan, Lexing Ying
This paper proposes a neural network approach for solving two classical problems in the two-dimensional inverse wave scattering: far field pattern problem and seismic imaging.
no code implementations • 25 Nov 2019 • Yuwei Fan, Lexing Ying
This paper introduces a neural network approach for solving two-dimensional traveltime tomography (TT) problems based on the eikonal equation.
no code implementations • 10 Oct 2019 • Yuwei Fan, Lexing Ying
Both the forward map from the optical properties to the albedo operator and the inverse map are high-dimensional and nonlinear.
no code implementations • 25 Sep 2019 • Lexing Ying, Yuandong Tian
For the two-layer networks, we derive the necessary condition of the stationary distributions of the mean field equation and explain an empirical phenomenon concerning training speed differences using the Wasserstein flow description.
no code implementations • 25 Sep 2019 • Xian Wu, Yuandong Tian, Lexing Ying
We apply our theoretical framework to different models for the noise distribution of the policy and value network as well as the distribution of rewards, and show that for these general models, the sample complexity is polynomial in D, where D is the depth of the search tree.
no code implementations • 16 Jun 2019 • Jordi Feliu-Faba, Yuwei Fan, Lexing Ying
This paper introduces a meta-learning approach for parameterized pseudo-differential operators with deep neural networks.
no code implementations • 6 Jun 2019 • Yuwei Fan, Lexing Ying
Both the forward map from the electrical conductivity to the DtN map and the inverse map are high-dimensional and nonlinear.
no code implementations • 20 Oct 2018 • Yuwei Fan, Cindy Orozco Bohorquez, Lexing Ying
This paper proposes a novel neural network architecture inspired by the nonstandard form proposed by Beylkin, Coifman, and Rokhlin in [Communications on Pure and Applied Mathematics, 44(2), 141-183].
1 code implementation • 4 Aug 2018 • Yuwei Fan, Jordi Feliu-Faba, Lin Lin, Lexing Ying, Leonardo Zepeda-Nunez
In recent years, deep learning has led to impressive results in many fields.
Numerical Analysis
1 code implementation • 5 Jul 2018 • Yuwei Fan, Lin Lin, Lexing Ying, Leonardo Zepeda-Nunez
This network generalizes the latter to the nonlinear case by introducing a local deep neural network at each spatial scale.
Numerical Analysis
no code implementations • 21 May 2018 • Jing An, Jianfeng Lu, Lexing Ying
The resulting SME of Langevin type extracts more information about the ASGD dynamics and elucidates the relationship between different types of stochastic gradient algorithms.
no code implementations • 28 Feb 2018 • Yuehaw Khoo, Jianfeng Lu, Lexing Ying
In this note we propose a method based on artificial neural network to study the transition between states governed by stochastic processes.
1 code implementation • 11 Jul 2017 • Yuehaw Khoo, Jianfeng Lu, Lexing Ying
The representability of such quantity using a neural-network can be justified by viewing the neural-network as performing time evolution to find the solutions to the PDE.
Numerical Analysis 65Nxx
1 code implementation • 27 Sep 2016 • Anil Damle, Victor Minden, Lexing Ying
We present a new algorithm for spectral clustering based on a column-pivoted QR factorization that may be directly used for cluster assignment or to provide an initial guess for k-means.
Numerical Analysis Numerical Analysis Social and Information Networks Physics and Society 68W01, 65F99
3 code implementations • 26 Sep 2016 • Victor Minden, Kenneth L. Ho, Anil Damle, Lexing Ying
We introduce the strong recursive skeletonization factorization (RS-S), a new approximate matrix factorization based on recursive skeletonization for solving discretizations of linear integral equations associated with elliptic partial differential equations in two and three dimensions (and other matrices with similar hierarchical rank structure).
Numerical Analysis 65R20 (primary), 65F08, 65F05 (secondary)