Search Results for author:
Found 49 papers, 16 papers with code
Embedding physics domain knowledge into a Bayesian network enables layer-by-layer process innovation for photovoltaics
Process optimization of photovoltaic devices is a time-intensive, trial-and-error endeavor, which lacks full transparency of the underlying physics and relies on user-imposed constraints that may or may not lead to a global optimum.
An invertible crystallographic representation for general inverse design of inorganic crystals with targeted properties
Realizing general inverse design could greatly accelerate the discovery of new materials with user-defined properties.
An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks
Deep learning is formulated as a discrete-time optimal control problem.
Towards Robust Neural Networks via Close-loop Control
We connect the robustness of neural networks with optimal control using the geometrical information of underlying data to design the control objective.
Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)
Needle-in-a-Haystack problems exist across a wide range of applications including rare disease prediction, ecological resource management, fraud detection, and material property optimization.
Inverse Approximation Theory for Nonlinear Recurrent Neural Networks
We prove an inverse approximation theorem for the approximation of nonlinear sequence-to-sequence relationships using recurrent neural networks (RNNs).
StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization
In this paper, we investigate the long-term memory learning capabilities of state-space models (SSMs) from the perspective of parameterization.
OnsagerNet: Learning Stable and Interpretable Dynamics using a Generalized Onsager Principle
We further apply this method to study Rayleigh-Benard convection and learn Lorenz-like low dimensional autonomous reduced order models that capture both qualitative and quantitative properties of the underlying dynamics.
Self-Healing Robust Neural Networks via Closed-Loop Control
While numerous attack and defense techniques have been developed, this work investigates the robustness issue from a new angle: can we design a self-healing neural network that can automatically detect and fix the vulnerability issue by itself?
Constructing Custom Thermodynamics Using Deep Learning
Specifically, we learn three interpretable thermodynamic coordinates and build a dynamical landscape of polymer stretching, including the identification of stable and transition states and the control of the stretching rate.
Personalized Algorithm Generation: A Case Study in Learning ODE Integrators
As a case study, we develop a machine learning approach that automatically learns effective solvers for initial value problems in the form of ordinary differential equations (ODEs), based on the Runge-Kutta (RK) integrator architecture.
Adversarial Invariant Learning
To the best of our knowledge, this is one of the first to adopt differentiable environment splitting method to enable stable predictions across environments without environment index information, which achieves the state-of-the-art performance on datasets with strong spurious correlation, such as Colored MNIST.
Maximum Principle Based Algorithms for Deep Learning
The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms.
Stochastic modified equations and adaptive stochastic gradient algorithms
We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential equations.
A Mean-Field Optimal Control Formulation of Deep Learning
This paper introduces the mathematical formulation of the population risk minimization problem in deep learning as a mean-field optimal control problem.
A Quantitative Analysis of the Effect of Batch Normalization on Gradient Descent
Despite its empirical success and recent theoretical progress, there generally lacks a quantitative analysis of the effect of batch normalization (BN) on the convergence and stability of gradient descent.
Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations
We develop the mathematical foundations of the stochastic modified equations (SME) framework for analyzing the dynamics of stochastic gradient algorithms, where the latter is approximated by a class of stochastic differential equations with small noise parameters.
On the Convergence and Robustness of Batch Normalization
Despite its empirical success, the theoretical underpinnings of the stability, convergence and acceleration properties of batch normalization (BN) remain elusive.
Computing committor functions for the study of rare events using deep learning with importance sampling
The committor function is a central object of study in understanding transitions between metastable states in complex systems.
Computing Committor Functions for the Study of Rare Events Using Deep Learning
The committor function is a central object of study in understanding transitions between metastable states in complex systems.
Distributed Optimization for Over-Parameterized Learning
Moreover, we show that the more local updating can reduce the overall communication, even for an infinity number of steps where each node is free to update its local model to near-optimality before exchanging information.
Deep Learning via Dynamical Systems: An Approximation Perspective
We build on the dynamical systems approach to deep learning, where deep residual networks are idealized as continuous-time dynamical systems, from the approximation perspective.
Collaborative Inference for Efficient Remote Monitoring
While current machine learning models have impressive performance over a wide range of applications, their large size and complexity render them unsuitable for tasks such as remote monitoring on edge devices with limited storage and computational power.
Optimization in Machine Learning: A Distribution Space Approach
We present the viewpoint that optimization problems encountered in machine learning can often be interpreted as minimizing a convex functional over a function space, but with a non-convex constraint set introduced by model parameterization.
On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization Analysis
We study the approximation properties and optimization dynamics of recurrent neural networks (RNNs) when applied to learn input-output relationships in temporal data.
Optimising Stochastic Routing for Taxi Fleets with Model Enhanced Reinforcement Learning
The future of mobility-as-a-Service (Maas)should embrace an integrated system of ride-hailing, street-hailing and ride-sharing with optimised intelligent vehicle routing in response to a real-time, stochastic demand pattern.
Amata: An Annealing Mechanism for Adversarial Training Acceleration
However, conducting adversarial training brings much computational overhead compared with standard training.
A Data Driven Method for Computing Quasipotentials
The quasipotential is a natural generalization of the concept of energy functions to non-equilibrium systems.
QROSS: QUBO Relaxation Parameter Optimisation via Learning Solver Surrogates
In this way, we are able capture the common structure of the instances and their interactions with the solver, and produce good choices of penalty parameters with fewer number of calls to the QUBO solver.
Approximation Theory of Convolutional Architectures for Time Series Modelling
We study the approximation properties of convolutional architectures applied to time series modelling, which can be formulated mathematically as a functional approximation problem.
On the approximation properties of recurrent encoder-decoder architectures
Our results provide the theoretical understanding of approximation properties of the recurrent encoder-decoder architecture, which characterises, in the considered setting, the types of temporal relationships that can be efficiently learned.
Gradient-based Meta-solving and Its Applications to Iterative Methods for Solving Differential Equations
In science and engineering applications, it is often required to solve similar computational problems repeatedly.
Short optimization paths lead to good generalization
Through our approach, we show that, with a proper initialization, gradient flow converges following a short path with an explicit length estimate.
Unraveling Model-Agnostic Meta-Learning via The Adaptation Learning Rate
In this paper, we study the effect of the adaptation learning rate in meta-learning with mixed linear regression.
Computing the Invariant Distribution of Randomly Perturbed Dynamical Systems Using Deep Learning
The potential component of the decomposition gives the generalized potential.
From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality
Optimization and generalization are two essential aspects of statistical machine learning.
Principled Acceleration of Iterative Numerical Methods Using Machine Learning
Iterative methods are ubiquitous in large-scale scientific computing applications, and a number of approaches based on meta-learning have been recently proposed to accelerate them.
Tackling Data Scarcity with Transfer Learning: A Case Study of Thickness Characterization from Optical Spectra of Perovskite Thin Films
Transfer learning increasingly becomes an important tool in handling data scarcity often encountered in machine learning.
Deep Neural Network Approximation of Invariant Functions through Dynamical Systems
We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems.
A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms
Meta-learning of numerical algorithms for a given task consists of the data-driven identification and adaptation of an algorithmic structure and the associated hyperparameters.
On the Universal Approximation Property of Deep Fully Convolutional Neural Networks
We study the approximation of shift-invariant or equivariant functions by deep fully convolutional networks from the dynamical systems perspective.
A Brief Survey on the Approximation Theory for Sequence Modelling
We survey current developments in the approximation theory of sequence modelling in machine learning.
Approximation Rate of the Transformer Architecture for Sequence Modeling
The Transformer architecture is widely applied in sequence modeling applications, yet the theoretical understanding of its working principles remains limited.
Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks
We present a theoretical analysis of the approximation properties of convolutional architectures when applied to the modeling of temporal sequences.
Interpolation, Approximation and Controllability of Deep Neural Networks
We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory.
Asymptotically Fair Participation in Machine Learning Models: an Optimal Control Perspective
Moreover, we design a surrogate retention system based on existing literature on evolutionary population dynamics to approximate the dynamics of distribution shifts on active user counts, from which the objective of achieving asymptotically fair participation is formulated as an optimal control problem, and the control variables are considered as the model parameters.
Mitigating distribution shift in machine learning-augmented hybrid simulation
We study the problem of distribution shift generally arising in machine-learning augmented hybrid simulation, where parts of simulation algorithms are replaced by data-driven surrogates.
DynGMA: a robust approach for learning stochastic differential equations from data
Benefiting from the robust density approximation, our method exhibits superior accuracy compared to baseline methods in learning the fully unknown drift and diffusion functions and computing the invariant distribution from trajectory data.
PID Control-Based Self-Healing to Improve the Robustness of Large Language Models
This approach reduces the computational cost to that of using just the P controller, instead of the full PID control.
