Search Results for author:
Found 36 papers, 16 papers with code
SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
In clinical text summarization on the MIMIC-III dataset (Johnson et al., 2016), S2L again outperforms training on the full dataset using only 50% of the data.
A universal approximation theorem for nonlinear resistive networks
Resistor networks have recently had a surge of interest as substrates for energy-efficient self-learning machines.
An operator preconditioning perspective on training in physics-informed machine learning
In this paper, we investigate the behavior of gradient descent algorithms in physics-informed machine learning methods like PINNs, which minimize residuals connected to partial differential equations (PDEs).
How does over-squashing affect the power of GNNs?
In this paper, we provide a rigorous analysis to determine which function classes of node features can be learned by an MPNN of a given capacity.
A Structured Matrix Method for Nonequispaced Neural Operators
The computational efficiency of many neural operators, widely used for learning solutions of PDEs, relies on the fast Fourier transform (FFT) for performing spectral computations.
A Survey on Oversmoothing in Graph Neural Networks
Node features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth.
Multi-Scale Message Passing Neural PDE Solvers
We propose a novel multi-scale message passing neural network algorithm for learning the solutions of time-dependent PDEs.
Convolutional Neural Operators for robust and accurate learning of PDEs
Although very successfully used in conventional machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs.
Neural Inverse Operators for Solving PDE Inverse Problems
A large class of inverse problems for PDEs are only well-defined as mappings from operators to functions.
Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuities
A large class of hyperbolic and advection-dominated PDEs can have solutions with discontinuities.
Gradient Gating for Deep Multi-Rate Learning on Graphs
We present Gradient Gating (G$^2$), a novel framework for improving the performance of Graph Neural Networks (GNNs).
Ranked #3 on
Node Classification
on arXiv-year
wPINNs: Weak Physics informed neural networks for approximating entropy solutions of hyperbolic conservation laws
Physics informed neural networks (PINNs) require regularity of solutions of the underlying PDE to guarantee accurate approximation.
Error analysis for deep neural network approximations of parametric hyperbolic conservation laws
We derive rigorous bounds on the error resulting from the approximation of the solution of parametric hyperbolic scalar conservation laws with ReLU neural networks.
Agnostic Physics-Driven Deep Learning
This work establishes that a physical system can perform statistical learning without gradient computations, via an Agnostic Equilibrium Propagation (Aeqprop) procedure that combines energy minimization, homeostatic control, and nudging towards the correct response.
Variable-Input Deep Operator Networks
Existing architectures for operator learning require that the number and locations of sensors (where the input functions are evaluated) remain the same across all training and test samples, significantly restricting the range of their applicability.
Generic bounds on the approximation error for physics-informed (and) operator learning
We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning.
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
This large and diverse collection of tasks enables rigorous benchmarking of cross-task generalization under instructions -- training models to follow instructions on a subset of tasks and evaluating them on the remaining unseen ones.
Error estimates for physics informed neural networks approximating the Navier-Stokes equations
We prove rigorous bounds on the errors resulting from the approximation of the incompressible Navier-Stokes equations with (extended) physics informed neural networks.
Graph-Coupled Oscillator Networks
This demonstrates that the proposed framework mitigates the oversmoothing problem.
Long Expressive Memory for Sequence Modeling
We propose a novel method called Long Expressive Memory (LEM) for learning long-term sequential dependencies.
Ranked #1 on
Time Series Classification
on EigenWorms
Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings
In this work, we provide a fuzzy-set interpretation of box embeddings, and learn box representations of words using a set-theoretic training objective.
Error analysis for physics informed neural networks (PINNs) approximating Kolmogorov PDEs
Moreover, we prove that the size of the PINNs and the number of training samples only grow polynomially with the underlying dimension, enabling PINNs to overcome the curse of dimensionality in this context.
On the approximation of functions by tanh neural networks
We derive bounds on the error, in high-order Sobolev norms, incurred in the approximation of Sobolev-regular as well as analytic functions by neural networks with the hyperbolic tangent activation function.
UnICORNN: A recurrent model for learning very long time dependencies
The design of recurrent neural networks (RNNs) to accurately process sequential inputs with long-time dependencies is very challenging on account of the exploding and vanishing gradient problem.
Ranked #2 on
Time Series Classification
on EigenWorms
Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies
Circuits of biological neurons, such as in the functional parts of the brain can be modeled as networks of coupled oscillators.
Physics Informed Neural Networks for Simulating Radiative Transfer
We propose a novel machine learning algorithm for simulating radiative transfer.
Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks
We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems.
Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
Physics informed neural networks (PINNs) have recently been widely used for robust and accurate approximation of PDEs.
Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs
Physics informed neural networks (PINNs) have recently been very successfully applied for efficiently approximating inverse problems for PDEs.
Enhancing accuracy of deep learning algorithms by training with low-discrepancy sequences
We propose a deep supervised learning algorithm based on low-discrepancy sequences as the training set.
On the approximation of rough functions with deep neural networks
Deep neural networks and the ENO procedure are both efficient frameworks for approximating rough functions.
A Multi-level procedure for enhancing accuracy of machine learning algorithms
We propose a multi-level method to increase the accuracy of machine learning algorithms for approximating observables in scientific computing, particularly those that arise in systems modeled by differential equations.
Statistical solutions of hyperbolic systems of conservation laws: numerical approximation
Statistical solutions are time-parameterized probability measures on spaces of integrable functions, that have been proposed recently as a framework for global solutions and uncertainty quantification for multi-dimensional hyperbolic system of conservation laws.
Numerical Analysis Fluid Dynamics 35L65, 65M08, 65C05, 65C30
Deep learning observables in computational fluid dynamics
Under the assumption that the underlying neural networks generalize well, we prove that the deep learning MC and QMC algorithms are guaranteed to be faster than the baseline (quasi-) Monte Carlo methods.
A machine learning framework for data driven acceleration of computations of differential equations
We propose a machine learning framework to accelerate numerical computations of time-dependent ODEs and PDEs.
Numerical approximation of statistical solutions of scalar conservation laws
We propose efficient numerical algorithms for approximating statistical solutions of scalar conservation laws.
Numerical Analysis 35L65, 65M08, 65C05, 65C30
