no code implementations • 20 Nov 2012 • Oleksandr Manzyuk, Barak A. Pearlmutter, Alexey Andreyevich Radul, David R. Rush, Jeffrey Mark Siskind
The essence of Forward AD is to attach perturbations to each number, and propagate these through the computation.
Symbolic Computation Mathematical Software Differential Geometry
1 code implementation • 15 Jan 2013 • Vamsi K. Potluru, Sergey M. Plis, Jonathan Le Roux, Barak A. Pearlmutter, Vince D. Calhoun, Thomas P. Hayes
However, present algorithms designed for optimizing the mixed norm L$_1$/L$_2$ are slow and other formulations for sparse NMF have been proposed such as those based on L$_1$ and L$_0$ norms.
no code implementations • 28 Apr 2014 • Atilim Gunes Baydin, Barak A. Pearlmutter
Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age.
4 code implementations • 20 Feb 2015 • Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Jeffrey Mark Siskind
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning.
no code implementations • 10 Nov 2016 • Atılım Güneş Baydin, Barak A. Pearlmutter, Jeffrey Mark Siskind
DiffSharp is an algorithmic differentiation or automatic differentiation (AD) library for the . NET ecosystem, which is targeted by the C# and F# languages, among others.
no code implementations • 10 Nov 2016 • Jeffrey Mark Siskind, Barak A. Pearlmutter
Heretofore, automatic checkpointing at procedure-call boundaries, to reduce the space complexity of reverse mode, has been provided by systems like Tapenade.
no code implementations • 10 Nov 2016 • Atılım Güneş Baydin, Barak A. Pearlmutter, Jeffrey Mark Siskind
The deep learning community has devised a diverse set of methods to make gradient optimization, using large datasets, of large and highly complex models with deeply cascaded nonlinearities, practical.
2 code implementations • 12 Sep 2018 • Robert Kelly, Barak A. Pearlmutter, Phil Maguire
In this paper we examine the issues involved in adding concurrency to the Robin Hood hash table algorithm.
Distributed, Parallel, and Cluster Computing
no code implementations • 20 May 2020 • Mansura Habiba, Barak A. Pearlmutter
(ii)~can Neural ODEs solve the irregular sampling rate challenge of existing neural network models for a continuous time series, i. e., length and dynamic nature, (iii)~how to reduce the training and evaluation time of existing Neural ODE systems?
no code implementations • 20 May 2020 • Mansura Habiba, Barak A. Pearlmutter
Practical applications, e. g., sensor data, healthcare, weather, generates data that is in truth continuous in time, and informative missingness is a common phenomenon in these datasets.
1 code implementation • 13 May 2021 • Mehrdad Maleki, Mansura Habiba, Barak A. Pearlmutter
There is an analogy between the ResNet (Residual Network) architecture for deep neural networks and an Euler solver for an ODE.
no code implementations • 30 Oct 2021 • Mansura Habiba, Barak A. Pearlmutter
Recent work in deep learning focuses on solving physical systems in the Ordinary Differential Equation or Partial Differential Equation.
no code implementations • 30 Oct 2021 • Mansura Habiba, Barak A. Pearlmutter
In a typical case, where the ``wormhole'' connections are inactive, this is inexpensive; but when they are active, the network takes a longer time to settle down, and the gradient calculation is also more laborious, with an effect similar to making the network deeper.
no code implementations • 30 Oct 2021 • Mansura Habiba, Eoin Brophy, Barak A. Pearlmutter, Tomas Ward
Continuous medical time series data such as ECG is one of the most complex time series due to its dynamic and high dimensional characteristics.
2 code implementations • 17 Feb 2022 • Atılım Güneş Baydin, Barak A. Pearlmutter, Don Syme, Frank Wood, Philip Torr
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning.
1 code implementation • 16 Nov 2023 • Thomas Flinkow, Barak A. Pearlmutter, Rosemary Monahan
Extensive research on formal verification of machine learning (ML) systems indicates that learning from data alone often fails to capture underlying background knowledge.
no code implementations • 9 Feb 2024 • Bradley T. Baker, Barak A. Pearlmutter, Robyn Miller, Vince D. Calhoun, Sergey M. Plis
Our understanding of learning dynamics of deep neural networks (DNNs) remains incomplete.