Search Results for author:
Found 17 papers, 6 papers with code
Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse
Our understanding of learning dynamics of deep neural networks (DNNs) remains incomplete.
Comparing Differentiable Logics for Learning Systems: A Research Preview
Extensive research on formal verification of machine learning (ML) systems indicates that learning from data alone often fails to capture underlying background knowledge.
Gradients without Backpropagation
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning.
Neural Network based on Automatic Differentiation Transformation of Numeric Iterate-to-Fixedpoint
In a typical case, where the ``wormhole'' connections are inactive, this is inexpensive; but when they are active, the network takes a longer time to settle down, and the gradient calculation is also more laborious, with an effect similar to making the network deeper.
Continuous Convolutional Neural Networks: Coupled Neural PDE and ODE
Recent work in deep learning focuses on solving physical systems in the Ordinary Differential Equation or Partial Differential Equation.
ECG synthesis with Neural ODE and GAN models
Continuous medical time series data such as ECG is one of the most complex time series due to its dynamic and high dimensional characteristics.
HeunNet: Extending ResNet using Heun's Methods
There is an analogy between the ResNet (Residual Network) architecture for deep neural networks and an Euler solver for an ODE.
Neural Ordinary Differential Equation based Recurrent Neural Network Model
(ii)~can Neural ODEs solve the irregular sampling rate challenge of existing neural network models for a continuous time series, i. e., length and dynamic nature, (iii)~how to reduce the training and evaluation time of existing Neural ODE systems?
Neural ODEs for Informative Missingness in Multivariate Time Series
Practical applications, e. g., sensor data, healthcare, weather, generates data that is in truth continuous in time, and informative missingness is a common phenomenon in these datasets.
Concurrent Robin Hood Hashing
In this paper we examine the issues involved in adding concurrency to the Robin Hood hash table algorithm.
Distributed, Parallel, and Cluster Computing
Binomial Checkpointing for Arbitrary Programs with No User Annotation
Heretofore, automatic checkpointing at procedure-call boundaries, to reduce the space complexity of reverse mode, has been provided by systems like Tapenade.
DiffSharp: An AD Library for .NET Languages
DiffSharp is an algorithmic differentiation or automatic differentiation (AD) library for the . NET ecosystem, which is targeted by the C# and F# languages, among others.
Tricks from Deep Learning
The deep learning community has devised a diverse set of methods to make gradient optimization, using large datasets, of large and highly complex models with deeply cascaded nonlinearities, practical.
Automatic differentiation in machine learning: a survey
Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in machine learning.
Automatic Differentiation of Algorithms for Machine Learning
Automatic differentiation---the mechanical transformation of numeric computer programs to calculate derivatives efficiently and accurately---dates to the origin of the computer age.
Block Coordinate Descent for Sparse NMF
However, present algorithms designed for optimizing the mixed norm L$_1$/L$_2$ are slow and other formulations for sparse NMF have been proposed such as those based on L$_1$ and L$_0$ norms.
Confusion of Tagged Perturbations in Forward Automatic Differentiation of Higher-Order Functions
The essence of Forward AD is to attach perturbations to each number, and propagate these through the computation.
Symbolic Computation Mathematical Software Differential Geometry
