Search Results for author:
Found 16 papers, 7 papers with code
Equivalence of Equilibrium Propagation and Recurrent Backpropagation
Recurrent Backpropagation and Equilibrium Propagation are supervised learning algorithms for fixed point recurrent neural networks which differ in their second phase.
Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation
Because the objective function is defined in terms of local perturbations, the second phase of Equilibrium Propagation corresponds to only nudging the prediction (fixed point, or stationary distribution) towards a configuration that reduces prediction error.
Generalization of Equilibrium Propagation to Vector Field Dynamics
The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists.
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
In this work, we show that a bias in the gradient estimate of EP, inherent in the use of finite nudging, is responsible for this phenomenon and that cancelling it allows training deep ConvNets by EP.
Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input
Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i. e. RNNs that are fed by a static input x and settle to a steady state.
Energy-based learning algorithms for analog computing: a comparative study
Additionally, we establish new SOTA results with DCHNs on all five datasets, both in performance and speed.
A Fast Algorithm to Simulate Nonlinear Resistive Networks
In the quest for energy-efficient artificial intelligence systems, resistor networks are attracting interest as an alternative to conventional GPU-based neural networks.
Feedforward Initialization for Fast Inference of Deep Generative Networks is biologically plausible
We find conditions under which a simple feedforward computation is a very good initialization for inference, after the input units are clamped to observed values.
Extending the Framework of Equilibrium Propagation to General Dynamics
The biological plausibility of the backpropagation algorithm has long been doubted by neuroscientists.
Equilibrium Propagation with Continual Weight Updates
However, in existing implementations of EP, the learning rule is not local in time: the weight update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.
Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation
On the other hand, the biological plausibility of EP is limited by the fact that its learning rule is not local in time: the synapse update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.
Training End-to-End Analog Neural Networks with Equilibrium Propagation
We introduce a principled method to train end-to-end analog neural networks by stochastic gradient descent.
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
Equilibrium Propagation (EP) is a biologically-inspired counterpart of Backpropagation Through Time (BPTT) which, owing to its strong theoretical guarantees and the locality in space of its learning rule, fosters the design of energy-efficient hardware dedicated to learning.
A deep learning theory for neural networks grounded in physics
Traditionally in deep learning, neural networks are differentiable mathematical functions, and the loss gradients required for SGD are computed with the backpropagation algorithm.
Agnostic Physics-Driven Deep Learning
This work establishes that a physical system can perform statistical learning without gradient computations, via an Agnostic Equilibrium Propagation (Aeqprop) procedure that combines energy minimization, homeostatic control, and nudging towards the correct response.
A universal approximation theorem for nonlinear resistive networks
Resistor networks have recently had a surge of interest as substrates for energy-efficient self-learning machines.
