Search Results for author:
Found 21 papers, 7 papers with code
Unsupervised End-to-End Training with a Self-Defined Bio-Inspired Target
Current unsupervised learning methods depend on end-to-end training via deep learning techniques such as self-supervised learning, with high computational requirements, or employ layer-by-layer training using bio-inspired approaches like Hebbian learning, using local learning rules incompatible with supervised learning.
Bayesian Metaplasticity from Synaptic Uncertainty
Catastrophic forgetting remains a challenge for neural networks, especially in lifelong learning scenarios.
Synaptic metaplasticity with multi-level memristive devices
In this work, we extend this solution to quantized neural networks (QNNs) and present a memristor-based hardware solution for implementing metaplasticity during both inference and training.
Voltage-Dependent Synaptic Plasticity (VDSP): Unsupervised probabilistic Hebbian plasticity rule based on neurons membrane potential
To validate the system-level performance of VDSP, we train a single-layer spiking neural network (SNN) for the recognition of handwritten digits.
Energy Efficient Learning with Low Resolution Stochastic Domain Wall Synapse Based Deep Neural Networks
We demonstrate that extremely low resolution quantized (nominally 5-state) synapses with large stochastic variations in Domain Wall (DW) position can be both energy efficient and achieve reasonably high testing accuracies compared to Deep Neural Networks (DNNs) of similar sizes using floating precision synaptic weights.
Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equations
Here we show that a dynamical neural network, trained on a minimal amount of data, can predict the behavior of spintronic devices with high accuracy and an extremely efficient simulation time, compared to the micromagnetic simulations that are usually employed to model them.
Model of the Weak Reset Process in HfOx Resistive Memory for Deep Learning Frameworks
However, the resistive change behavior in this regime suffers many fluctuations and is particularly challenging to model, especially in a way compatible with tools used for simulating deep learning.
Training Dynamical Binary Neural Networks with Equilibrium Propagation
We first train systems with binary weights and full-precision activations, achieving an accuracy equivalent to that of full-precision models trained by standard EP on MNIST, and losing only 1. 9% accuracy on CIFAR-10 with equal architecture.
Synaptic metaplasticity in binarized neural networks
Unlike the brain, artificial neural networks, including state-of-the-art deep neural networks for computer vision, are subject to "catastrophic forgetting": they rapidly forget the previous task when trained on a new one.
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
Equilibrium Propagation (EP) is a biologically-inspired counterpart of Backpropagation Through Time (BPTT) which, owing to its strong theoretical guarantees and the locality in space of its learning rule, fosters the design of energy-efficient hardware dedicated to learning.
EqSpike: Spike-driven Equilibrium Propagation for Neuromorphic Implementations
Finding spike-based learning algorithms that can be implemented within the local constraints of neuromorphic systems, while achieving high accuracy, remains a formidable challenge.
In-Memory Resistive RAM Implementation of Binarized Neural Networks for Medical Applications
With new memory technology available, emerging Binarized Neural Networks (BNNs) are promising to reduce the energy impact of the forthcoming machine learning hardware generation, enabling machine learning on the edge devices and avoiding data transfer over the network.
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
In this work, we show that a bias in the gradient estimate of EP, inherent in the use of finite nudging, is responsible for this phenomenon and that cancelling it allows training deep ConvNets by EP.
Equilibrium Propagation with Continual Weight Updates
However, in existing implementations of EP, the learning rule is not local in time: the weight update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.
Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation
On the other hand, the biological plausibility of EP is limited by the fact that its learning rule is not local in time: the synapse update is performed after the dynamics of the second phase have converged and requires information of the first phase that is no longer available physically.
Synaptic Metaplasticity in Binarized Neural Networks
In this work, we interpret the hidden weights used by binarized neural networks, a low-precision version of deep neural networks, as metaplastic variables, and modify their training technique to alleviate forgetting.
Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction
One of the most exciting applications of Spin Torque Magnetoresistive Random Access Memory (ST-MRAM) is the in-memory implementation of deep neural networks, which could allow improving the energy efficiency of Artificial Intelligence by orders of magnitude with regards to its implementation on computers and graphics cards.
Stochastic Computing for Hardware Implementation of Binarized Neural Networks
In this work, we propose a stochastic computing version of Binarized Neural Networks, where the input is also binarized.
Emerging Technologies
Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input
Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i. e. RNNs that are fed by a static input x and settle to a steady state.
Spatio-temporal Learning with Arrays of Analog Nanosynapses
By translating the database into the time domain and using variable integration windows, up to 95% classification accuracy is achieved.
Exploiting the Short-term to Long-term Plasticity Transition in Memristive Nanodevice Learning Architectures
This approach outperforms a conventional ELM-inspired system when the first layer is imprinted before training and testing, and especially so when variability in device timing evolution is considered: variability is therefore transformed from an issue to a feature.
