Search Results for author:
Found 34 papers, 7 papers with code
TeLU Activation Function for Fast and Stable Deep Learning
We propose the Hyperbolic Tangent Exponential Linear Unit (TeLU), a neural network hidden activation function defined as TeLU(x)=xtanh(exp(x)).
Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks
Energy-based learning algorithms, such as predictive coding (PC), have garnered significant attention in the machine learning community due to their theoretical properties, such as local operations and biologically plausible mechanisms for error correction.
Precision, Stability, and Generalization: A Comprehensive Assessment of RNNs learnability capability for Classifying Counter and Dyck Languages
However, our research challenges this notion by demonstrating that RNNs primarily operate as state machines, where their linguistic capabilities are heavily influenced by the precision of their embeddings and the strategies used for sampling negative examples.
Exploring Learnability in Memory-Augmented Recurrent Neural Networks: Precision, Stability, and Empirical Insights
This study explores the learnability of memory-less and memory-augmented RNNs, which are theoretically equivalent to Pushdown Automata.
A Unified Framework for Continual Learning and Unlearning
Continual learning and machine unlearning are crucial challenges in machine learning, typically addressed separately.
Investigating Symbolic Capabilities of Large Language Models
However, most research has predominantly focused on language-based reasoning and word problems, often overlooking the potential of LLMs in handling symbol-based calculations and reasoning.
Neuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps
The major challenge that makes crafting such a system difficult is known as catastrophic forgetting - an agent, such as one based on artificial neural networks (ANNs), struggles to retain previously acquired knowledge when learning from new samples.
A Review of Neuroscience-Inspired Machine Learning
As a result, it accommodates hardware and scientific modeling, e. g. learning with physical systems and non-differentiable behavior.
Stable and Robust Deep Learning By Hyperbolic Tangent Exponential Linear Unit (TeLU)
In this paper, we introduce the Hyperbolic Tangent Exponential Linear Unit (TeLU), a novel neural network activation function, represented as $f(x) = x{\cdot}tanh(e^x)$.
Stability Analysis of Various Symbolic Rule Extraction Methods from Recurrent Neural Network
We sampled the datasets from $7$ Tomita and $4$ Dyck grammars and trained them on $4$ RNN cells: LSTM, GRU, O2RNN, and MIRNN.
On the Computational Complexity and Formal Hierarchy of Second Order Recurrent Neural Networks
In this work, we extend the theoretical foundation for the $2^{nd}$-order recurrent network ($2^{nd}$ RNN) and prove there exists a class of a $2^{nd}$ RNN that is Turing-complete with bounded time.
Brain-Inspired Computational Intelligence via Predictive Coding
Artificial intelligence (AI) is rapidly becoming one of the key technologies of this century.
The Predictive Forward-Forward Algorithm
We propose the predictive forward-forward (PFF) algorithm for conducting credit assignment in neural systems.
Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images
In this work, we develop convolutional neural generative coding (Conv-NGC), a generalization of predictive coding to the case of convolution/deconvolution-based computation.
Like a bilingual baby: The advantage of visually grounding a bilingual language model
Unlike most neural language models, humans learn language in a rich, multi-sensory and, often, multi-lingual environment.
Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems
In this article, we propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC), designing an agent built completely from powerful predictive coding/processing circuits that facilitate dynamic, online learning from sparse rewards, embodying the principles of planning-as-inference.
A Robust Backpropagation-Free Framework for Images
While current deep learning algorithms have been successful for a wide variety of artificial intelligence (AI) tasks, including those involving structured image data, they present deep neurophysiological conceptual issues due to their reliance on the gradients that are computed by backpropagation of errors (backprop).
Neural JPEG: End-to-End Image Compression Leveraging a Standard JPEG Encoder-Decoder
In light of this, we propose a system that learns to improve the encoding performance by enhancing its internal neural representations on both the encoder and decoder ends, an approach we call Neural JPEG.
An Empirical Analysis of Recurrent Learning Algorithms In Neural Lossy Image Compression Systems
Recent advances in deep learning have resulted in image compression algorithms that outperform JPEG and JPEG 2000 on the standard Kodak benchmark.
Backprop-Free Reinforcement Learning with Active Neural Generative Coding
In humans, perceptual awareness facilitates the fast recognition and extraction of information from sensory input.
OmniLayout: Room Layout Reconstruction from Indoor Spherical Panoramas
A common approach has been to use standard convolutional networks to predict the corners and boundaries, followed by post-processing to generate the 3D layout.
Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Units
Two particular tasks that test this type of reasoning are (1) mathematical equation verification, which requires determining whether trigonometric and linear algebraic statements are valid identities or not, and (2) equation completion, which entails filling in a blank within an expression to make it true.
A provably stable neural network Turing Machine
We introduce a neural stack architecture, including a differentiable parametrized stack operator that approximates stack push and pop operations for suitable choices of parameters that explicitly represents a stack.
Recognizing Long Grammatical Sequences Using Recurrent Networks Augmented With An External Differentiable Stack
In this paper, we improve the memory-augmented RNN with important architectural and state updating mechanisms that ensure that the model learns to properly balance the use of its latent states with external memory.
Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment
Training deep neural networks on large-scale datasets requires significant hardware resources whose costs (even on cloud platforms) put them out of reach of smaller organizations, groups, and individuals.
Sibling Neural Estimators: Improving Iterative Image Decoding with Gradient Communication
For lossy image compression, we develop a neural-based system which learns a nonlinear estimator for decoding from quantized representations.
The Neural State Pushdown Automata
The NSPDA is also compared to a classical analog stack neural network pushdown automaton (NNPDA) as well as a wide array of first and second-order RNNs with and without external memory, trained using different learning algorithms.
Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting
In lifelong learning systems based on artificial neural networks, one of the biggest obstacles is the inability to retain old knowledge as new information is encountered.
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations
We compare our model and learning procedure to other back-propagation through time alternatives (which also tend to be computationally expensive), including real-time recurrent learning, echo state networks, and unbiased online recurrent optimization.
A Neural Temporal Model for Human Motion Prediction
We propose novel neural temporal models for predicting and synthesizing human motion, achieving state-of-the-art in modeling long-term motion trajectories while being competitive with prior work in short-term prediction and requiring significantly less computation.
Like a Baby: Visually Situated Neural Language Acquisition
We examine the benefits of visual context in training neural language models to perform next-word prediction.
Biologically Motivated Algorithms for Propagating Local Target Representations
Finding biologically plausible alternatives to back-propagation of errors is a fundamentally important challenge in artificial neural network research.
Learned Neural Iterative Decoding for Lossy Image Compression Systems
For lossy image compression systems, we develop an algorithm, iterative refinement, to improve the decoder's reconstruction compared to standard decoding techniques.
Conducting Credit Assignment by Aligning Local Representations
Using back-propagation and its variants to train deep networks is often problematic for new users.
