Search Results for author:
Found 16 papers, 7 papers with code
CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory
Large Language Models (LLMs) struggle to handle long input sequences due to high memory and runtime costs.
Neuron-Astrocyte Associative Memory
In their known biological implementations the ratio of stored memories to the number of neurons remains constant, despite the growth of the network size.
Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
The generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks.
Long Sequence Hopfield Memory
Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions.
End-to-end Differentiable Clustering with Associative Memories
Clustering is a widely used unsupervised learning technique involving an intensive discrete optimization problem.
Sparse Distributed Memory is a Continual Learner
Continual learning is a problem for artificial neural networks that their biological counterparts are adept at solving.
Energy Transformer
Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory.
Associative Learning for Network Embedding
The network embedding task is to represent the node in the network as a low-dimensional vector while incorporating the topological and structural information.
Hierarchical Associative Memory
Dense Associative Memories or Modern Hopfield Networks have many appealing properties of associative memory.
Can a Fruit Fly Learn Word Embeddings?
In this work we study a mathematical formalization of this network motif and apply it to learning the correlational structure between words and their context in a corpus of unstructured text, a common natural language processing (NLP) task.
Large Associative Memory Problem in Neurobiology and Machine Learning
We show that these models are effective descriptions of a more microscopic (written in terms of biological degrees of freedom) theory that has additional (hidden) neurons and only requires two-body interactions between them.
Bio-Inspired Hashing for Unsupervised Similarity Search
Building on inspiration from FlyHash and the ubiquity of sparse expansive representations in neurobiology, our work proposes a novel hashing algorithm BioHash that produces sparse high dimensional hash codes in a data-driven manner.
Local Unsupervised Learning for Image Analysis
Local Hebbian learning is believed to be inferior in performance to end-to-end training using a backpropagation algorithm.
Unsupervised Learning by Competing Hidden Units
It is widely believed that the backpropagation algorithm is essential for learning good feature detectors in early layers of artificial neural networks, so that these detectors are useful for the task performed by the higher layers of that neural network.
Dense Associative Memory is Robust to Adversarial Inputs
Third, adversarial images constructed by models with small power of the interaction vertex, which are equivalent to DNN with rectified linear units (ReLU), fail to transfer to and fool the models with higher order interactions.
Dense Associative Memory for Pattern Recognition
The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning.
