Search Results for author:
Found 5 papers, 1 papers with code
Scope is all you need: Transforming LLMs for HPC Code
With easier access to powerful compute resources, there is a growing trend in the field of AI for software development to develop larger and larger language models (LLMs) to address a variety of programming tasks.
Memory in humans and deep language models: Linking hypotheses for model augmentation
The computational complexity of the self-attention mechanism in Transformer models significantly limits their ability to generalize over long temporal durations.
Memory-Augmented Graph Neural Networks: A Brain-Inspired Review
We provide a comprehensive review of the existing literature on memory-augmented GNNs.
Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decay
Altogether, we found that LSTM with the proposed forget gate can learn long-term dependencies, outperforming other recurrent networks in multiple domains; such gating mechanism can be integrated into other architectures for improving the learning of long timescale information in recurrent neural networks.
Multi-timescale Representation Learning in LSTM Language Models
Earlier work has demonstrated that dependencies in natural language tend to decay with distance between words according to a power law.
