Search Results for author:
Found 7 papers, 2 papers with code
Eye Gaze and Self-attention: How Humans and Transformers Attend Words in Sentences
Attention describes cognitive processes that are important to many human phenomena including reading.
Do Smaller Language Models Answer Contextualised Questions Through Memorisation Or Generalisation?
We train two Language Models in a multitask fashion whereby the second model differs from the first only in that it has two additional datasets added to the training regime that are designed to impart simple numerical reasoning strategies of a sort known to improve performance on some of our evaluation datasets but not on others.
Challenges in Annotating Datasets to Quantify Bias in Under-represented Society
Recent advances in artificial intelligence, including the development of highly sophisticated large language models (LLM), have proven beneficial in many real-world applications.
Neuromodulation Gated Transformer
We introduce a novel architecture, the Neuromodulation Gated Transformer (NGT), which is a simple implementation of neuromodulation in transformers via a multiplicative effect.
Input-length-shortening and text generation via attention values
Because of the attention mechanism's high computational cost, transformer models usually have an input-length limitation caused by hardware constraints.
AbductionRules: Training Transformers to Explain Unexpected Inputs
Transformers have recently been shown to be capable of reliably performing logical reasoning over facts and rules expressed in natural language, but abductive reasoning - inference to the best explanation of an unexpected observation - has been underexplored despite significant applications to scientific discovery, common-sense reasoning, and model interpretability.
Relating Blindsight and AI: A Review
If we assume that artificial networks have no form of visual experience, then deficits caused by blindsight give us insights into the processes occurring within visual experience that we can incorporate into artificial neural networks.
