Search Results for author:
Found 9 papers, 5 papers with code
Unified Scaling Laws for Routed Language Models
The performance of a language model has been shown to be effectively modeled as a power-law in its parameter count.
Do Transformers Need Deep Long-Range Memory?
Deep attention models have advanced the modelling of sequential data across many domains.
Multiplicative Interactions and Where to Find Them
We explore the role of multiplicative interaction as a unifying framework to describe a range of classical and modern neural network architectural motifs, such as gating, attention layers, hypernetworks, and dynamic convolutions amongst others.
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
This paper aims to quantify and reduce a particular type of bias exhibited by language models: bias in the sentiment of generated text.
Training language GANs from Scratch
Generative Adversarial Networks (GANs) enjoy great success at image generation, but have proven difficult to train in the domain of natural language.
Neural Arithmetic Logic Units
Neural networks can learn to represent and manipulate numerical information, but they seldom generalize well outside of the range of numerical values encountered during training.
Relational recurrent neural networks
Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods.
Ranked #46 on
Language Modelling
on WikiText-103
Unsupervised Predictive Memory in a Goal-Directed Agent
Animals execute goal-directed behaviours despite the limited range and scope of their sensors.
Model-Free Episodic Control
State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance.
