Relational Weight Priors in Neural Networks for Abstract Pattern Learning and Language Modelling

no code implementations • 10 Mar 2021 • Radha Kopparti, Tillman Weyde

Abstract patterns are the best known examples of a hard problem for neural networks in terms of generalisation to unseen data.

Inductive Bias Language Modelling +1

Weight Priors for Learning Identity Relations

no code implementations • 6 Mar 2020 • Radha Kopparti, Tillman Weyde

In this work, we extend RBP by realizing it as a Bayesian prior on network weights to model the identity relations.

Inductive Bias

Factors for the Generalisation of Identity Relations by Neural Networks

no code implementations • 13 Jun 2019 • Radha Kopparti, Tillman Weyde

In this work we explore various factors in the neural network architecture and learning process whether they make a difference to the generalisation on equality detection of Neural Networks without and and with DR units in early and mid fusion architectures.

Inductive Bias