1 code implementation • 15 Jan 2024 • Andreas Madsen, Sarath Chandar, Siva Reddy
For example, if an LLM says a set of words is important for making a prediction, then it should not be able to make its prediction without these words.
1 code implementation • 11 Oct 2023 • Andreas Madsen, Siva Reddy, Sarath Chandar
This is achieved by using a novel fine-tuning method that incorporates masking, such that masking tokens become in-distribution by design.
1 code implementation • 15 Oct 2021 • Andreas Madsen, Nicholas Meade, Vaibhav Adlakha, Siva Reddy
The principle is that this should result in worse model performance compared to masking random tokens.
no code implementations • 10 Aug 2021 • Andreas Madsen, Siva Reddy, Sarath Chandar
Neural networks for NLP are becoming increasingly complex and widespread, and there is a growing concern if these models are responsible to use.
3 code implementations • ICLR 2020 • Andreas Madsen, Alexander Rosenberg Johansen
We present two new neural network components: the Neural Addition Unit (NAU), which can learn exact addition and subtraction; and the Neural Multiplication Unit (NMU) that can multiply subsets of a vector.
4 code implementations • 4 Oct 2019 • Andreas Madsen, Alexander Rosenberg Johansen
The goal of NALU is to learn perfect extrapolation, which requires learning the exact underlying logic of an unknown arithmetic problem.