Search Results for author:
Found 4 papers, 4 papers with code
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases.
Probing via Prompting
We then examine the usefulness of a specific linguistic property for pre-training by removing the heads that are essential to that property and evaluating the resulting model's performance on language modeling.
Vision Matters When It Should: Sanity Checking Multimodal Machine Translation Models
Multimodal machine translation (MMT) systems have been shown to outperform their text-only neural machine translation (NMT) counterparts when visual context is available.
Differentiable Subset Pruning of Transformer Heads
Multi-head attention, a collection of several attention mechanisms that independently attend to different parts of the input, is the key ingredient in the Transformer.
