Attention Mechanisms

Location-based Attention

Introduced by Luong et al. in Effective Approaches to Attention-based Neural Machine Translation

Location-based Attention is an attention mechanism in which the alignment scores are computed from solely the target hidden state $\mathbf{h}_{t}$ as follows:

$$ \mathbf{a}_{t} = \text{softmax}(\mathbf{W}_{a}\mathbf{h}_{t}) $$

Source: Effective Approaches to Attention-based Neural Machine Translation

Papers


Paper Code Results Date Stars

Components


Component Type
Softmax
Output Functions

Categories