Sequence To Sequence Models

Sequence to Sequence

Introduced by Sutskever et al. in Sequence to Sequence Learning with Neural Networks

Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one LSTM, the encoder, to read the input sequence one timestep at a time, to obtain a large fixed dimensional vector representation (a context vector), and then to use another LSTM, the decoder, to extract the output sequence from that vector. The second LSTM is essentially a recurrent neural network language model except that it is conditioned on the input sequence.

(Note that this page refers to the original seq2seq not general sequence-to-sequence models)

Source: Sequence to Sequence Learning with Neural Networks


Paper Code Results Date Stars


Task Papers Share
Decoder 86 7.74%
Sentence 69 6.21%
Machine Translation 64 5.76%
Translation 60 5.40%
Language Modelling 45 4.05%
Text Generation 45 4.05%
Semantic Parsing 40 3.60%
Question Answering 25 2.25%
Speech Recognition 22 1.98%


Component Type
Recurrent Neural Networks