SAINT+: Integrating Temporal Features for EdNet Correctness Prediction

19 Oct 2020  ·  Dongmin Shin, Yugeun Shim, Hangyeol Yu, Seewoo Lee, Byungsoo Kim, Youngduck Choi ·

We propose SAINT+, a successor of SAINT which is a Transformer based knowledge tracing model that separately processes exercise information and student response information. Following the architecture of SAINT, SAINT+ has an encoder-decoder structure where the encoder applies self-attention layers to a stream of exercise embeddings, and the decoder alternately applies self-attention layers and encoder-decoder attention layers to streams of response embeddings and encoder output. Moreover, SAINT+ incorporates two temporal feature embeddings into the response embeddings: elapsed time, the time taken for a student to answer, and lag time, the time interval between adjacent learning activities. We empirically evaluate the effectiveness of SAINT+ on EdNet, the largest publicly available benchmark dataset in the education domain. Experimental results show that SAINT+ achieves state-of-the-art performance in knowledge tracing with an improvement of 1.25% in area under receiver operating characteristic curve compared to SAINT, the current state-of-the-art model in EdNet dataset.

PDF Abstract

Datasets


Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Knowledge Tracing EdNet SAINT+ AUC 0.7914 # 1
Acc 72.52 # 2
Knowledge Tracing EdNet SAKT AUC 0.7663 # 6
Acc 70.73 # 4
Knowledge Tracing EdNet DKVMN AUC 0.7668 # 5
Acc 70.79 # 3
Knowledge Tracing EdNet DKT AUC 0.7638 # 7
Acc 70.6 # 5

Methods