RelPos-DSAKT, a new knowledge tracing approach normally employed for online tutoring and learning, has been proposed. In the approach, a transformer model with two self-attention layers is constructed. The LSTM networks are used for data preprocessing. In addition, the relative position encoding technique is used. The LSTM network is well-known for its prominence in tackling problems raised by a long input sequence while retaining the long-term information contained in it. The relative position encoding technique can strengthen the self-attentive model in its ability of capturing correlation information between data elements, especially those with short distances. Since both long and short input sequences are commonly seen in knowledge tracing study area, a combination use of LSTM and the relative position encoding technique is expected to reinforce the power of the self-attentive model in handling the dependency of data elements in not only long but also short input sequences. Experiments on four broadly used data sets have been done on four models including ours as well as three classic knowledge tracing models. The results show the best performance of our model over the other three on all four data sets in AUC values, which verifies both the effectiveness and the reasonableness of our method in knowledge tracing studies. In addition, an ablation study has been conducted, the results of which show further the effectiveness of our method in capturing relation information in input sequences.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Self-attentive Knowledge Tracing with Relative Position Encoding

  • Jun Dai,
  • Fang Yu,
  • Xuan Yang,
  • Qin Li

摘要

RelPos-DSAKT, a new knowledge tracing approach normally employed for online tutoring and learning, has been proposed. In the approach, a transformer model with two self-attention layers is constructed. The LSTM networks are used for data preprocessing. In addition, the relative position encoding technique is used. The LSTM network is well-known for its prominence in tackling problems raised by a long input sequence while retaining the long-term information contained in it. The relative position encoding technique can strengthen the self-attentive model in its ability of capturing correlation information between data elements, especially those with short distances. Since both long and short input sequences are commonly seen in knowledge tracing study area, a combination use of LSTM and the relative position encoding technique is expected to reinforce the power of the self-attentive model in handling the dependency of data elements in not only long but also short input sequences. Experiments on four broadly used data sets have been done on four models including ours as well as three classic knowledge tracing models. The results show the best performance of our model over the other three on all four data sets in AUC values, which verifies both the effectiveness and the reasonableness of our method in knowledge tracing studies. In addition, an ablation study has been conducted, the results of which show further the effectiveness of our method in capturing relation information in input sequences.