Self-study with robots has attracted attention in education in recent years. To enhance the effectiveness of the robot’s learning support, it is crucial to monitor learners’ mental state and situation in real time and provide feedback at appropriate moments. Thus, we focus on confidence levels during learning and estimate them using eye movement and pupil information. Previous research didn’t consider temporal changes in confidence levels and physiological data. Therefore, we propose a method to measure the subjective temporal evaluation by considering the temporal dynamics of eye movement and confidence levels, aiming to increase the possibility of estimating confidence levels in real time. Based on the data collected from participants’ responses to 30 questions, we used a Convolutional Neural Network (CNN) to classify ‘Unconfident’ and ‘Confident.’ The highest classification using all participants’ data accuracy was 85.5% in case using saccade and pupil information without data augmentation. The results suggest that these methods are suitable for real-time estimation of confidence levels. The test using all participants’ data, which didn’t consider individual differences, could not clarify the necessity of data augmentation. On the other hand, from the comparison of individual results, it was confirmed that each feature’s effectiveness varies between individuals. Moreover, the ablation test on each participant showed the effectiveness of data augmentation, but it was not significant. This indicates the possibility of improving estimation accuracy, and we need to analyze individual differences such as personality and learning skills and improve the quality of data for future work.