Detecção e predição de estados afetivos baseadas em mineração de dados educacionais: considerando a personalidade do aluno para aumentar a precisão da detecção

Abstract

It has already been shown in the literature that emotions, an affective state type, interfere in the learning process, as well as in the students’ engagement. Thus, it is important that learning educational environments that aim to provide an improvement in the student learning process, such as Intelligent Tutoring Systems (ITSs), also have this ability. It is known that several educational environments have presented different ways to realize the recognition of affect through specific sensors or hardware. However, such a strategy becomes unfeasible when it comes to the use of these environments in mass, by hundreds or even thousands of students. Thus, the strategy of sensor-free detection, through the use of interaction data of students with educational environments, has become an interesting solution. This work aims to detect students’ frustration, confusion, boredom, and engaged concentration through data mining in step-based ITSs. The research hypothesis of this work is that the addition of personality features of the students in the detectors training of these states can result an accuracy improvement in the detection of these four states since it is known that the personality influences the affective states. The method we used was the development of detectors trained with and without personality features. To collect the training data of these detectors, we carried out a data collection with 55 students from a private school, who used the PAT2Math ITS during ten sessions. During the sessions, students had their faces recorded along with ambient audio and the computer screen while using the system. From the data obtained from these sessions, a total of 5525 interaction logs between the student and the system was selected. For each of these logs, 348 features were calculated, containing information from the (i) student interactions with the PAT2Math interface, (ii) the student module, (iii) the personality traits, and (iv) the affective states and behaviors of the students. We developed a new annotation protocol of affective states and behaviors based on the analysis of the videos generated during the data collection. This protocol follows a set of phases, including training and testing of the coders, and it can be flexible and generalizable for different applications and scalable because it does not require expert coders during data collection. Thus, it is considered an additional contribution to this work. Through this protocol, 2099 labels of affective states and 2059 labels of behaviors were collected. We developed two detectors for each affective state, a trained version with and another without the personality features, allowing to verify the impact of the personality in the detection of the affective states. We applied Cohen’s Kappa metric to identify the agreement between the affective state labels generated by the coders and the outputs of the developed affective state detectors. As results, it was possible to identify that only the engaged concentration detector, trained with personality data of the students, obtained a small improvement in the precision of the detection. But, one personality feature was selected automatically during detectors training. Thus, this work points to the evidence that personality can positively impact the detection of students’ affective states in learning environments. We highlighted the possible viability of a new data source and a new annotation protocol for affective states as the contributions of this work. Both contributions go towards the goal of performing automatic and real-time detection of students’ affective states, allowing instantaneous adaptation of the learning environments according to the students’ emotions.