Affective states and state tests | Proceedings of the Third International Conference on Learning Analytics and Knowledge
Affective states and state tests: investigating how affect throughout the school year predicts end of year learning outcomes
Abstract
In this paper, we investigate the correspondence between student affect in a web-based tutoring platform throughout the school year and learning outcomes at the end of the year, on a high-stakes mathematics exam. The relationships between affect and learning outcomes have been previously studied, but not in a manner that is both longitudinal and finer-grained. Affect detectors are used to estimate student affective states based on post-hoc analysis of tutor log-data. For every student action in the tutor the detectors give us an estimated probability that the student is in a state of boredom, engaged concentration, confusion, and frustration, and estimates of the probability that they are exhibiting off-task or gaming behaviors. We ran the detectors on two years of log-data from 8th grade student use of the ASSISTments math tutoring system and collected corresponding end of year, high stakes, state math test scores for the 1,393 students in our cohort. By correlating these data sources, we find that boredom during problem solving is negatively correlated with performance, as expected; however, boredom is positively correlated with performance when exhibited during scaffolded tutoring. A similar pattern is unexpectedly seen for confusion. Engaged concentration and frustration are both associated with positive learning outcomes, surprisingly in the case of frustration.
References
[1]
Aleven, V., McLaren, B., Roll, I., and Koedinger, K. 2004. Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. In J. C. Lester, R. M. Vicario, and F. Paraguaçu (Eds.), Proceedings of Seventh International Conference on Intelligent Tutoring Systems, ITS 2004, 227--239.
[2]
Arnold, K. E. 2010. Signals: Applying Academic Analytics. Educause Quarterly, 33, 1.
[3]
Baker, R. S. J. d. 2007. Modeling and Understanding Students' Off-Task Behavior in Intelligent Tutoring Systems. In Proceedings of ACM CHI 2007: Computer-Human Interaction, 1059--1068.
[4]
Baker, R. S. J. d., D'Mello, S. K., Rodrigo, M. M. T., and Graesser, A. C. 2010. Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments. Int'l. J. Human-Computer Studies. 68, 4, 223--241.
[5]
Baker, R. S. J. d., Goldstein, A. B., and Heffernan, N. T. 2011. Detecting Learning Moment-by-Moment. International Journal of Artificial Intelligence in Education. 21, 1--2, 5--25.
[6]
Baker, R. S. J. d., Gowda, S., Corbett, A. T. 2011. Towards predicting future transfer of learning. In Proceedings of 15th International Conference on Artificial Intelligence in Education, 23--30.
[7]
Baker, R. S. J. d., Gowda, S. M., Wixon, M., Kalka, J., Wagner, A. Z., Salvi, A., Aleven, V., Kusbit, G., Ocumpaugh, J., and Rossi, L. 2012. Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. In Proceedings of the 5 th International Conference on Educational Data Mining, 126--133.
[8]
Bartel, C. A., and Saavedra, R. 2000. The Collective Construction of Work Group Moods. Administrative Science Quarterly 45, 2, 197--231.
[9]
Cocea, M., Hershkovitz, A., Baker, R. S. J. D. 2009. The Impact of Off-task and Gaming Behaviors on Learning: Immediate or Aggregate? In Proceedings of the 14th International Conference on Artificial Intelligence in Education, 507--514.
[10]
Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 1, 37--46.
[11]
Conati, C., and Maclaren, H. 2009. Empirically building and evaluating a probabilistic model of user affect. User Modeling and User-Adapted Interaction 19, 3, 267--303.
[12]
Craig, S. D., Graesser, A. C., Sullins, J., and Gholson, B. 2004. Affect and learning: an exploratory look into the role of affect in learning. Journal of Educational Media 29, 241--250
[13]
D'Mello, S. K., Craig, S. D., Witherspoon, A. W., McDaniel, B. T., and Graesser, A. C. 2008. Automatic Detection of Learner's Affect from Conversational Cues. User Modeling and User-Adapted Interaction 18, 1--2, 45--80.
[14]
Feng, M., Heffernan, N. T., and Koedinger, K. R. 2009. Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. Journal of User Modeling and User-Adapted Interaction, 19, 243--266.
[15]
Hanley, J., and McNeil, B. 1980. The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve. Radiology 143, 29--36.
[16]
Lee, D. M., Rodrigo, M. M., Baker, R. S. J. D., Sugay, J., Coronel, A. 2011. Exploring the Relationship Between Novice Programmer Confusion and Achievement. In Proceedings of the 4th bi-annual International Conference on Affective Computing and Intelligent Interaction.
[17]
Lehman, B., D'Mello, S. K., and Graesser, A. C. 2012. Confusion and Complex Learning during Interactions with Computer Learning Environments. The Internet and Higher Education, 15, 3, 184--194.
[18]
Pardos, Z. A., Wang, Q. Y., Trivedi, S. 2012. The real world significance of performance prediction. In Proceedings of the 5th International Conference on Educational Data Miningm, 192--195
[19]
Pekrun, R., Goetz, T., Titz, W., and Perry, R. P. 2002. Academic emotions in students' self-regulated learning and achievement: A program of quantitative and qualitative research. Educational Psychologist, 37, 91--106.
[20]
Planalp, S., DeFrancisco, V. L., and Rutherford, D. Varieties of Cues to Emotion in Naturally Occurring Settings. 1996. Cognition and Emotion 10, 2, 137--153.
[21]
Rodrigo, M. M. T., Baker, R. S., Jadud, M. C., Amarra, A. C. M., Dy, T., Espejo-Lahoz, M. B. V., Lim, S. A. L., Pascua, S. A. M. S., Sugay, J. O., Tabanao, E. S. 2009. Affective and Behavioral Predictors of Novice Programmer Achievement. In Proceedings of the 14th ACM-SIGCSE Annual Conference on Innovation and Technology in Computer Science Education, 156--160.
[22]
Sabourin, J., Mott, B., and Lester, J. Modeling Learner Affect with Theoretically Grounded Dynamic Bayesian Networks. 2011. In Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction, 286--295.
[23]
San Pedro, M. O. C., Baker, R., Rodrigo, M. M. 2011. Detecting Carelessness through Contextual Estimation of Slip Probabilities among Students Using an Intelligent Tutor for Mathematics. In Proceedings of 15th International Conference on Artificial Intelligence in Education, 304--311.
[24]
Sayette, M. A., Cohn, J. F., Wertz, J. M., Perrott, M. A., and Parrott, D. J. 2001. A psychometric evaluation of the facial action coding system for assessing spontaneous expression. J. Nonverbal Behavior 25, 3.
[25]
Van Rijsbergen, C. J. 1974. Foundation of evaluation. Journal of Documentation, 30, 365--373.
Information & Contributors
Information
Published In
LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge
April 2013
300 pages
Copyright © 2013 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 April 2013
Permissions
Request permissions for this article.
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
Acceptance Rates
LAK '13 Paper Acceptance Rate 16 of 58 submissions, 28%;
Overall Acceptance Rate 236 of 782 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- Downloads (Last 12 months)42
- Downloads (Last 6 weeks)6
Reflects downloads up to 25 Feb 2025
Other Metrics
Citations
- Le Tallec JPrihar EKäser T(2025)The Effect of Different Support Strategies on Student AffectProceedings of the 15th International Learning Analytics and Knowledge Conference10.1145/3706468.3706469(783-789)Online publication date: 3-Mar-2025
- Liu CWei Y(2025)Computerized Adaptive Testing Framework Based on Excitation Block and Gumbel-SoftmaxIEEE Access10.1109/ACCESS.2024.352400913(3475-3484)Online publication date: 2025
- Yu JZhuang YHuang ZLiu QLi XLi RChen ESalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)A unified adaptive testing system enabled by hierarchical structure searchProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3694454(57803-57817)Online publication date: 21-Jul-2024
- Wang ZWu WZeng CLuo HSun J(2024)Psychological factors enhanced heterogeneous learning interactive graph knowledge tracing for understanding the learning processFrontiers in Psychology10.3389/fpsyg.2024.135919915Online publication date: 10-May-2024
- Tadeu P(2024)A synopsis of the importance of teaching fractions to children until K-10Eurasia Journal of Mathematics, Science and Technology Education10.29333/ejmste/1487820:8(em2485)Online publication date: 2024
- Borchers CZhang JBaker RAleven V(2024)Using Think-Aloud Data to Understand Relations between Self-Regulation Cycle Characteristics and Student Performance in Intelligent Tutoring SystemsProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636911(529-539)Online publication date: 18-Mar-2024
- Cai MRebolledo Mendez GArevalo GTang SAbdullah YDemmans Epp C(2024)Toward Supporting Adaptation: Exploring Affect’s Role in Cognitive Load when Using a Literacy GameProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642150(1-17)Online publication date: 11-May-2024
- Zhao GHuang ZZhuang YBi HWang YWang FMa ZZhao Y(2024)A Diffusion-Based Cognitive Diagnosis Framework for Robust Learner AssessmentIEEE Transactions on Learning Technologies10.1109/TLT.2024.349221417(2227-2241)Online publication date: 2024
- Rahman MMunir MRahman MBadiuzzaman M(2024)EmoDetect: A Learning-Centred Affective Database for Detecting Student Frustration in Online Learning2024 5th International Conference on Advancements in Computational Sciences (ICACS)10.1109/ICACS60934.2024.10473235(1-6)Online publication date: 19-Feb-2024
- Rahman MOllington RYeom SOllington N(2024)Generalisable sensor-free frustration detection in online learning environments using machine learningUser Modeling and User-Adapted Interaction10.1007/s11257-024-09402-434:4(1493-1527)Online publication date: 1-Sep-2024
- Show More Cited By
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.