Forecasting Trends in Educational Measurement and Assessment Topics in the AI Era: Text Mining Analysis with Bayesian Multilevel Piecewise Regression Model
Keywords:
research trend forecasting, text mining, artificial intelligence in education, Bayesian multilevel piecewise regression modelAbstract
This study aimed to 1) identify latent research topics in educational measurement and evaluation from academic databases covering the years 2008–2025 using Structural Topic Modeling (STM), 2) analyze topic prevalence trends and forecast their growth trajectories for 2027 using Bayesian Multilevel Piecewise Regression (BMPR), and 3) synthesize strategic research themes from the topic classification and forecasting results. The dataset consisted of 1,695 research articles from the proceedings of three international conferences. The data were processed to extract 10,858 standardized keywords and key phrases, which were then analyzed using STM to identify latent research topics. Topic labeling and interpretation were conducted through a human-in-the-loop process in collaboration with large language models. The findings showed that STM identified 95 latent research topics, which could be semantically organized into two major domains: assessment, evaluation, and policy; and innovation and technology for learning. BMPR results revealed high variability in topic prevalence trends, reflecting significantly different patterns of growth, slowdown, or stability across topics. Forecasting and topic screening identified 40 high-potential topics, which were subsequently grouped into 13 subclusters and synthesized into four strategic research themes: AI-driven learning systems, measurement and validity in large-scale assessment, equity and engagement, and responsible use of artificial intelligence.
References
ภาษาอังกฤษ
Arieli-Attali, M., Ward, S., Thomas, J., Deonovic, B., & von Davier, A. A. (2019). The expanded Evidence-Centered Design (e-ECD) for learning and assessment systems: A framework for incorporating learning goals and processes within assessment design. Frontiers in Psychology, 10. Article 853. https://doi.org/10.3389/fpsyg.2019.00853
Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In R. K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 253-272). Cambridge University Press. https://doi.org/10.1017/cbo9781139519526.016
Blei, D. M. (2011). Dynamic and supervised topic models for literature-based discovery. https://apps.dtic.mil/sti/tr/pdf/ADA554210.pdf
Bulut, O., Beiting-Parrish, M., Casabianca, J. M., Slater, S. C., Jiao, H., Song, D., Ormerod, C., Fabiyi, D. G., Ivan, R., Walsh, C., Rios, O.,
Wilson, J., Yildirim-Erbasli, S. N., Wongvorachan, T., Liu, J. X., Tan, B., & Morilova, P. (2024). The rise of artificial intelligence in educational measurement: Opportunities and ethical challenges. Chinese/English Journal of Educational Measurement and Evaluation, 5(3), Article 3. https://doi.org/10.59863/MIQL7785
Chen, X., Zou, D., Cheng, G., & Xie, H. (2020). Detecting latent topics and trends in educational technologies over four decades using structural topic modelling: A retrospective of all volumes of Computers & Education. Computers & Education, 151, Article 103855. https://doi.org/10.1016/j.compedu.2020.103855
von Davier, A. A., Deonovic, B., Yudelson, M., Polyak, S. T., & Woo, A. (2019). Computational psychometrics approach to holistic learning and assessment systems. Frontiers in Education, 4, Article 69. https://doi.org/10.3389/feduc.2019.00069
Ozyurt, O., & Ayaz, A. (2022). Twenty-five years of education and information technologies: Insights from a topic modeling–based bibliometric analysis. Education and Information Technologies, 27, 11025–11054. https://doi.org/10.1007/s10639-022-11071-y
Maddox, B. (2023). The uses of process data in large-scale educational assessments (OECD Education Working Papers No. 286). OECD. https://doi.org/10.1787/5d9009ff-en
Han, X. (2020). Evolution of research topics in LIS between 1996 and 2019: An analysis based on latent Dirichlet allocation topic model. Scientometrics, 125, 2561–2595. https://doi.org/10.1007/s11192-020-03721-0
Holmes, W., & Tuomi, I. (2022). State of the art and practice in AI in education. European Journal of Education, 57(4), 542–570. https://doi.org/10.1111/ejed.12533
Natarajan, S., Mathur, S., Sidheekh, S., Stammer, W., & Kersting, K. (2025). Human-in-the-loop or AI-in-the-loop? automate or collaborate? Proceedings of the AAAI Conference on Artificial Intelligence, 39(27), 28594–28600. https://doi.org/10.1609/aaai.v39i27.35083
OECD. (2021). The state of global education: 18 months into the pandemic. https://www.oecd.org/content/dam/oecd/en/publications/reports/2021/09/the-state-of-global-education_f3f08b36/1a23bb23-en.pdf
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380-1400. https://doi.org/10.1177/0002764213498851
Sievert, C., & Shirley, K. E. (2014). LDAvis: A method for visualizing and interpreting topics. In J. Chuang, S. Green, M. Hearst, J. Heer, & P. Koehn (Eds.), Proceedings of the workshop on interactive language learning, visualization, and interfaces (pp. 63–70). Association for Computational Linguistics. https://doi.org/10.3115/v1/W14-3110
Singun, A. (2025). Unveiling the barriers to digital transformation in higher education institutions: A systematic literature review. Discover Education, 4, Article 37. https://doi.org/10.1007/s44217-025-00430-9
Tang, X., Li, X., Ding, Y., Song, M., & Bu, Y. (2020). The pace of artificial intelligence innovations: Speed, talent, and trial-and-error. Journal of Informetrics, 14(4), Article 101094. https://doi.org/10.1016/j.joi.2020.101094
UNESCO. (2021). Reimagining our futures together: A new social contract for education. https://unesdoc.unesco.org/ark:/48223/pf0000379707
Vehtari, A., Gelman, A., Simpson, D., Carpenter, B., & Bürkner, P.-C. (2021). Rank-normalization, folding, and localization: An improved R ̂ for assessing convergence of MCMC. Bayesian Analysis, 16(2), 667–718. https://doi.org/10.1214/20-BA1221
Williamson, B., & Eynon, R. (2020). Historical threads, missing links, and future directions in AI in education. Learning, Media and Technology, 45(3), 223–235. https://doi.org/10.1080/17439884.2020.1798995
Xiong, J., & Li, F. (2026). A decade of reflection and thematic review on Artificial Intelligence’s impact on educational measurement. Educational Research Review, 51, Article 100789. https://doi.org/10.1016/j.edurev.2026.100789
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
