The Development of Automatic Parallel Multiple-Choice Question Generation System Using Automatic Item Generation for Distance Education
Article Sidebar
Main Article Content
Abstract
The aims of this research were to: 1) create item models and design a data warehouse for mathematics and statistics for science and technology; 2) develop an automatic parallel multiple-choice question generation system using automatic item generation; and 3) investigate parallelism between the prototype test and computer-generated test. The research methodology was divided into three phases: Phase 1 involved item model creation and evaluation to create 42 models, alongside the design of a data warehouse to store parameter values of the created item models for all item models to share stored data. Phase 2 focused on system development in web application form to create automatic parallel multiple-choice questions, together with quality evaluation of the developed system. Phase 3 consisted of parallelism investigation by having experts evaluate the parallelism of each item between the prototype test and computer-generated test, followed by experimental testing with 30 students enrolled in Mathematics and Statistics for Science and Technology at the School of Science and Technology, Sukhothai Thammathirat Open University. The results revealed that: 1) All created item models of mathematics and statistics for science and technology passed the required criteria. 2) The automatic parallel multiple-choice question generation system successfully created parallel questions that aligned with the system's design objectives, with the overall system quality at a high level. 3) Expert evaluation confirmed that all items in both tests (the prototype and computer-generated tests) met the parallelism criteria. The experimental results showed that: (1) The prototype test had an average difficulty of 0.52, average discrimination of 0.41, variance of 0.24, mean score of 21.90, reliability of 0.89, and standard error of measurement of 2.98. (2) The computer -generated test had an average difficulty of 0.51, average discrimination of 0.40, variance of 0.25, mean score of 21.00, reliability of 0.90, and standard error of measurement of 2.84. Statistical analysis indicated no significant differences between the two test versions. Based on both expert evaluation and experimental results, it was concluded that the prototype test and computer-generated test were parallel.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The content and information contained in the published article in the Journal of Educational Measurement Mahasarakham University represent the opinions and responsibilities of the authors directly. The editorial board of the journal is not necessarily in agreement with or responsible for any of the content.
The articles, data, content, images, etc. that have been published in the Journal of Educational Measurement Mahasarakham University are copyrighted by the journal. If any individual or organization wishes to reproduce or perform any actions involving the entirety or any part of the content, they must obtain written permission from the Journal of Educational Measurement Mahasarakham University.
References
Gierl, M. J., & Haladyna, T. (Eds.) (2013). Automatic Item Generation: Theory and Practice. Routledge.
Gierl, M. J., & Lai, H. (2012). The Role of Item Models in Automatic Item Generation. International Journal of Testing, 12(3), 273-298.
Gierl, M. J., & Lai, H. (2013). Instructional Topics in Educational Measurement (ITEMS) Module: Using Automated Processes to Generate Test Items. Educational Measurement: Issues and Practice, 32(3), 36-50.
Gierl, M. J., Lai, H., & Turner, S. (2012). Using Automatic Item Generation to Create Multiple-Choice Items for Assessments in Medical Education. Medical Education, 46(8), 757-765.
Gierl, M.J., Zhou, J., & Alves, C. (2008). Developing a Taxonomy of Item Model Types to Promote Assessment Engineering. Journal of Technology, Learning, and Assessment, 7(2), 1-51.
Chantharamaha, S., & Chadcham, S. (2018). Automatic Item Generation in Mathematics for Grade Six Students Using Computer Software. Research Methodology & Cognitive Science, 16(1), 138-149. (in Thai)
Charuthat, S. (2003). Study of Parallel of Prathomsuksa 6 Mathematics Achievement Test Constructed with Facet Design [Master’s thesis]. Chiang Mai University. (in Thai)
Jantana, S., & Pradujprom. P. (2020). Development of Automatic Item Generation Program in Mathematics for Grade Nine Level. Journal of Educational Measurement, 37(102), 162-177. (in Thai)
Kanchanapenkul, S. (2011) A Construction of Parallel Domain-Referenced Test by Using Facet Design on “Circles” for Matthayomsuksa 3. Sripatum Chonburi Academic Journal, 9(4), 56-60. (in Thai)
Pradujprom, P., & Panthong, K. (2019). A Computer Program for Developing Multiple Choice Test Items Using Automatic Item Generation Method. Journal of Southern Technology, 12(2), 74-87. (in Thai)
Preechapanich, O. (2014). Guide book: System Analysis and Design, Complete version. IDC Premier. (in Thai)
Premthongsuk, P., Chadcham, S., & Pradujprom, P. (2017). Development of the Next Item Selection Procedure Using Hurwicz Criterion and Item Exposure Control for Computerized Adaptive Testing. Veridian E-Journal, Science and Technology Silpakorn University, 4(6), 32-50. (in Thai)
Raksombat, S., Pradujphom,. P., & Panthong, K. (2022). Development of an Automatic Item Generation System for Item Bank Classified by Content and Difficulty Levels: Application of Assessment Engineering Concepts. Journal of Educational Measurement, Mahasarakham University, 28(1), 311-330. (in Thai)
Sawangboon, N., Phattiyathani, S., & Wongrattana, C. (2012). A Construction of Mathematics Parallel Tests Using Facet Design on “Factoring of Degree-2 Polynomials” for Student in Matthayomsuksa 2. Journal of Educational Measurement, Mahasarakham University, 17(1), 369-378. (in Thai)
Srisa-ard, O. (1995). Verification of the validity of measurement tools by experts. Journal of Educational Measurement, Mahasarakham University, 1(1), 45-49. (in Thai)
