Development of an Automatic Item Generation System for Item Bank Classified by Content and Difficulty Levels: Application of Assessment Engineering Concepts

Main Article Content

Surachai Raksombat
Piyathip Pradujphom
Kanok Panthong

Abstract

The objectives of this research were 1) to create a mathematics item model in mathematics of grade 7, and 2) to develop an automatic item generation system for item bank classified by content and difficulty levels in mathematics of grade 7. The research was conducted in 2 phases. Phase 1 dealt with creating a mathematics item model in mathematics of grade 7. Phase 2 dealt with development of an automatic item generation system and to organize the item bank for grade 7, classified by content and difficulty levels in mathematics of grade 7.
The results showed that:
1. The mathematics item model in mathematics of grade 7 that was created consisted of 5 main parts: 1) stems, 2) elements, 3) options, 4) auxiliary Information and 5) keys. The items appropriate for creating an item model were 203 questions and were used to create 10 item model formats. On the whole, the item model was valid and passed the criteria.
2. The development of an automatic item generation system for item bank classified by content and difficulty levels in mathematics for grade 7 was carried out in the form of a web application which can be used via the Internet at https://www.aig-system.com. The overall program was suitable at a high level and accepted by the experts. The evaluation of the efficiency of the program was at a very good level, satisfying the program's users.

Downloads

Download data is not yet available.

Article Details

Section
Research Article

References

Arendasy, M. E., & Sommer, M. (2012). Using automatic item generation to meet the increasing item demands of high-stakes educational and occupational assessment. Learning and Individual Differences, 22(1), 112-117. doi:10.1016/j.lindif.2011.11.005

Gierl, M. J., & Lai, H. (2012). The Role of Item Models in Automatic Item Generation. International Journal of Testing, 12(3), 273-298. doi:10.1080/15305058.2011.635830

Gierl, M. J., & Lai, H. (2013). Instructional Topics in Educational Measurement (ITEMS) Module: Using Automated Processes to Generate Test Items. Educational Measurement: Issues and Practice, 32(3), 36-50.

Gierl, M. J., Lai, H., & Turner, S. R. (2012). Using automatic item generation to create multiple-choice test items. Med Educ, 46(8), 757-765. doi:10.1111/j.1365-2923.2012.04289.x

Gierl, M. J., Zhou, J., & Alves, C. (2008). Developing a Taxonomy of Item Model Types to Promote Assessment Engineering. The Journal of Technology, Learning, and Assessment, 7(2), 1-51.

Guyer, R., & Thompson, N.A. (2014). User’s Manual for Xcalibre item response theory calibration software, version 4.2.2 and later. Assessment Systems Corporation.

Luecht, R. M. (2013). Assessment Engineering Task Model Maps, Task Models and Templates as a New Way to Develop and Implement Test Specifications. Journal of Applied Testing Technology, 14, 1-38.

Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health, 29(5), 489-497. doi:10.1002/nur.20147

Urry, V. W. (1977). Tailored Testing: A Successful Application of Latent Trait Theory. Journal of Education Measurement, 14(2), 181-196.

Chantharamaha, S. & Chadcham, S. (2018). Automatic Item Generation in Mathematics for Grade Six Students Using Computer Software. Research Methodology & Cognitive Science, 16(1), 138-149. (in Thai)

Jantana, S. (2019). Development of Automatic Item Generation in Mathematics Assessment for Grade Nine Level. [Master’s thesis]. Burapha University. (in Thai)

Pradujprom, P. & Panthong, K. (2019). A Computer Program for Developing Multiple Choice Test Items Using Automatic Item Generation Method. Journal of Southern Technology, 12(2), 74-87. (in Thai)

Preechapanich, O. (2014). Guide book: System Analysis and Design, Complete version. IDC Premier. (in Thai)

Tiansawad, S (2007). Content validity index: Critique and recommendation for computation. Nursing Journal, 34(4), 1-9. (in Thai)