Applying Polytomous Item Response Theory to the Development of an English Reading Literacy Item Bank
Article Sidebar
Main Article Content
Abstract
This study aimed to develop an English reading literacy item bank by applying Polytomous Item Response Theory (IRT) using the Graded Response Model (GRM). The sample consisted of 4,000 upper secondary school students in northeastern Thailand, selected through multi-stage random sampling. The instrument used was a situational English reading test comprising 438 items. All items demonstrated acceptable content validity based on expert ratings (4.00–5.00 on a five-point scale).The items were divided into 18 test forms, each containing 24 items, with 6 anchor items shared across forms. The scoring was based on an 8-level rubric. The tests were administered to the sample, and data were analyzed using the mirt package in R. A total of 339 items met the quality criteria. The overall item parameter estimates showed that the average discrimination () across the three levels was 1.295 with a standard deviation of 0.466. The average difficulty (
) across all thresholds was –0.643 with a standard deviation of 1.181. The item bank covers a wide range of ability levels, from low to high. The test items can assess learners from CEFR levels A1 to B1, with some items capable of measuring abilities higher than average (
> 2), reflecting the effectiveness of the item bank. The item bank was stored in an online database developed using PhpMyAdmin. This item bank will be used for further development into a Computerized Adaptive Testing (CAT) system.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The content and information contained in the published article in the Journal of Educational Measurement Mahasarakham University represent the opinions and responsibilities of the authors directly. The editorial board of the journal is not necessarily in agreement with or responsible for any of the content.
The articles, data, content, images, etc. that have been published in the Journal of Educational Measurement Mahasarakham University are copyrighted by the journal. If any individual or organization wishes to reproduce or perform any actions involving the entirety or any part of the content, they must obtain written permission from the Journal of Educational Measurement Mahasarakham University.
References
Alderson, J. C. (2010). Assessing Reading. Cambridge University Press.
Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice. Oxford University Press.
Baker, F. B. (2001). The basics of item response theory (2nd ed.). ERIC Clearinghouse on Assessment and Evaluation.
Brown, D. H. (2004). Language assessment: Principles and classroom practices. Pearson Education.
Chalmers, R. P. (2012). MIRT: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29.
Chattergoon, R. (2020). Using polytomous item response theory models to validate learning progressions [Doctoral dissertation]. University of Colorado.
Dai, S., Vo, T. T., Kehinde, O. J., He, H., Xue, Y., Demir, C., & Wang, X. (2021). Performance of polytomous IRT models with rating scale data: An investigation over sample size, instrument length, and missing data. Frontiers in Education, 6, Article 721963.
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9–30.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Psychology Press.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Kluwer Academic.
Loken, E., & Rulison, K. L. (2010). Estimation of a four-parameter item response theory model. The British Journal of Mathematical and Statistical Psychology, 63(3), 509–525.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Associates.
Ostini, R., & Nering, M. L. (2006). Polytomous item response theory models. SAGE.
Pinitsakul, P., Senarat, S., & Senarat, B. (2020). Item response theory and testlet test. Ubon Ratchathani Journal of Research and Evaluation, 9(2), 74–84.
Ravand, H. (2015). Assessing testlet effect, impact, differential testlet, and item functioning using cross-classified multilevel measurement modeling. SAGE Open, 5(2), 1–9.
Richards, J. C. (2015). Key issues in language teaching. Cambridge University Press.
Samejima, F. (1969). Estimation of Latent Ability Using a Response Pattern of Graded Scores. Psychometrika Monograph.
Senarat, S., Tayraukham, S., Piyapimonsit, C., & Tongkhambanjong, S. (2013). Development of a computerized adaptive testing for diagnosing the cognitive process of grade 7 students in learning algebra, using multidimensional item response theory. Educational Research and Reviews, 8(13), 1009–1021.
Thissen, D., Mislevy, R. J., & Wainer, H. (2000). Testing algorithms. In H. Wainer (Ed.), Computerized adaptive testing: A primer (2nd ed.). Lawrence Erlbaum Associates.
van der Linden, W. J., & Glas, C. A. W. (Eds.). (2000). Computerized adaptive testing: Theory and practice. Kluwer Academic.
Wainer, H., Bradlow, E. T., & Wang, X. (2007). Testlet response theory and its applications. Cambridge University Press.
Waltz, C. F., Strickland, O. L., & Lenz, E. R. (2010). Measurement in nursing and health research (4th ed.). Springer.
Weiss, D. J. (1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 6(4), 473–492.
Wheadon, B. C. (2011). An item response theory approach to the maintenance of standards in public examinations in England [Doctoral dissertation]. Durham University. http://etheses.dur.ac.uk/615/
Yen, W. M. (1996). Polytomous item response theory models and their applications in large scale testing programs: Review of literature (ETS Research Memorandum No. 96-08). Educational Testing Service.
Kanjanawasee, S. (2020). Modern test theory (5th ed.). Chulalongkorn University. (in Thai).
Raksombat, S., Pradujphom, P., & Panthong, K. (2020). Item bank classified by content and difficulty levels using item response theory. Journal of Educational Measurement Mahasarakham University, 26(2), 322–333. (in Thai).
Romkaew, N., Senarat, S., & Saengprom, N. (2021). Application of item response theory in development of the item bank of basic knowledge in physical education. Journal of Educational Measurement Mahasarakham University, 27(2), 82–98. (in Thai).
