Development and Validation of an English Error Identification Test for Assessing English Grammar Competence of Grade 11 students: An Application of Computerized Dynamic Assessment
Article Sidebar
Main Article Content
Abstract
This research aimed to develop and validate an English error identification test for assessing grammar competence of Grade 11 students by application of computerized dynamic assessment. The research sample consisted of 100 Grade 11 students in academic year 2024 from schools under the Secondary Educational Service Office Lopburi. The research instrument was an English error identification test created using the iSpring Suite MAX program. The test was two-tier multiple-choice questions. Level 1 comprised an error identification test with four options, while Level 2 consisted of an error correction test with three options. The psychometric properties were analyzed based on content validity, item difficulty, discrimination index, and reliability. Additionally, user satisfaction of the human-computer interface was assessed using descriptive statistics, including the mean and standard deviation.
The test content covered eight grammatical components, totaling 20 items, i.e., (1) Part of Speech, (2) Tense, (3) Article, (4) Word Order, (5) Comparison, (6) If-clause,
(7) Parallelism, and (8) Subject-Verb Agreement. The index of item-objective congruence (IOC) for Level 1 questions ranged from 0.80 to 1.00, for Level 2 questions from 0.60 to 1.00, and for the mediation from 0.60 to 1.00. The difficulty index ranged from 0.41 to 0.65. The discrimination index ranged from 0.53 to 0.85. The internal consistency reliability of the test was 0.91. User satisfaction with the human-computer interface revealed that the highest level of satisfaction was in the system capabilities aspect, particularly the system speed, with the highest score (M = 4.35, SD = 0.83).
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The content and information contained in the published article in the Journal of Educational Measurement Mahasarakham University represent the opinions and responsibilities of the authors directly. The editorial board of the journal is not necessarily in agreement with or responsible for any of the content.
The articles, data, content, images, etc. that have been published in the Journal of Educational Measurement Mahasarakham University are copyrighted by the journal. If any individual or organization wishes to reproduce or perform any actions involving the entirety or any part of the content, they must obtain written permission from the Journal of Educational Measurement Mahasarakham University.
References
Adadan, E., & Savasci, F. (2012). An analysis of 16–17-year-old students' understanding of solution chemistry concepts using a two-tier diagnostic instrument. International Journal of Science Education, 34(4), 513–544.
Alduais, A. M. S. (2013). Language testing: An introductory course to design tests to language components and skills with exercises, assignments, and a final test. LAP LAMBERT.
Aljaafreh, A., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learning in the zone of proximal development. The Modern Language Journal, 78(4), 465–483. https://doi.org/10.1111/j.1540-4781.1994.tb02064.x
Crick, R. D., & Yu, G. (2008). Assessing learning dispositions: Is the effective lifelong learning inventory valid and reliable as a measurement tool? Educational Research, 50(4), 387–402. https://doi.org/10.1080/00131880802499886
Earl, L. M. (2003). Assessment as learning: Using classroom assessment to maximize student learning. Corwin.
Ebadi, S., Karimi, E., & Vakili, S. (2023). An exploration into EFL learners’ perspectives on online computerized listening comprehension dynamic assessment. Language Testing in Asia, 13(5). https://doi.org/10.1186/s40468-023-00221-9
Ebel, R. L. (1972). Why is a longer test usually a more reliable test? Educational and Psychological Measurement, 32, 249–253.
Gergely, D. (2007). Investigating the performance of alternative types of grammar items. language testing, 24(1), 65–97. https://doi.org/10.1177/0265532207071512
Grade University. (2023). Step by step: Typical grammar structures for each level. https://grade-university.com/blog/typical-grammar-structures-for-each-level
Iamsirirak, E. (2021). A development of English grammar proficiency test for teacher students majoring in English. Proceedings of the 13th Humanities and Social Sciences in Face of Concurrent Disruptions, 1–16.
Ibrahim, M. (2018). Utilizing dynamic assessment to activate EFL inert grammar. In S. Hidri (Ed.), Revisiting the assessment of second language abilities: From theory to practice (pp. 471–488). Springer.
Issacs, T., Zara, C., Herbert, G., Coombs, S. J., & Smith, C. (2014). Key concepts in educational assessment. SAGE.
Khoii, R., & Shamsi, N. (2011). The effect of the method on the trait: Investigating the function of no-error options in grammar error-identification items in admission tests. The Iranian EFL Journal, 7(2), 166–179.
Klungthong, D., & Wasanasomsithi, P. (2024). Effects of dynamic assessment on improvement of academic vocabulary knowledge of Thai EFL low-proficiency university students. LEARN Journal: Language Education and Acquisition Research Network, 17(1), 599–631. https://so04.tci-thaijo.org/index.php/LEARN/article/view/270429
Kozulin, A., & Garb, E. (2002). Dynamic assessment of EFL text comprehension. School Psychology International, 23(1), 112–127. https://doi.org/10.1177/0143034302023001733
Lidz, C. S., & Gindis, B. (2003). Dynamic assessment of the evolving cognitive functions in children. In A. Kozulin, B. Gindis, V. S. Ageyev, & S. M. Miller (Eds.), Vygotsky's educational theory in cultural context (pp. 99–116). Cambridge University. https://doi.org/10.1017/CBO9780511840975.007
Lunrasri, Y., Tangdhanakanond, K., & Pasiphol, S. (2022). Effects of prompting type and learning achievement on reading literacy of ninth graders. Kasetsart Journal of Social Sciences, 43(2), 369–378. https://doi.org/10.34044/j.kjss.2022.43.2.14
Madsen, H. (1983). Techniques in testing. Oxford University.
Mann, M., & Knowles, S. T. (2006). Destination B1 grammar & vocabulary with answer key. Macmillan.
Nihae, N., & Chiramanee, T. (2014). Multiple-choice and error recognition test: Effects of test anxiety on test performance. International Journal of Language Education, 2(2), 78–88. https://doi.org/10.5296/ijele.v2i2.5854
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
Nunnally, J. C., & Bearden, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.
Pamudyaningrum, F. E., Rante, H., Zainuddin, M. A., & Lund, M. (2020). UI/UX design for Metora: A gamification of learning journalism interviewing method. E3S Web of Conferences, 188, Article 00008. https://doi.org/10.1051/e3sconf/202018800008
Poehner, M. E., & Van Compernolle, R. A. (2011). Frames of interaction in dynamic assessment: developmental diagnoses of second language learning. Assessment in Education: Principles, Policy & Practice, 18(2), 183–198. https://doi.org/10.1080/0969594X.2011.567116
Poehner, M. E., Zhang, J., & Lu, X. (2015). Computerized dynamic assessment (C-DA): Diagnosing L2 development according to learner responsiveness to mediation. Language Testing, 32(3), 337–357. https://doi.org/10.1177/0265532214560390
Siwathaworn, P., & Wudthayagorn, J. (2018). The impact of dynamic assessment on tertiary EFL students’ speaking skills. The Asian Journal of Applied Linguistics, 5(1), 142–155. https://caes.hku.hk/ajal/index.php/ajal/article/view/514
Tracktest. (n.d.). English grammar CEF level requirements. https://tracktest.eu/english-grammar-cef-level-requirements/
Troop, M., White, D., Wilson, K. E., & Zeni, P. (2020). The user experience design for learning (UXDL) framework: The undergraduate student perspective. The Canadian Journal for the Scholarship of Teaching and Learning, 11(3), Article 8328. https://doi.org/10.5206/cjsotl-rcacea.2020.3.8328
Viechtbauer, W., Smits, L., Kotz, D., Budé, L., Spigt, M., Serroyen, J., & Crutzen, R. (2015). A simple formula for the calculation of sample size in pilot studies. Journal of Clinical Epidemiology, 68(11), 1375–1379. https://doi.org/10.1016/j.jclinepi.2015.04.014
Wing, J., & Putney, L. (2002). A vision of Vygotsky. Allyn & Bacon.
Chaimongkol, N., Pasiphol, S., & Kanjanawasee, S. (2016). Computerized adaptive testing with reflective feedback in information technology professional examination. Information Technology Journal, 12(2), 58–64. (in Thai)
Kanjanawasee, S. (2013). Classical test theory (7th ed.). Chulalongkorn University. (in Thai)
Office of the Basic Education Commission. (2015). Ministry of Education guidelines on the policy for reforming English language teaching and learning. Office of the Basic Education Commission. (in Thai)
Polyota, T. (2007). Learning in the zone of proximal development. MANUTSAT PARITAT: Journal of Humanities, 29(2), 5–17. https://ejournals.swu.ac.th/index.php/hm/issue/view/390 (in Thai)
The Secondary Educational Service Area Office Lopburi. (2024). Educational information academic year 2024. https://online.pubhtml5.com/jiut/xlvj/index.html (in Thai)
Worathanaphimmakorn, N., & Tangdhanakanond, K. (2024). Development of critical thinking scale of upper secondary school students in social studies by applying computerized dynamic assessment. Journal of Education Studies, Chulalongkorn University, 52(2), 1–14. https://doi.org/10.14456/educu.2024.18 (in Thai)
