Classification Accuracy and Consistency of the DINA Model And the DINO Model in Cognitive Diagnostic Assessment
Article Sidebar
Main Article Content
Abstract
The objective of this research was to compare classification accuracy and classification consistency of the DINA model and the DINO model under different conditions. Data were generated by using Monte Carlo technique under 150 conditions consisting of: (1) test lengths included 4, 8, 12, 20 and 40 items; (2) six sample sizes containing 10, 25, 50, 100, 200 and 500 units; (3) five levels of percentage of multidimensionality, which were 0, 25, 50, 75 and 100 percentages; and (4) three attributes to be measured. Each condition was repeated 1,000 times. The data was simulated and assessed with R program.
The research results were: (1) there were statistically significant differences in classification accuracy and classification consistency of the DINA model and the DINO model based on the test lengths; (2) there were no statistically significant differences in classification accuracy and classification consistency of the DINA model and the DINO model based on the sample sizes; (3) there were statistically significant differences in classification accuracy and classification consistency of the DINA model and the DINO model based on the percentages of multidimensionality. In terms of practical recommendations, teachers and educational personnel should select 8 – 12 items and 25 – 50 percent of multidimensional measurement for the DINA model or the DINO model when they are used.
Article Details
The content and information contained in the published article in the Journal of Educational Measurement Mahasarakham University represent the opinions and responsibilities of the authors directly. The editorial board of the journal is not necessarily in agreement with or responsible for any of the content.
The articles, data, content, images, etc. that have been published in the Journal of Educational Measurement Mahasarakham University are copyrighted by the journal. If any individual or organization wishes to reproduce or perform any actions involving the entirety or any part of the content, they must obtain written permission from the Journal of Educational Measurement Mahasarakham University.
References
พิศิษฐ ตัณฑวณิช. (2558). การพัฒนาแบบสอบวินิจฉัยโดยแนวคิดด้านพุทธิปัญญา. วารสารบัณฑิตวิจัย, 6(2), 1-13.
ศิริชัย กาญจนวาสี. (2556). ทฤษฎีการทดสอบแบบดั้งเดิม. พิมพ์ครั้งที่ 7. กรุงเทพมหานคร: โรงพิมพ์แห่งจุฬาลงกรณ์มหาวิทยาลัย.
Aryadoust, V. (2011). Cognitive diagnostic assessment as an alternative measurement model. JALT Testing & Evaluation SIG Newsletter, 15(1), 2-6.
Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850-866.
Cui, Y., Gierl, M. J., & Chang, H. H. (2012). Estimating classification consistency and accuracy for cognitive diagnostic assessment. Journal of Educational Measurement, 49(1), 19-38.
de La Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115-130.
de La Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47(2), 227-249.
de La Torre, J., & Karelitz, T. M. (2009). Impact of diagnosticity on the adequacy of models for cognitive diagnosis under a linear attribute structure: A simulation study. Journal of Educational Measurement, 46(4), 450-469.
de La Torre, J., & Lee, Y. S. (2013). Evaluating the wald test for item level comparison of saturated and reduced models in cognitive diagnosis. Journal of Educational Measurement, 50(4), 355-373.
de La Torre, J., & Minchen, N. (2014). Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicología Educativa, 20(2), 89-97.
George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2), 1-24.
Guo, L., Bao, Y., Wang, Z., & Bian, Y. (2014). Cognitive diagnostic assessment with different weight for attribute: Based on the Dina Model. Psychological reports, 114(3), 802-822.
Lee, Y. S., de la Torre, J., & Park, Y. S. (2012). Relationships between cognitive diagnosis, CTT, and IRT indices: An empirical investigation. Asia Pacific Education Review, 13(2), 333-345.
Köhn, H. F., & Chiu, C. Y. (2016). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33(2), 171-184.
Kuo, B. C., Chen, C. H., & de la Torre, J. (2017). A cognitive diagnosis model for identifying coexisting skills and misconceptions. Applied Psychological Measurement, 41, 1-13.
Studer, C. (2012). Incorporating learning over time into the cognitive assessment framework (Unpublished doctoral dissertation). Carnegie Mellon University, Pittsburgh, PA.
Thummaphan, P., Li, M., & de la Torre, J. (2558, 25 สิงหาคม). Models for cognitive diagnostic modeling. รวมบทความวิธีวิทยาและทฤษฎีเพื่อการวิจัยทางพฤติกรรมในชุมชนและโรงเรียน, 9-22.
Wang, W., Song, L., Chen, P., Meng, Y., & Ding, S. (2015). Attribute level and pattern level classification consistency and accuracy indices for cognitive diagnostic assessment. Journal of Educational Measurement, 52(4), 457-476.