Classification Accuracy and Consistency of the DINA Model And the DINO Model in Cognitive Diagnostic Assessment

Main Article Content

Waranyu Chayaban
Sukumarn Noklang
Korawut Phanprom
Siwachoat Srisuttiyakorn

Abstract

             The objective of this research was to compare classification accuracy and classification consistency of the DINA model and the DINO model under different conditions. Data were generated by using Monte Carlo technique under 150 conditions consisting of: (1) test lengths included 4, 8, 12, 20 and 40 items; (2) six sample sizes containing 10, 25, 50, 100, 200 and 500 units; (3) five levels of percentage of multidimensionality, which were 0, 25, 50, 75 and 100 percentages; and (4) three attributes to be measured. Each condition was repeated 1,000 times. The data was simulated and assessed with R program.


            The research results were: (1) there were statistically significant differences in classification accuracy and classification consistency of the DINA model and the DINO model based on the test lengths; (2) there were no statistically significant differences in classification accuracy and classification consistency of the DINA model and the DINO model based on the sample sizes; (3) there were statistically significant differences in classification accuracy and classification consistency of the DINA model and the DINO model based on the percentages of multidimensionality. In terms of practical recommendations, teachers and educational personnel should select 8 – 12 items and 25 – 50 percent of multidimensional measurement for the DINA model or the DINO model when they are used.

Downloads

Download data is not yet available.

Article Details

Section
Research Article

References

พิศิษฐ ตัณฑวณิช. (2558). การพัฒนาแบบสอบวินิจฉัยโดยแนวคิดด้านพุทธิปัญญา. วารสารบัณฑิตวิจัย, 6(2), 1-13.
ศิริชัย กาญจนวาสี. (2556). ทฤษฎีการทดสอบแบบดั้งเดิม. พิมพ์ครั้งที่ 7. กรุงเทพมหานคร: โรงพิมพ์แห่งจุฬาลงกรณ์มหาวิทยาลัย.
Aryadoust, V. (2011). Cognitive diagnostic assessment as an alternative measurement model. JALT Testing & Evaluation SIG Newsletter, 15(1), 2-6.
Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850-866.
Cui, Y., Gierl, M. J., & Chang, H. H. (2012). Estimating classification consistency and accuracy for cognitive diagnostic assessment. Journal of Educational Measurement, 49(1), 19-38.
de La Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115-130.
de La Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47(2), 227-249.
de La Torre, J., & Karelitz, T. M. (2009). Impact of diagnosticity on the adequacy of models for cognitive diagnosis under a linear attribute structure: A simulation study. Journal of Educational Measurement, 46(4), 450-469.
de La Torre, J., & Lee, Y. S. (2013). Evaluating the wald test for item level comparison of saturated and reduced models in cognitive diagnosis. Journal of Educational Measurement, 50(4), 355-373.
de La Torre, J., & Minchen, N. (2014). Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicología Educativa, 20(2), 89-97.
George, A. C., Robitzsch, A., Kiefer, T., Groß, J., & Ünlü, A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2), 1-24.
Guo, L., Bao, Y., Wang, Z., & Bian, Y. (2014). Cognitive diagnostic assessment with different weight for attribute: Based on the Dina Model. Psychological reports, 114(3), 802-822.
Lee, Y. S., de la Torre, J., & Park, Y. S. (2012). Relationships between cognitive diagnosis, CTT, and IRT indices: An empirical investigation. Asia Pacific Education Review, 13(2), 333-345.
Köhn, H. F., & Chiu, C. Y. (2016). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33(2), 171-184.
Kuo, B. C., Chen, C. H., & de la Torre, J. (2017). A cognitive diagnosis model for identifying coexisting skills and misconceptions. Applied Psychological Measurement, 41, 1-13.
Studer, C. (2012). Incorporating learning over time into the cognitive assessment framework (Unpublished doctoral dissertation). Carnegie Mellon University, Pittsburgh, PA.
Thummaphan, P., Li, M., & de la Torre, J. (2558, 25 สิงหาคม). Models for cognitive diagnostic modeling. รวมบทความวิธีวิทยาและทฤษฎีเพื่อการวิจัยทางพฤติกรรมในชุมชนและโรงเรียน, 9-22.
Wang, W., Song, L., Chen, P., Meng, Y., & Ding, S. (2015). Attribute level and pattern level classification consistency and accuracy indices for cognitive diagnostic assessment. Journal of Educational Measurement, 52(4), 457-476.