Designing Methods to Diagnose Scientific Misconceptions by using Automated feedback through Machine Learning
Article Sidebar
Main Article Content
Abstract
This research had the objectives to: (1) analyze student response patterns regarding construct map, misconceptions, objective test transformation, cut scores, and the students’ baseline data, to predict their level; and (2) design diagnostic methods for scientific misconceptions using the automated feedback system through machine learning. The research was conducted under design research model. The participants comprised 713 grade 10 students as respondents. The instrument was a diagnosing multidimensional scientific misconceptions test that included knowledge dimension and rational thinking dimension. The analysis of the student response patterns was based on the multidimensional model under MRCML model. Then, the diagnostic methods for scientific misconceptions were developed.
The results were as follows:
1. On the respondent patterns analysis, it was found that (1) the test was capable of measuring the student’s ability according to the construct map, accompanied with superior level evaluation in some particular segments; (2) most students had their misconceptions on each item in conceptual misunderstanding and vernacular misconceptions. Their misconceptions on the whole test in the knowledge dimension were at the no understanding level (NU) and in the rational thinking dimension were at the partial understanding (PU) level. (3) Objective test transformation derived from the student responses contained multiple choices with varied scores elicited on individual choice capability. (4) The setting cut scores on Wright map in the knowledge dimension were divided into five levels and four cut scores, at -0.50, 0.50, 0.42 and 1.19 respectively. The rational thinking dimension featured four levels with three cut scores at -0.19, 2.29 and 5.49 respectively. And (5) the students’ baseline data to predict the level of the students’ ability included the overall GPA, science GPA and hours per week of self-study in science.
2. The design of the diagnostic methods for scientific misconceptions using the automated feedback system through machine learning was developed on the basis of 4 elements: 1. input, 2. processing, 3. automated feedback, and 4. assessment reporting.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The content and information contained in the published article in the Journal of Educational Measurement Mahasarakham University represent the opinions and responsibilities of the authors directly. The editorial board of the journal is not necessarily in agreement with or responsible for any of the content.
The articles, data, content, images, etc. that have been published in the Journal of Educational Measurement Mahasarakham University are copyrighted by the journal. If any individual or organization wishes to reproduce or perform any actions involving the entirety or any part of the content, they must obtain written permission from the Journal of Educational Measurement Mahasarakham University.
References
American Educational Research Association, American Psychological Association & National Council on Measurement in Education. (2014). Standards for Educational and Psychological Testing.Standards for Educational and Psychological Testing. The American Educational Research Association
Adams, R.J., Wilson, M., & Wang, W.C. (1997). The multidimensional random coefficients multinomial logic model. Applied Psychological Measurement, 21(1), 1-23.
Gadzella, B. M., Baloglu, M., & Stephens, R. (2002). Prediction of GPA with Educational psychology grades and critical thinking scores. Project Innovation (Alabama), 122(3), 618-623.
Gouli, E., Gogoulou, A., & Grigoriadou, M. (2008). Supporting Self- Peer- and Collaborative-Assessment in E-Learning: The Case of the PEer and Collaborative ASSessment Environment (PECASSE). Journal of Interactive Learning Research, 19(4), 615-647.
Gurel, K. G, Eryılmaz, A., & McDermott, L. C. (2015). A Review and Comparison of Diagnostic Instruments to Identify Students’ Misconceptionss in Science. Eurasia Journal of Mathematics. Science & Technology Education, 11(5). https://doi.org/10.12973/eurasia.2015.1369a
Kantahan, S., Junpeng, P., Punturat, S., Tang,K. N., P. Gochyyev., & Wilson, M. (2020).Designing and verifying a tool for diagnosing multidimensional scientific misconceptions in genetics topic. International Journal of Evaluation and Research in Education, 9(3), 564-571. https:// doi.org/10.11591/ijere.v9i3.20544
Kalas, P., O’Neill. A., Pollock, C., & Birol, G. (2013). Development of a meiosis concept inventory. CBE Life Sciences Education, 12(4), 655–664.https://doi.org/10.1187/cbe.12-10-0174
Ling, T.W. (2017). Fostering Understanding and Reducing Misconceptions about Image Formation by a Plane Mirror Using Constructivist-Based Hands-On Activities. Springer.
Mitchell, T. M. (1997). Does Machine Learning Really Work?. AI Magazine, 18(3), 11. https://doi.org/10.1609/aimag.v18i3.1303
Mohammed, M., Khan, M.B., and Bashier, E.B.M. (2016). Machine learning: algorithms and applications. CRC Press. https://doi.org/10.1201/9781315371658
Nafea, I. (2018). Machine Learning in Educational Technology. Machine Learning – Advanced Techniques and Emerging Applications, 175-183. https://doi.org /10.5772/intechopen.72906
National Research Council. (1997). Science teaching reconsidered. National Academies Press.
Paneaw, S., Junpeng, P., & Tang, K. N. (2021). Designing Standards-Setting for Levels of Mathematical Proficiency in Measurement and Geometry: Multidimensional Item Response Model. Journal of Education and Learning, 10(6), 103-111. https://doi.org/10.5539/jel.v10n6p103
Soeharto, S., Csapó, B., Sarimanah, E., Dewi, F.I., and Sabri, T. (2019). A Review of Students’ Common Misconceptionss in Science and Their Diagnostic Assessment Tools. Journal Pendidikan IPA Indonesia, 8(2), 247-266. https://doi.org/10.15294/jpii.v8i2.18649
Westbrook, S. L., & Marek, E. A. (1992). “A cross-age study of student understanding of the concept of diffusion”. Journal of Research in Science Teaching, 29(1), 51-61.
Widiyatmoko, A., & Shimizu, K. (2018). Literature Review of Factors Contributing to Students’ Misconceptions in Light and Optical Instruments. International Journal of Environmental and Science Education, 13(10), 853-863.
Wilson, M. (2005). Constructing measures: An item response modeling approach. Evaluation and Program Planning, 28(4), 433-434. https://doi.org/10.1016/j.evalprogplan.2005.07.008
Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ACER Conquest: Generalized item response modelling software. ACER.
Wu, J. Y., Hsiao, Y. C., & Nian, M. W. (2018). Using supervised machine learning on large-scale online forums to classify course-related Facebook messages in predicting learning achievement within the personal learning environment. Interactive Learning Environments, 28(1), 1-16. https://doi.org/10.1080/10494820.2018.1515085
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program Evaluation standards: A guide for evaluators and evaluation users (3rd Ed.). Sage.
Zhang, F., Zhang, X., Luo, M., & Geng, H. (2016). The Effects of Feedback on Memory Strategies of Younger and Older Adults. PLOS ONE. https://doi.org/10.1371/journal.pone.0168896
Junpeng, P. (2018). Application of Multidimensional Item Response Theory to Research. Khon Kaen University Press. (in Thai)
Junpeng, P., Marwiang, M., Chinjunthuk, S., Suwannatrai, P., Grotha, J., Chanayotha, K., Tawarungruang, C., Thuanman, J., Ngang, T.K., & Wilson, M. R. (2020). Developing Students’ Mathematical Proficiency Level Diagnostic Tools through Information Technology in Assessment for Learning Report. Thailand Science Research and Innovation. (in Thai)
Kantahan, S. (2020). Designing and verifying a tool for diagnosing multidimensional scientific misconceptionss in genetics topic [Master’s thesis]. Khon Kaen University (in Thai)
Kerdkliang, S., Intasuwan, S., & Wannasuth, N. (2019). Effect of Inquiry Learning Cycle (7E) and Jigsaw Technique on Plants Reproduction and Development Topic to Develop Biological Learning Achievement and Social Skill for Mattayomseuksa 5 Student. Research and Innovation for Sustainability Development, 29(29), 1943-1950. (in Thai)
Masantiah, J. (2017). Development of A Computer-Based Testing System with Immediate Feedback for Different Student’s Ability Levels: Application of Rasch Sirt Model [Master’s thesis]. Chulalongkorn University. (in Thai)
Paneaw, S. (2021). Standard Setting of Mathematical Proficiency Level for the Mixed Format Test through Digital Technology [Master’s thesis]. Khon Kaen University. (in Thai)
Thakho, P., & Termtachatipongsa, P. (2015). The Development of Grade 11 Students’ Conception of Photosynthesis using Inquiry-Based Approach Associated with Scaffolding Strategies. Journal of Education Khon Kaen University, 38(3), 94-104. (in Thai)
Varasunun, P. (2011). The Program Evaluation Standards (3rd Ed.) in 2010. Journal of Research Methodology, 24(2), 273-278. (in Thai)
Wongwanich, S. (2017). Design Research in Education. Chulalongkorn University Press. (in Thai)