Development of a Measure of Student’s Scientific Argumentation Skills: An Application of Construct Modeling

Abstract

The research aimed to (1) develop a construct map of scientific argumentation skills of lower secondary school students, and (2) develop and verify the quality of the measure for assessment of scientific argumentation skills. The sample consisted of 514 lower secondary school students in the academic year 2020 under the Secondary Education Service Area Office 25, obtained from the total population of 31,744 students using multistage sampling. The development of the assessment measure followed the construct modeling approach which had four steps: 1) developing a construct map of scientific argumentation skills; 2) designing items; 3) determining an outcome space; and 4) analyzing data through the MRCML model based on Wright Map. The results were as follows:
(1) The construct map of scientific argumentation skills featured two dimensions, namely development of argumentation elements and use of scientific knowledge. The former had 4 levels, ranging from level one—drawing an irrelevant conclusion, to level four—able to construct a counterclaim with justification. The latter also had 4 levels, ranging from level one—explaining with irrelevant scientific content or the student cannot recall scientific content, to level four—the student can use complex scientific knowledge to explain the matter.
(2) The measure for assessment of scientific argumentation skills consisted of 22 open-ended questions using dichotomous scoring (0-1) and polytomous scoring (0-3); 12 of the questions measured development of argumentation elements while the rest were concerned with the use of scientific knowledge.
(3) The results of the verification of the measure reflected validity evidences. It was found that: on the aspect of the validity of the content of the questions, their difficulty covered the range of students' skills; that is, they could explain scientific argumentation skills; on the aspect of students’ responses, it was found that the students could understand the content or situation of the questions as intended in this study; and the aspect of internal structure, the questions could be employed to measure scientific argumentation skills (Infit MNSQ range of 0.74 to 1.35) through the multidimensional model which was consistent with students’ responses, with statistical significance (²=17.8, df=2, p<.001). The AIC and BIC of the multidimensional model were lower than those of the unidimensional model. Considering validity evidence, the EAP/PV reliability of both dimensions of the argumentation construct was 0.89, which was in the acceptable range, and the standard errors of measurement was low, having SEM ₁ (development of argumentation elements) in the range of 0.38 – 0.56 while SEM ₂ (use of scientific knowledge) in that of 0.58 – 1.87.