Development of a Measure for Assessment of Scientific Process Skills, Using Construct Modeling Approach

Abstract

The main objective of this study was to develop a measure for assessment of multidimensional scientific process skills on the topic of Stoichiometry for grade 11.  There were two specific objectives: (1) to develop construct maps of scientific process skills; and  (2) to create and check the quality of the measure. The sample consisted of 1,200 grade 11 students in the academic year 2018 from schools under the Secondary Educational Service Area Offices in the Northeast, Thailand. The measure was developed according to the construct modeling approach which consisted of 4 steps: (1) development of the multidimensional scientific process skill construct maps; (2) designing the test items; (3) scoring the responses; and (4) analysis of the data according to the multidimensional random coefficients multinomial logit model, using ConQuest program. The research findings were as follows:

           1) The multidimensional scientific process skill construct maps had 2 dimensions: the predicting dimension that had 4 levels, from the lowest level that the student knew the reaction occurred but could not answer or explain the occurrence correctly, to the highest level that the student could give the reasons and tell the difference of the reactions, and could make use of the knowledge of the rate of chemical reaction consistently with the authentic context; and the dimension of organizing data and communication which had 4 levels, from the lowest level that the student knew and could tell that the rate of reaction changed according to the area of the surface, but could not tell which reactant reacted to the limestone, to the highest level that the student could apply basic scientific knowledge and the thinking process in showing how to calculate the amount of hydrogen ion that reacted with the limestone and could tell the temperature which affected the reaction.

          2) The measure for assessment of the science process skills consisted of 15 open-ended items and they were polytomous items to which a score from 0 to 3 could be given, according to the level of skill in the multidimensional scientific process skill construct maps that had been developed. There were 8 items on the predicting dimension and 7 items for the organizing data and communication dimension.

          3) The results of the assessment of the science process skills were validated. On the aspect of validity evidence of the items, it was found that the item difficulties covered the range of the student skill estimates. That means the content of the items could be the definition of the student scientific process skills. On the aspect of validity evidence of the response process, it was found that the students understood the content of the items as intended. And on the aspect of validity evidence related to the internal structure, it was found that the items could be used to measure the process skills dimensionally (Infit MNSQ between 0.81 and 1.11) as the multidimensional model fitted the response data significantly (=332.48, df =2, p < .01), and the  AIC and BIC were lower than the unidimensional model. The measure showed the EAP/PV reliabilities of 0.78 for the predicting dimension, 0.70 for the organizing data and communication dimension, respectively, which were acceptable; and the standard errors of measurement was low.