Development of a Measure for Assessment of Scientific Process Skills, Using Construct Modeling Approach

Main Article Content

Bussakorn Singku
Putcharee Junpeng

Abstract

The main objective of this study was to develop a measure for assessment of multidimensional scientific process skills on the topic of Stoichiometry for grade 11.  There were two specific objectives: (1) to develop construct maps of scientific process skills; and  (2) to create and check the quality of the measure. The sample consisted of 1,200 grade 11 students in the academic year 2018 from schools under the Secondary Educational Service Area Offices in the Northeast, Thailand. The measure was developed according to the construct modeling approach which consisted of 4 steps: (1) development of the multidimensional scientific process skill construct maps; (2) designing the test items; (3) scoring the responses; and (4) analysis of the data according to the multidimensional random coefficients multinomial logit model, using ConQuest program. The research findings were as follows:


           1) The multidimensional scientific process skill construct maps had 2 dimensions: the predicting dimension that had 4 levels, from the lowest level that the student knew the reaction occurred but could not answer or explain the occurrence correctly, to the highest level that the student could give the reasons and tell the difference of the reactions, and could make use of the knowledge of the rate of chemical reaction consistently with the authentic context; and the dimension of organizing data and communication which had 4 levels, from the lowest level that the student knew and could tell that the rate of reaction changed according to the area of the surface, but could not tell which reactant reacted to the limestone, to the highest level that the student could apply basic scientific knowledge and the thinking process in showing how to calculate the amount of hydrogen ion that reacted with the limestone and could tell the temperature which affected the reaction.


          2) The measure for assessment of the science process skills consisted of 15 open-ended items and they were polytomous items to which a score from 0 to 3 could be given, according to the level of skill in the multidimensional scientific process skill construct maps that had been developed. There were 8 items on the predicting dimension and 7 items for the organizing data and communication dimension.


          3) The results of the assessment of the science process skills were validated. On the aspect of validity evidence of the items, it was found that the item difficulties covered the range of the student skill estimates. That means the content of the items could be the definition of the student scientific process skills. On the aspect of validity evidence of the response process, it was found that the students understood the content of the items as intended. And on the aspect of validity evidence related to the internal structure, it was found that the items could be used to measure the process skills dimensionally (Infit MNSQ between 0.81 and 1.11) as the multidimensional model fitted the response data significantly (gif.latex?\chi&space;^{2}=332.48, df =2, p < .01), and the  AIC and BIC were lower than the unidimensional model. The measure showed the EAP/PV reliabilities of 0.78 for the predicting dimension, 0.70 for the organizing data and communication dimension, respectively, which were acceptable; and the standard errors of measurement was low.

Article Details

Section
Research Article

References

กระทรวงศึกษาธิการ. (2551). หลักสูตรแกนกลางการศึกษาขั้นพื้นฐาน พุทธศักราช 2551. กรุงเทพมหานคร: โรงพิมพ์คุรุสภาลาดพร้าว.

พนิดา กระทุ่มนอก และ พัชรี จันทร์เพ็ง. (2560). ความก้าวหน้าในการเรียนรู้มโนมติทางวิทยาศาสตร์ของ นักเรียน ระดับชั้นมัธยมศึกษาปีที่ 4 เรื่อง รูปร่างโมเลกุลโควาเลนต์. ขอนแก่น: มหาวิทยาลัยขอนแก่น.

พัชรี จันทร์เพ็ง. (2561). การประยุกต์ทฤษฎีการตอบสนองข้อสอบแบบพหุมิติเพื่อการวิจัย. ขอนแก่น: โรงพิมพ์มหาวิทยาลัยขอนแก่น.

วีรภัทร์ สุขศิริ และ ชนม์ชกรณ์ วรอินทร์. (2559). การตรวจคะแนนจุดตัดขั้นต้นสำหรับกลุ่มสาระการเรียนรู้ วิทยาศาสตร์จากคะแนนสอบระดับชาติขั้นพื้นฐานด้วยวิธีทำแผนที่ความสามารถแฝงเชิงประจักษ์: การวิเคราะห์เพื่อการตัดสิน. กรุงเทพมหานคร: สถาบันทดสอบทางการศึกษาแห่งชาติ.

สำนักวิชาการและมาตรฐานการศึกษา. (2551). ตัวชี้วัดและสาระการเรียนรู้แกนกลาง กลุ่มสาระการเรียนรู้ วิทยาศาสตร์ ตามหลักสูตรแกนกลางการศึกษาขั้นพื้นฐาน พุทธศักราช 2551. กรุงเทพมหานคร: โรง พิมพ์ชุมนุมสหกรณ์การเกษตรแห่งประเทศไทย

Adams, R. J. (2005). Reliability as a measurement design effect. Studies in Educational Evaluation, 31, 162-172.

Adams, R. J., & Khoo, S. T. (1996). Quest. Melbourne, Australia: Australian Council for Educational Research.

Adams, R. J., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1-23

Akaike, H. (1974). A new look at the statistical model identification. IEEE Transaction on Automatic Control, 19(6), 716-723.

American Educational Research Association, American Psychological Association, & Nation Council on Measurement in Education. (2014). Standards for education and psychological testing. Washington, DC: American Educational Research Association.

American Association for the Advancement of Science. (1993). American Association for the Advancement of Science Project 2061: Science for all Americans. Washington, DC: American Association for the Advancement of Science

Arantika, J., Saputro, S., & Mulyani, S. (2019). Effectiveness of guided inquiry-based module to improve science process skills. Journal of Physics: Conference Series, 1157(4).

Azizah, N., Ibrahim, M., & Widodo, W. (2017). Process skill assessment instrument: Innovation to measure student’s learning result holistically. Journal of Physics Conference, 947(1).

Chiappetta, E. L., & Koballa, T. R. (2002). Science instruction in the middle and secondary schools. Upper Saddle River, NJ: Merrill/Prentice Hall.

Cizek, G. J. (2012). Defining and distinguishing validity: Interpretations of score meaning and justifications of test use. Psychological Methods, 17(1), 31.

DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford: Oxford University Press.

Junpeng, P., Inprasitha, M., & Wilson, M. (2018). Modelling of the open-ended items for assessing multiple proficiencies in mathematical problem solving. The Turkish Online Journal of Educational Technology (TOJET), 17.

Linacre, J. M. (1994). Sample size and item calibration [or person measure] stability. Rasch Measurement Transactions, 7(4), 328.

Linacre, J. M. (1998). Many-facet Rasch measurement. Chicago: MESA Press.

Ongowo, R. O., & Indoshi, F. C. (2013). Science process skills in the Kenya certificate of secondary education biology practical examinations. Creative Education, 11(4), 713-717.

Ozgelen, S. (2012). Scientists’ science process skills within a cognitive domain framework. Eurasia Journal of Mathematics, Science & Technology Education, 8, 283-292.

Padilla, M. J. (1990). The science process skills: Research matters to the science teacher. Reston, VA: National Association for Research in Science Teaching. http://www.narst. org/publications/research/skill.cfm

Rambuda, A. M., & Fraser, W. J. (2004). Perceptions of teachers of the application of science process skills in teaching geography in secondary schools in the free state province in South African. Journal of Education, 24, 10-17.

Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press. (Original work published 1960).

Sass D. A., Schmitt T. A., & Walker C. M. (2008). Estimating non-normal latent trait distributions with item response theory using true and estimated item parameters. Applied Measurement in Education, 21, 65-88.

Schwarz, G. (1978). Estimating the Dimension of a Model. The Annals of Statistics, 6(2), 461-464.

Webb, N. (1997). Criteria for alignment of expectations and assessments on mathematics and science education. Washington, DC: CCSSO

Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Routledge.

Wilson, M., & Draney, K. (2002). A technique for setting standards and maintaining them over the time. In S. Nishisato, Y. Baba, H. Bozdogan, & K. Kanefugi (Eds.), Measurement and multivariate analysis (pp. 325-332). New York, NY: SpringVerlag

Wu, M. L., Adams, R. J., Wilson, M. R., & Haldane, S. A. (2007). ACER ConQuest: Generalised Item Response Modelling Software [Computer software]. Version 2. Camberwell, Victoria: Australian Council for Educational Research.

Wright, B. D., & Stone, M. H. (1979). Best test design. Chicago, IL: Mesa Press.