Journal of Educational Measurement Mahasarakham University

Development of Multicultural Competency in Pre-Service Teacher Students: Evidence from a University in Thailand

2024-07-24T14:37:10+07:00

In today's globalized world, the diverse cultural dynamics within classrooms necessitate that educators possess strong multicultural competencies. This study sought to enhance these competencies through a structured Multicultural Competency Training Program (MCTP). The research aimed to assess the effectiveness of the MCTP in improving the multicultural competencies of these students. This quasi-experimental study involved 60 third-year pre-service teacher students, divided into an intervention group and a control group, each with 30 students. The intervention group underwent 14 sessions of the MCTP over seven weeks, while the control group continued their regular curriculum. Multicultural competency was measured using the Multicultural Competency for Pre-Service Teacher Students Test with reliability values .972 before the intervention, immediately after, and one month later. Data were analyzed using descriptive statistics and repeated measures ANOVA. The results showed that multicultural competence scores of the intervention group after participating in the Multicultural Competency Training Program (MCTP) and at follow-up was higher than before the experiment with statistical significance at the level of p<.001, and the Multicultural competence scores of the intervention group after participating in the MCTP and at follow-up was higher than that of the control group with statistical significance at the level of p<.001. In conclusion, the MCTP effectively enhances the multicultural awareness, knowledge, and personal skills of pre-service teachers, which are essential for their professional preparation in diverse educational environments.

The Development of a Critical Thinking Skills Measurement Tool in Science and Technology for Grade 6 Students

2024-06-23T17:56:52+07:00

This research aimed to: 1) identify the components and behavioral indicators of critical thinking skills in sixth-grade students; 2) develop and examine the quality of a critical thinking skills assessment tool for sixth-grade students; and 3) establish cutoff scores for critical thinking skills using a design-based research approach. The sample consisted of 500 sixth-grade students selected through multi-stage random sampling for the main phase of data collection. The instrument was a multiple-choice test with four options per item; a correct response received 1 point, while an incorrect response received 0 points. Data were analyzed using classical test theory (CTT) and item response theory (IRT). The findings as follows:
1. The synthesis of critical thinking components resulted in five key elements: 1) problem identification, 2) hypothesis formulation, 3) data collection, 4) data analysis, and 5) conclusion drawing.
2. The quality assessment of the instrument revealed that the item-objective congruence index was 1.00 for all 40 items. The quality after using with item response theory, it was found that the discrimination parameter (a_i) was between 0.39 and 5.03, the difficulty parameter (b_i) was between -1.29 and 5.14, and the guessing parameter (c_i) was between 0.13 and 0.27.
3. Model fit indices suggested that a three-class latent model—comprising high, medium, and low critical thinking skill groups—was the most appropriate (Likelihood = -11858.198, df = 378, AIC = 23960.395, BIC = 24474.578, Adj.BIC = 24087.342, E_k = 0.846). The cutoff scores were 23.7 (between high and medium groups) and 16.1 (between medium and low groups).

The Development of a Scale to Measure Growth Mindsets for Lower Secondary School Students with the Application of Anchoring Vignettes

2024-06-22T11:27:12+07:00

This research aimed to: 1) develop a scale and a set of anchoring vignettes to measure growth mindsets for lower secondary school students, and 2) validate the developed scale by applying the anchoring vignettes as supplementary tools to aid students in interpreting and responding to the scale items effectively. The sample for this study comprised 300 lower secondary school students enrolled in the LamPang Primary Educational Service Area Offices 1 to 3. They were selected using a multi-stage random sampling method to ensure representation across the schools. The research utilized a combination of a growth mindsets scale and anchoring vignettes as its primary data collection tools. Data analysis was conducted using Cronbach's alpha coefficient to assess internal consistency, Pearson’s correlation coefficient to explore relationships between variables, confirmatory factor analysis to validate the scale structure, and item response theory to evaluate item performance and characteristics. The results of the study revealed two main findings. Firstly, the developed growth mindset scale exhibited acceptable content validity, with a content validity index ranging from .60 to 1.00. Additionally, the high Cronbach’s alpha coefficients of .91 indicated a high level of internal consistency within the scale. Furthermore, the analysis confirmed that the developed vignettes met the assumption of vignette equivalence, as respondents perceived the situations described in the vignettes uniformly. Secondly, the utilization of anchoring vignettes to mitigate response biases resulted in an improved quality of growth mindset measurement. This improvement was evidenced by enhancements in Cronbach’s alpha coefficients, item discrimination, factor structure, and construct validity. Moreover, the data analysis employing item response theory demonstrated that the incorporation of anchoring vignettes in measuring growth mindsets led to enhanced measurement precision across a wide range of proficiency levels.

Development of an Assessment Package for Physical Fitness and Serving Ability in Youth Sepak Takraw Athletes

2024-06-02T09:39:43+07:00

This research aimed to 1) investigate the components of physical fitness and serving ability assessments for youth sepak takraw athletes and 2) examine the quality of the assessment tools for physical fitness and serving ability. The study was conducted in two phases. Phase 1 investigated the components of physical fitness and serving ability. Data were collected from 50 key informants—coaches/assistant coaches, team managers, and female sepak takraw athletes in the serving position—using questionnaires. The data were analyzed using descriptive statistics, including frequency, percentage, mean, and standard deviation. Phase 2 examined the quality of the assessment tools. Data were collected from 30 female sepak takraw athletes from Bangkok Sports School and analyzed using descriptive statistics, content validity analysis via the Content Validity Index (CVI), and test–retest reliability analysis using Pearson’s correlation coefficient and the intraclass correlation coefficient (ICC). The research findings revealed the following:
1. The top four physical fitness components most significantly associated with serving were: 1) muscular strength (M = 4.88, SD = 0.33), 2) flexibility (M = 4.86, SD = 0.35), 3) balance (M = 4.72, SD = 0.61), and 4) muscular power (M = 4.64, SD = 0.53). These components reflect the serving performance of youth sepak takraw athletes in terms of speed, power, and accuracy. Serving ability can be assessed through three elements: 1) body coordination, which evaluates the completeness of the serving motion in three phases—preparation and backswing, ball contact, and force transmission; 2) serving accuracy, assessed either by the ability to hit specific target zones on the court or by hitting predefined target points; and 3) serving speed, measured by selecting the fastest attempts from five total serves.
2. The content validity index (CVI) of the assessment package for physical fitness and serving ability was 1.00 for each item. The reliability coefficients ranged from .72 to .99, and the consistency, measured by ICC, ranged from .96 to 1.00.

Development of a Training Curriculum to Enhance Teaching Standards Aligned with Formative and Summative Indicators for Quality School Curriculum Development

2024-08-13T15:22:29+07:00

การวิจัยครั้งนี้ มีความมุ่งหมายเพื่อ 1) ศึกษาความต้องการยกระดับมาตรฐานการสอนที่สอดคล้องกับตัวชี้วัดระหว่างทาง – ตัวชี้วัดปลายทาง สู่การพัฒนาหลักสูตรสถานศึกษาอย่างมีคุณภาพ 2) พัฒนาหลักสูตรฝึกอบรมฯ 3) ทดลองใช้หลักสูตรฯ และ 4) ประเมินและปรับปรุงหลักสูตรฯ โดยใช้รูปแบบการวิจัยและพัฒนา (R&D) กลุ่มตัวอย่างจำนวนทั้งสิ้น 80 คน ใช้วิธีการเลือกแบบเจาะจง เครื่องมือ ได้แก่ แบบสัมภาษณ์ ประเด็นสนทนากลุ่ม แบบทดสอบความรู้และแบบประเมินผลการใช้หลักสูตร การวิเคราะห์ข้อมูลเชิงคุณภาพวิเคราะห์เชิงเนื้อหา และข้อมูลเชิงปริมาณ วิเคราะห์ความถี่ ร้อยละ ค่าเฉลี่ย ส่วนเบี่ยงเบนมาตรฐาน และ t-test โดยสรุปผลการวิจัยได้ดังนี้
1. ผลการศึกษาความต้องการฯ ได้ 3 ประเด็น ดังนี้ 1) ควรเน้นการสร้าง/ออกแบบหลักสูตรให้สอดคล้องกับบริบทในสถานศึกษา 2) ควรเน้นเกี่ยวกับการวิเคราะห์ตัวชี้วัดต้นทาง ระหว่างทาง และปลายทางให้สอดคล้องกับบริบทในสถานศึกษา และ 3) การนำหลักสูตรสถานศึกษาไปใช้ในห้องเรียน ประกอบด้วยตัวชี้วัดการเรียนรู้ แผนการจัดการเรียนรู้ วิธีการจัดการเรียนรู้ เครื่องมือที่ใช้การวัดและประเมินผลการเรียนรู้ และวิธีการวัดและประเมินผลการเรียนรู้
2. ผลพัฒนาหลักสูตรฝึกอบรมฯ พบว่า องค์ประกอบของหลักสูตรออกเป็น 5 หมวด ได้แก่ หมวดที่ 1 ข้อมูลทั่วไป หมวดที่ 2 ข้อมูลเฉพาะของหลักสูตรฝึกอบรม หมวดที่ 3 การดำเนินการ และโครงสร้างของหลักสูตรฝึกอบรม หมวดที่ 4 หลักเกณฑ์ในการประเมินผล และหมวดที่ 5 การประเมินและปรับปรุงกระบวนการการดำเนินการของหลักสูตรฝึกอบรม ส่วนการวิพากษ์ความเหมาะสมของหลักสูตรฉบับร่าง พบว่า มีความสอดคล้องเหมาะสม ลักษณะของหลักสูตรเป็นการบรรยายให้ความรู้ และฝึกปฏิบัติโดยการวิพากษ์ตัวชี้วัดในระดับต่าง ๆ พร้อมกับส่งเสริมความรู้ความเข้าใจเกี่ยวกับ การยกระดับมาตรฐานการสอน ที่สอดคล้องกับตัวชี้วัดระหว่างทาง – ตัวชี้วัดปลายทาง สู่การพัฒนาหลักสูตรสถานศึกษาอย่างมีคุณภาพที่แท้จริง
3. การทดลองใช้หลักสูตรฝึกอบรมฯ พบว่า ผู้เข้าอบรมมีค่าเฉลี่ยของผลสัมฤทธิ์หลังการเข้ารับการฝึกอบรมสูงกว่าก่อนการเข้ารับการฝึกอบรม
4. การประเมินผลและปรับปรุงหลักสูตรฝึกอบรมฯ พบว่า ผลการประเมินการใช้หลักสูตรฝึกอบรมฯ ในภาพรวมอยู่ในระดับมากที่สุด

Development of New Normal Learning Adaptation Model for High School Students Post-COVID-19 Crisis: A Second-Order Confirmatory Factor Analysis

2024-07-14T21:17:56+07:00

This research aimed to develop and validate a measurement model of students' learning adjustment in the new normal after the COVID-19 pandemic situation among 1,295 high school students in the upper northern region of Thailand under the Office of Basic Education Commission. The research instrument was a 54-item adaptive learning questionnaire with reliability values ranging from 0.846 to 0.905. Data analysis encompassed descriptive statistics, correlation analysis, and second-order confirmatory factor analysis. Findings revealed that the developed model for measuring students' adaptive learning in the new normal following the outbreak of the COVID-19 pandemic comprised 4 components with 15 indicators: 1) Health Care in Learning, consisting of two indicators: reducing disease risk and enhancing physical resilience. 2) Self-Sentiment in Learning, consisting of four indicators: self-regulation of emotions, adaptability to learning situations, practicing mindfulness, and self-solving learning issues. 3) Roles of Learning, consisting of five indicators: learning technology, enthusiasm, self-development, utilizing media and applications, and time allocation between learning and other activities. 4) Interaction in Learning, consisting of four indicators: accepting differences among students, appropriate student behavior, teaching peers willingly, and utilizing full potential in learning. The measurement model demonstrates good fit, as evidenced by the following indices: The chi-square (²) value is 34.402 with 94 degrees of freedom (df), yielding a p-value of 0.7571. The Comparative Fit Index (CFI) is 1.000, and the Tucker-Lewis Index (TLI) is 1.002. The Root Mean Square Error of Approximation (RMSEA) is 0.000, and the Standardized Root Mean Square Residual (SRMR) is 0.011. The model's validity was confirmed, with first-order loading values ranging from 0.649 to 0.898, second-order loading values ranging from 0.752 to 0.888, and a variance of student adaptive learning of 0.152.

Construction of Career Decision-Making Self-Efficacy Scale in the Digital Age of High School Students: Application of the Graded Response Model

2024-08-27T14:51:37+07:00

The objectives of this research were to 1) construct a career decision-making self-efficacy scale for high school students in the digital age with two different test lengths: 25 items and 35 items, and 2) compare the psychometric properties of the scales using the graded response model. The sample consisted of 572 high school students from Princess Chulabhorn Science High School Buriram and Phetchaburi, selected through cluster random sampling. The research instruments were the 25-item and 35-item versions of the Career Decision-Making Self-Efficacy Scale in the digital age. Data were analyzed using frequency, percentage, mean, and the graded response model via the MULTILOG 7.03 program. The findings were as follows:
1. The Career Decision-Making Self-Efficacy Scale, based on the framework of Taylor & Betz, comprised five factors and ten indicators: (1) accurate self-appraisal, (2) gathering occupational information, (3) goal selection, (4) planning for the future, and (5) problem-solving. These factors were measured using both the 25-item and 35-item versions of the scale.
2. Regarding psychometric properties, the 25-item version yielded a higher mean slope parameter ( = 2.088) and average threshold parameters (₁= –2.944, ₂ = –1.526, ₃ = 0.171, ₄= 1.787). The test information function (TIF) was 35.021 at = 0.0, and the reliability was 0.9655. In contrast, the 35-item version had a lower mean slope parameter ( = 1.655) and threshold parameters (₁ = –2.921, ₂ = –4.733, ₃ = –0.020, ₄= 1.526). The TIF was 32.492 at = –1.4, and the reliability was 0.9650. In conclusion, the 25-item version demonstrated superior psychometric qualities—including higher slope parameters, threshold values, test information, and reliability—compared to the 35-item version.

Development of a Scientific Explanation Test on the topic of Gases for Upper Secondary School Students

2024-08-30T08:39:25+07:00

This research aimed to develop a scientifically sound test of students’ ability to construct scientific explanations on the topic of gases for upper secondary school students. The test development process comprised six steps: 1) defining the operational definition of scientific explanation in the context of gases, 2) creating a test blueprint and item specifications, 3) constructing test items and their corresponding analytic rubrics, 4) examining content validity and overall suitability, 5) finalizing the test and conducting a pilot study, and 6) evaluating the finalized test in terms of item difficulty, discrimination, reliability, and rubric quality. The results indicated that the extended-response, constructed-response test demonstrated appropriate content validity and alignment with the intended construct. Following a pilot study with 41 eleventh-grade students from a large public school in Bangkok—students who had studied the topic of gases but had not received explicit instruction in constructing scientific explanations—the test showed acceptable item difficulty (P_i = .33–.57) and discrimination indices (R_i = .22–.42). The test demonstrated satisfactory reliability (Cronbach’s alpha = .732). In addition, the analytic rubrics were of high quality, with strong inter-rater consistency (ICC = .862). Therefore, the developed test is valid and reliable for accurately assessing students’ scientific explanation skills related to the topic of gases.

A Comparison of the Quality of Measurement and Evaluation Models under the Study Plans of Sukhothai Thammathirat Open University

2024-09-21T15:32:41+07:00

This research aimed to: (1) compare the total scores obtained from examinations and activity-based assessments in Educational Plans A1 and A2 with those in Plans A1 and A3 ; (2) compare the differences in the proportion of students who passed the examination based on the grading results of Plans A1 and A2, and Plans A1 and A3; and (3) compare the concordance between grading results using raw scores (before equating) and grading results using equated scores in Plans A1 and A2, and Plans A1 and A3. This research consisted of three phases. Phase 1 involved equating the total scores from examination and activity-based assessments. Phase 2 involved comparing the differences in the proportion of students who passed the examination using the Mann–Whitney U test. Phase 3 involved comparing the consistency of grading results before and after score equating using Kappa coefficient statistics. The research findings were as follows: (1) There was no difference in the equated scores between Plans A1 and A2 and Plans A1 and A3; (2) There was no difference in the proportion of students who passed the examination under Plans A1 and A2, and Plans A1 and A3; and (3) The comparison of grading concordance before and after score equating showed strong agreement. The Kappa value for Plans A1 and A2 was 0.893, and for Plans A1 and A3 was 0.830.

Development of Thai sign language recognition model using deep learning for communication between student teachers and student with special needs: application of CNN and RNN models

2024-10-24T14:48:26+07:00

This research aims to 1) analyze the Thai sign language used by teachers to communicate with students with special needs in the classroom, and 2) develop and evaluate the performance of a Thai sign language recognition model using deep learning. The research subjects include special education teachers in schools for the deaf and volunteers proficient in sign language. The research tools include a questionnaire on the frequency of sign language sentences. Data analysis is divided into two parts: Part 1 involves analyzing the Thai sign language used by teachers to communicate with students with special needs in the classroom using mean and percentage from the questionnaire data. Part 2 involves developing and evaluating the performance of a Thai sign language recognition model using deep learning. The researcher selected four deep learning models combining CNN and RNN to develop a model that recognizes 20 Thai sign language sentences using video data recorded with volunteers. The models' performance was compared in terms of accuracy, model size, and training time, to identify the most effective model for evaluating the overall and individual sentence recognition using accuracy, precision, recall, and F1-score.
The research findings revealed that 1) the sign language used in the classroom tends to be shorter and more concise than spoken language, with reinforcement phrases being the most commonly used; 2) the development of a Thai sign language recognition model using the EfficientNetB0 combined with GRU achieved the highest performance in terms of accuracy, model size, and training time; and 3) the evaluation of the Thai sign language recognition model's performance using a test dataset showed that the model achieved an accuracy of 73%. Sentences that the model recognized well included "proud," "are you ready?" "look at the picture," and "understand," while sentences it struggled with included "beautiful work," "can you do it?" and "right or wrong."

Development of the Digital Competency Test for Bachelor’s Degree Using the Multidimensional Item Response Theory

2024-11-08T11:23:28+07:00

This research aimed to: 1) develop a digital competency model for bachelor’s degree students, 2) develop a digital competency assessment using the multidimensional item response theory (MIRT), and 3) examine the digital competency levels of undergraduate students. The study employed a research and development methodology. The sample consisted of 1,200 undergraduate students from higher education institutions, selected through a multi-stage sampling technique. The research instrument was a four-option multiple-choice digital competency test comprising 30 indicators and 240 items. Data were analyzed using descriptive statistics, item quality analysis based on MIRT, and confirmatory factor analysis (CFA). The findings revealed that the digital competency model for bachelor’s degree students consisted of three components: 1) Media, Information, and Digital Literacy, 2) Skills in Using, Developing, and Solving Problems with Digital Tools, and 3) Adaptive Digital Transformation. The test was consistent with empirical data and aligned with the between-item multidimensional 3-parameter model. A total of 211 items (87.92%) met the quality criteria, with an average difficulty of 1.44, average discrimination power of 0.24, and an average guessing parameter of 0.13. Item-fit indices ranged from 0.62 to 1.05 for OUTFIT MNSQ and 0.74 to 1.14 for INFIT MNSQ. The test demonstrated high reliability, with an EAP reliability coefficient of 0.865. The digital competency model exhibited good fit with the empirical data (= 233.011, p = .217, /df = 1.074, AGFI = 0.975, RMSEA = 0.008). Results from the competency assessment indicated that students scored highest in Digital Communication Skills and the lowest score in Computational Thinking (M = 5.99 and 2.40 respectively).

The Development of Automatic Parallel Multiple-Choice Question Generation System Using Automatic Item Generation for Distance Education

2024-12-07T08:43:51+07:00

The aims of this research were to: 1) create item models and design a data warehouse for mathematics and statistics for science and technology; 2) develop an automatic parallel multiple-choice question generation system using automatic item generation; and 3) investigate parallelism between the prototype test and computer-generated test. The research methodology was divided into three phases: Phase 1 involved item model creation and evaluation to create 42 models, alongside the design of a data warehouse to store parameter values of the created item models for all item models to share stored data. Phase 2 focused on system development in web application form to create automatic parallel multiple-choice questions, together with quality evaluation of the developed system. Phase 3 consisted of parallelism investigation by having experts evaluate the parallelism of each item between the prototype test and computer-generated test, followed by experimental testing with 30 students enrolled in Mathematics and Statistics for Science and Technology at the School of Science and Technology, Sukhothai Thammathirat Open University. The results revealed that: 1) All created item models of mathematics and statistics for science and technology passed the required criteria. 2) The automatic parallel multiple-choice question generation system successfully created parallel questions that aligned with the system's design objectives, with the overall system quality at a high level. 3) Expert evaluation confirmed that all items in both tests (the prototype and computer-generated tests) met the parallelism criteria. The experimental results showed that: (1) The prototype test had an average difficulty of 0.52, average discrimination of 0.41, variance of 0.24, mean score of 21.90, reliability of 0.89, and standard error of measurement of 2.98. (2) The computer -generated test had an average difficulty of 0.51, average discrimination of 0.40, variance of 0.25, mean score of 21.00, reliability of 0.90, and standard error of measurement of 2.84. Statistical analysis indicated no significant differences between the two test versions. Based on both expert evaluation and experimental results, it was concluded that the prototype test and computer-generated test were parallel.

The Development of Learning Design Competency for Academic Support Staff of King Mongkut's University of Technology Thonburi through Feedback Processes in Experiential Learning

2024-12-16T14:51:03+07:00

This research aimed to develop experiential learning activities integrated with feedback to enhance the learning design competency of academic support staff at King Mongkut's University of Technology Thonburi, and to investigate the competency development of those who participated in the program. The sample consisted of 31 academic support staff selected through purposive sampling. The research instruments included activity worksheets and behavioral checklist forms. Data were analyzed using mean, standard deviation, one-way repeated measures ANOVA, and effect size. The developed learning activities comprised three components: (1) self-directed learning of core concepts, (2) practice-based learning, and (3) real-world problem-based scenarios. The implementation significantly enhanced participants’ learning design competency (p < .001). Repeated measures ANOVA showed a statistically significant difference in competency scores across three time points, F(1.483, 44.487) = 898.729, p < .001. Competency scores increased in each successive assessment.

The Effects of the Process Writing Approach on Thai Essay Writing Ability of Grade 6 Students: A Time Series Experiment

2024-12-22T12:25:09+07:00

This research aims to: (1) compare the Thai essay writing ability of Grade 6 students before and after instruction using the process writing approach; (2) compare the Thai essay writing ability between students taught using the process writing approach and those taught using the traditional approach; and (3) examine the retention of Thai essay writing ability among students taught using the process writing approach. This study employed a time series quasi-experimental design. The sample consisted of sixty-nine Grade 6 students. The research instruments included: (1) lesson plans based on the process writing approach and the traditional approach; (2) a feedback manual for evaluating Thai essay writing; (3) a Thai essay writing assessment form; and (4) an assessment rubric for evaluating Thai essay writing. Data were analyzed using mean, standard deviation, dependent samples t-test, and repeated measures ANOVA. The findings were as follows: (1) The Thai essay writing ability of students significantly improved after being taught using the process writing approach (p < .05). (2) Students taught using the process writing approach demonstrated significantly higher essay writing ability than those taught using the traditional approach (p < .05). (3) Students who learned through the process writing approach retained their Thai essay writing ability over time at a statistically significant level (p < .05).

Development of a Multidimensional Mathematical Literacy Test in an Application Platform for Grade 9 Students with Feedback Integration

2025-01-21T10:50:28+07:00

This research aimed to develop a Multidimensional Mathematical Literacy Test in an application platform for Grade 9 students with integrated feedback. The sample consisted of 483 students from the Secondary Educational Service Area Office in Tak Province, selected through a multi-stage sampling method. The results from developing the test using Multidimensional Item Response Theory (MIRT) indicated that the assessment demonstrated high quality in terms of validity. A panel of five experts evaluated the content for coherence and relevance, with 30 items meeting the established criteria and achieving Item-Objective Congruence (IOC) values ranging from .60 to 1.00. The data were analyzed using MIRT with the NOHARM software, applying a multivariate cumulative normal model. The analysis revealed that the test exhibited multidimensional discrimination (MDISC) values ranging from 0.247 to 3.029, and difficulty (MDIFF) values ranging from -1.023 to 2.830. The item information function indicated that the test provided the most information for students within the moderate ability range.