A Diagnosis of English Grammar Ability in Matthayomseuksa 2 Students Using Computer Based Testing

Abstract

The purposes of the research were to create a diagnostic test in English grammar
ability for Mattayomseuksa 2 students; and to diagnose their English grammar ability using
computer based testing. The sample was 1,271 Mattayomseuksa 2 students in the
‘Secondary Educational Service Area 27’ for the academic year 2012 which was obtained by
using Multi-stage Random Sampling. The tools used for the research consisted of four series
of tests with different levels of approach. The first test was based on survey relating to the
English deficiencies of the students; and to identify their mis- conception about Grammar.
Here, 155 students were part of the experiment which included a completion test with 80
questions. In the Second test 147 students participated. The test was diagnostic in nature
and was derived from the previous test. There were 80 questions with four sub-divided
questions. Classical Test Theory was used to analyze the quality of the second test; in term
of discriminating power, item difficulty and reliability. In the third test 599 students
volunteered. There were 60 questions given, and Item Response Theory was the tool used
to analyze the authenticity of the test; according to a-parameter, b-parameter, and cparameter. For the fourth test we had 370 volunteer students. It was the ultimate test to
resolve all discrepancies related to the incompleteness of the aforementioned series of
tests. In this test there were 40 questions which were run on a computer program called
Diagnostic Test Program. This test has 4 sections, namely; (i) Verb (ii) Preposition (iii) Pronoun
and, (iv) Comparison. The findings are as follows:
1. The qualities of the items according to CTT were as follows:
1.1 The content validity between purposes and items was valued at 0.80 to 1.00.
1.2 The content validity between incorrect choices and mistake were
consistent at the value of 0.66 to 1.00.
1.3 The discriminating powers of the items were valued at 0.21 to 0.66 and
the difficulty items were valued at 0.28 to 0.71.
1.4 The discriminating powers of the incorrect choices from the items were
valued at 0.05 to 0.50 and the difficulty items of the incorrect choices were valued at 0.05
to 0.49.
1.5 The reliability of the test was valued at 0.864.
2. The qualities of items according to IRT were as follows:
2.1 The value of a-parameter was at 0.302 to 2.818.
2.2 The value of b-parameter was at - 2.913 to 2.976
2.3 The value of c-parameter was at 0.101 to 0.298.
3. The results of the diagnostic test with 370 students participating through the
computer program accessible to them. It was divided into four parts: Namely, (i) verb (ii)
preposition (iii) pronoun and (iv) comparison. It was discovered that 346 students were
unable to give the right answers to questions framed under comparison. It became obvious
that the number of mistakes committed were predominately in this part. Statistically, it had
the maximum percentage of 93.51. In the scenario under question based on prepositions,
336 students were found to have committed mistakes which summed up to a percentage
rate of 90.81. The third least mistakes were committed under the questions framed from
pronouns. Here, 331 students made these errors at a percentage rate of 89.46. The last
questions, under the sub title verb had errors committed by 328 students which gave this
category or part the least percentage of 88.65.
4. The efficiency of the Diagnostic Test Program which was evaluated based on
professional analysis by elite group of expert, experienced teachers and sample students
were as follows:
4.1 Accuracy of the program as per evaluation done by the elite group of
experts found that the value of Mean was at 4.43 and the Standard Deviation was at 0.54.
4.2 The accuracy of the instruction of the program was evaluated by experts
as well. The value of Mean was derived at 4.17 and the Standard Deviation was at 0.72.
4.3 The utility, feasibility, propriety and accuracy were evaluated by
experienced teachers. The aggregate of all mean was valued at 4.33 and the Standard
Deviation was valued at 0.66.
4.4 The result from the evaluation of the sample students was summed up
to a total of 4.05 and the Standard Deviation was valued at 0.79.