Evaluating the effectiveness of an online standardization platform for english as a foreign language (EFL) oral examiners

Date

2019

Author

Yıldız, Gökhan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

5
views

1
downloads

This study aimed to evaluate the effectiveness of an online standardization platform prior to speaking exams at a foundation university in Ankara. The study was conducted in December 2018 and twenty-four instructors of English participated in the study. This study investigated how consistent instructors were in face-to-face and online standardization groups in grading ten sample oral exams within their groups. Intraclass Correlation Coefficient (ICC) analysis was conducted for both groups. The results demonstrated that online and face-to-face standardization groups scored ten sample exams consistently with excellent agreement values (ICC > .90). This study also aimed to find out whether there was a significant difference between the scores of oral exams given by the raters trained in a face-to-face standardization training and the ones trained on an online standardization platform. The results of the study showed that there was no significant difference considering the abovementioned condition. Another objective of the study was to see whether there was an agreement (consistency) between instructors in the official oral examination of the school. The Kappa statistics suggested that the majority of pairs in the official examination scored students consistently with a substantial agreement value (K > .60). In addition, semi-structured interviews with the instructors from both mediums revealed that instructors appreciated the content of the training to a great extent. Instructors in the online standardization platform reported their positive feelings about the platform such as instructional design, layout and practicality of the platform, and the flexibility of time and place.

Subject Keywords

Computer-assisted instruction., Online Standardization, Standardizing Speaking Exams, Teacher Professional Development, Inter-rater Reliability (Agreement), Testing Speaking.

URI

http://etd.lib.metu.edu.tr/upload/12623640/index.pdf
https://hdl.handle.net/11511/43996

Collections

Graduate School of Natural and Applied Sciences, Thesis