The effect of standardisation sessions conducted before english language writing exams on inter-rater and intra-rater reliability

Download

index.pdf

Date

2019

Author

Karadenizli-Çilingir, Mahmure Nur

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

198
views

150
downloads

Reliability refers to the consistency of a measure. There are two types of reliability: (a) inter-rater reliability which means consistency between two or more markers; and (b) intra-rater reliability which refers to consistency between the assessments of the same work by the same rater at different times. There are many factors that affect rater reliability. Rater bias, educational background, rater language background and the amount of experience in teaching and/or assessment are some of them. The literature has a number of studies concerning how to eliminate, or reduce, the factors that decrease rater reliability. For instance, it discusses that using a rubric or multiple/double marking protocol could improve rater reliability. However, many in the field believed all these attempts could not yield the desired level of rater reliability per se so they suggested these practices be assisted with standardisation sessions, or in other words rater training sessions. These sessions are believed to reconcile teachers’ judgements as much as possible by providing them with clear understanding of the rubric and the criteria that teachers need to take into consideration during their assessment.

Subject Keywords

English language, English language Composition and exercises, English language Composition and exercises Study and teaching, English language Composition and exercises Study and teaching Evaluation., Keywords: Inter-rater reliability, intra-rater reliability, standardisation sessions, writing assessment, rater training

URI

http://etd.lib.metu.edu.tr/upload/12623599/index.pdf
https://hdl.handle.net/11511/43810

Collections

Graduate School of Social Sciences, Thesis

Suggestions

OpenMETU
Core

The effect of inspection error on quality and producer losses: the case of nominal-the-best type quality characteristic and rework Köksal, Gülser; Dolgun, Leman Esra; Batmaz, İnci (Inderscience Publishers, 2013-01-01) In this study, the effect of inspection error (IE) on quality and producer losses is investigated for a nominal-the-best type quality characteristic determined by a manufacturing process centre with a separate rework unit. The probability distribution and the first two moments of the resulting quality characteristic values are derived and used together with Markov chain models and simulation to study the quality and producer losses and the related statistics for various processes and gage capability levels....
The Relationship between students' preference for written feedback and improvement in writing : is the preferred one the best one? Kağıtcı, Burçin; Hatipoğlu, Çiler; Department of English Literature (2013) This study aimed to investigate a) which type of written feedback (direct feedback or use of error codes) university prep-school EFL students with elementary level of proficiency prefer to receive on their written texts, b) whether or not the (mis)match between students’ preferences and received feedback affect their level of improvement in writing, and c) to what extent the students’ previous writing experience affect their preference for the type of written feedback. In order to determine the students’ pr...
The Perceptions of pre-service ELT teachers on different modes of peer feedback and its relation to teacher efficacy / İnce, Burtay Hatice; Aksu, Meral; Department of Curriculum and Instruction (2016) The purpose of this study was to find out the perceptions of pre-service ELT teachers on different modes of feedback (i.e. written and oral peer feedback) and its relation to teacher efficacy. For this reason, a mixed-method design was used in order to answer different research questions. EFL Teacher Efficacy Scale (ETES) was used in order to find out efficacy levels of pre-service teachers before and after they were given different modes of peer feedback. E-journals and semi-structure group interviews were...
An Analysis of Turkish university level EFL learners’ pronunciation of the diphthongs and triphthongs in English Albağlar, Necmettin Anıl; Zeyrek Bozşahin, Deniz; Department of English Language Teaching (2015) The aim of this study is to investigate the development of diphthongs and triphthongs and the influence of orthography on the acquisition of the target sounds. This thesis answers three research questions regarding the pronunciation of /oʊ/, /aʊ/, /oʊǝ/ and /aʊǝ/ by Turkish EFL learners’ at different proficiency levels in English. The first research question aims to compare the pre-intermediate (PIN) and the advanced (ADV) level learners in terms of their pronunciation of the target sounds. The second one f...
THE JOINT EFFECT OF WORD FREQUENCY AND OUTPUT INTERFERENCE IN RECOGNITION MEMORY: TEST OF A MODEL PREDICTION Dedetaş, Hatice; Kılıç Özhan, Aslı; Department of Psychology (2022-5-10) Output interference (OI) is defined as a decline in memory performance throughout the test. Different models have separate explanations for OI. For item noise models, OI results from encoding during the test. When items are presented at the test, they are compared to all items in the memory trace and if a match occurs, the matched study item is updated, and if a match does not occur, a new memory trace is added. These updates during the test phase lead to confusion through the end of the test. Another hotly...

Citation Formats

M. N. Karadenizli-Çilingir, “The effect of standardisation sessions conducted before english language writing exams on inter-rater and intra-rater reliability,” Thesis (M.S.) -- Graduate School of Social Sciences. English literature., Middle East Technical University, 2019.