An investigation of the validity and reliability of the speaking exam at a Turkish university

Sak, Gonca
This thesis aims to investigate the validity and reliability of the speaking exam at a Turkish University. For this study, data were obtained through questionnaires, interviews, the students’ speaking exam results, TOEFL exam results and departmental speaking exam scores. The results of the questionnaire were used to explore the face validity of the speaking exam. The interviews conducted to examine the content validity of the exam were analyzed in detail and common points from each interview were highlighted. To determine the predictive validity of the exam, Pearson Product Moment Correlation Coefficient and Simple Linear Regression Analysis were conducted. Furthermore, to investigate the construct validity of the exam correlation coefficients between speaking test scores and TOEFL subtest scores were calculated. To estimate the intra and inter-rater reliability level of the exam, correlation coefficients were calculated as well. The analysis of the results of the questionnaire indicated that the exam has satisfactory face validity. Moreover, the results of the interviews showed that the exam possesses the quality of content validity to a moderately high degree. It was found out that the speaking exam given in preparatory year education does not seem to predict the performances of the students in the departmental speaking exam. Moreover, the statistical analyses done to investigate the construct validity of the exam indicated that there are very low correlations between the speaking exam scores and the other subtests. It was discovered that the inter-rater reliability of the exam was not as satisfactory as it was expected as the inter-rater reliability of one pair was found relatively low. However, the speaking exam seemed to have satisfactory intra-rater reliability.