Effects of Content Balancing and Item Selection Method on Ability Estimation in Computerized Adaptive Tests

Download

index.pdf

Date

2017-01-01

Author

Sahin, Alper
ÖZBAŞI, DURMUŞ

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

273
views

0
downloads

Purpose: This study aims to reveal effects of content balancing and item selection method on ability estimation in computerized adaptive tests by comparing Fisher's maximum information (FMI) and likelihood weighted information (LWI) methods. Research Methods: Four groups of examinees (250, 500, 750, 1000) and a bank of 500 items with 10 different content domains were generated through Monte Carlo simulations. Examinee ability was estimated by fixing all settings except for the item selection methods mentioned. True and estimated ability (theta) values were compared by dividing examinees into six subgroups. Moreover, the average number of items used was compared. Findings: The correlations decreased steadily as examinee theta level increased among all examinee groups when LWI was used. FMI had the same trend with the 250 and 500 examinees. Correlations for 750 examinees decreased as. level increased as well, but they were somewhat steady with FMI. For 1000 examinees, FMI was not successful in estimating examinee. accurately after. subgroup 4. Moreover, when FMI was used,. estimates had less error than LWI. The figures regarding the average items used indicated that LWI used fewer items in subgroups 1, 2, 3 and that FMI used less items in subgroups 4, 5, and 6. Implications for Research and Practice: The findings indicated that when content balancing is put into use, LWI is more suitable to estimate examinee theta for examinees between -3 and 0 and that FMI is more stable when examinee. is above 0. An item selection algorithm combining these two item selection methods is recommended. (C) 2017 Ani Publishing Ltd. All rights reserved

Subject Keywords

likelihood weighted information fisher's maximum information Estimation accuracy

URI

https://hdl.handle.net/11511/64466

Journal

EURASIAN JOURNAL OF EDUCATIONAL RESEARCH

DOI

https://doi.org/10.14689/ejer.2017.69.2

Collections

Department of Modern Languages, Article

Suggestions

OpenMETU
Core

The Effects of Test Length and Sample Size on Item Parameters in Item Response Theory Sahin, Alper; ANIL, DUYGU (2017-02-01) This study investigates the effects of sample size and test length on item-parameter estimation in test development utilizing three unidimensional dichotomous models of item response theory (IRT). For this purpose, a real language test comprised of 50 items was administered to 6,288 students. Data from this test was used to obtain data sets of three test lengths (10, 20, and 30 items) and nine different sample sizes (150, 250, 350, 500, 750, 1,000, 2,000, 3,000 and 5,000 examinees). These data sets were the...
Effects of web-based learning tool on student learning in science education: a case study Şengel, Erhan; Özden, M. Yaşar; Department of Computer Education and Instructional Technology (2005) This exploratory study provides a qualitative and quantitative report on the integration of a specific online information system into the science curriculum using authentic tasks in a class setting. It investigates how they use an online information database and tools to create meaning. An in-depth analysis was done to understand students̕ perceptions about the usability of a web-based learning tool used in science course, properties of a web-based learning tool, the quality of the content, and the structur...
Effects of different computerized adaptive testing strategies on recovery of ability Kalender, İlker; Berberoğlu, Halil Giray; Department of Secondary Science and Mathematics Education (2011) The purpose of the present study is to compare ability estimations obtained from computerized adaptive testing (CAT) procedure with the paper and pencil test administration results of Student Selection Examination (SSE) science subtest considering different ability estimation methods and test termination rules. There are two phases in the present study. In the first phase, a post-hoc simulation was conducted to find out relationships between examinee ability levels estimated by CAT and paper and pencil test...
A quantitative investigation of students' attitudes towards electronic book technology Bulur, Hatice Gonca (SAGE Publications, 2020-09-01) The purpose of this study is to analyse the factors that have an impact on technology adoption for e-books utilizing the Analytic Hierarchy Process and Multiple Regression Analysis methods. Findings indicate that perceived usefulness and ease of use are the most significant determinants in using e-books. Of key significance is that Analytic Hierarchy Process results show that consumers make pairwise comparisons, adding environmental concerns to the selection process. Recognizing the importance of all these ...
Improvements on hyperspectral classification algorithms Özdemir, Okan Bilge; Çetin, Yasemin (2013-06-28) This study investigates the effect of training set selection strategy on classification accuracy of hyperspectral images. This effect is analyzed in conjunction with three other factors, namely the use principal component analysis on the input data, and the use of spatial information and choice of classifier. Support Vector Machines (SVM) and Maximum Likelihood (ML) classifiers are used for demonstration. Meanshift segmentation and majority voting are used for inclusion of spatial information. The effect of...

Citation Formats

A. Sahin and D. ÖZBAŞI, “Effects of Content Balancing and Item Selection Method on Ability Estimation in Computerized Adaptive Tests,” EURASIAN JOURNAL OF EDUCATIONAL RESEARCH, pp. 21–36, 2017, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/64466.