The investigation of cognitive processes in mathematics learning with item response theory

Özkaya Seçil, Selcen
The importance of learning mathematics and using it in daily life is obvious. On the other hand, the results from many national and international assessment studies show that the achievement of Turkish students are very far away from the bare minimum performance. However, in the measurement and evaluation procedures of both primary and secondary educational system, there is a lack of identification of this “bare minimum” or qualitative and clear descriptors for performance levels. A great importance is dedicated to the national exam results expressed in percentage terms of the correct responses, or in total score points in weighted scale scores, but there is still no system of presenting to students their scores with descriptions of these scores in terms of levels of skills that they did or did not reach. Therefore, this study has aimed to identify the knowledge and skills required for different performance levels defined by setting cut points for the results of a 4th grade mathematics achievement test. The test was conducted in 2007-2008 academic year with 269 fourth grade students in eight different private primary schools in Istanbul. Then, in 2008-2009 academic year, a group of ten teachers of mathematics and assessment experts took part in the study for identifying the performance level descriptors for 4th grade mathematics performance. Two different methods of standard setting were used. One of the methods was based on the oneparameter model of Item Response Theory (IRT) and mostly named as Bookmark Method. The method depended on the statistical identification of the cut points on the scale for performance levels such as Below Basic, Basic, Proficient, and Advanced. The other method was a judgmental method which required the participant teachers to classify the item as carrying the characteristics of performance levels, again, as Below Basic, Basic, Proficient, and Advanced. The study revealed that the item mappings from two methods were congruent to each other. There was a hierarchical ordering in terms of skills among the performance levels. Also, the results demonstrated that understanding and computation skills were heavily characteristics of Below Basic and Basic levels, whereas, problem solving skill was reached by the students of Proficient and Advanced levels.
Citation Formats
S. Özkaya Seçil, “The investigation of cognitive processes in mathematics learning with item response theory,” Ph.D. - Doctoral Program, Middle East Technical University, 2009.