Comparing Estimated and Real Item Difficulty Using Multi-Facet Rasch Analysis

Authors : Ayfer Sayin, Sebahat Gören

Pages : 440-454

Doi:10.21031/epod.1310893

View : 92 | Download : 124

Publication Date : 2023-12-31

Article Type : Research Paper

Abstract :This study aimed to compare estimated item difficulty based on expert opinion with real item difficulty based on data. For security reasons, some high-stakes tests are not pre-tested and item difficulty is estimated by teachers in classroom assessments, so it is necessary to examine the extent to which experts make accurate predictions. In this study, we developed a 12-item assessment test like the Turkish teacher certification exam. Item difficulty was estimated and compared separately based on 1165 student responses and the opinions of 12 experts. The study revealed that the experts had a good ability to estimate item difficulty for items of moderate difficulty. However, they tended to underestimate item difficulty for items categorized as medium-easy.
Keywords : test development, item difficulty, subject matter experts, multi facet Rasch