Comparison of Passing Scores Determined by The Angoff Method in Different Item Samples

Authors : Hakan KARA, Sevda ÇETİN

Pages : 80-97

Doi:10.21449/ijate.699479

View : 14 | Download : 5

Publication Date : 2020-04-01

Article Type : Research Paper

Abstract :In this study, the efficiency of various random sampling methods to reduce the number of items rated by judges in an Angoff standard-setting study was examined and the methods were compared with each other. Firstly, the full-length test was formed by combining Placement Test 2012 and 2013 mathematics subsets. After then, simple random sampling insert ignore into journalissuearticles values(SRS);, content stratified insert ignore into journalissuearticles values(C-SRS);, item-difficulty stratified insert ignore into journalissuearticles values(D-SRS); and content-by-difficulty random sampling insert ignore into journalissuearticles values(CD-SRS); methods were used to constitute different length of subsets insert ignore into journalissuearticles values(30%, 40%, 50%, 70%); from the full-test. In total, 16 different study conditions insert ignore into journalissuearticles values(4 methods x 4 subsets); were investigated. In data analysis part, ANOVA analysis was conducted to examine whether minimum passing scores insert ignore into journalissuearticles values(MPSs); for the subsets were significantly different from the MPSs of the full-length test. As a follow-up analysis, RMSE and SEE insert ignore into journalissuearticles values(Standard Error of Estimation); values were calculated for each study condition. Results indicated that the estimated Angoff MPSs were significantly different from the full-test Angoff MPS insert ignore into journalissuearticles values(45.12); only in the study conditions of 30%-C-SRS, 40% C-SRS, 30% D-SRS and 30%-CD-SRS. According to RMSE values, the C-SRS method had the smallest error while the SRS method had the biggest one. Moreover, SEE examinations revealed that to achieve estimations similar to the full-test Angoff MPS insert ignore into journalissuearticles values(within one SEE);, it is sufficient to get 50% of items with the C-SRS method. C-SRS method was the more effective one compared to the others in reducing the number of items rated by judges in MPS setting studies conducted with the Angoff method.
Keywords : Standard setting, Angoff, Random sampling methods, Minimum passing scores