The Effect of Aberrant Responses on Ability Estimation in Computer Adaptive Tests

Authors : Sebahat GÖREN, Hakan KARA, Başak ERDEM KARA, Hülya KELECİOĞLU

Pages : 256-268

Doi:10.21031/epod.1067307

View : 14 | Download : 15

Publication Date : 2022-09-30

Article Type : Research Paper

Abstract :In computer adaptive test (CAT), aberrant responses caused by some factors such as lucky guesses and carelessness errors may cause significant bias in ability estimation. Correct responses resulting from lucky guesses and false responses resulting from carelessness or anxiety may reveal aberrant responses and the impact of these types of aberrant responses may cause an erroneous estimation of the examinee’s actual ability because they do not reflect the examinee’s actual knowledge. In this study, the performances of regarding ability estimation were examined comparatively in the context of CAT simulations in case of aberrant responses.Under different conditions, twelve different CAT simulations were conducted with 10 replications for each of the conditions. Correlation, RMSE, bias, and mean absolute error (MAE) values were calculated and interpreted for each condition. Results generally indicated that the 4PL IRT model provided a more efficient and robust ability estimation than the 3PL IRT model and the 4PL model increased the precision and effectiveness of the CAT applications.
Keywords : Computer adaptive tests CAT, 3PL IRT model, 4PL IRT model, aberrant responses, early mistake