Using Generative Artificial Intelligence in Exams: A Research on KPSS with ChatGPT

Authors : Ali Öter, Betül Ersöz, Halil İbrahim Bülbül, Şeref Sağıroğlu

Pages : 269-274

Doi:10.24331/ijere.1414256

View : 334 | Download : 160

Publication Date : 2024-10-18

Article Type : Research Paper

Abstract :ChatGPT is a language model developed by OpenAI that uses generative artificial intelligence (GenAI) algorithms for natural language processing (NLP) tasks. It is a popular Chatbot due to its impressive ability to respond consistently to users\' various prompts. In this study, the Public Personnel Selection Examination (KPSS) questions, organized by ÖSYM (Turkish Student Selection and Placement Center) every year, were asked to ChatGPT, and its performance was evaluated. The evaluation was made based on fundamental text questions asked in the last three years (2021-2023) in Biology, History, Geography, Turkish Language Literature, and Social Studies in KPSS. This study also aims to investigate the accuracy and reliability of the questions asked to ChatGPT in the field of KPSS educational sciences. According to the research results, ChatGPT\'s KPSS knowledge and interpretation skills are significantly lower than the university graduate candidates who took the exam. Some suggestions are included for better Chatbot performance, more effective prompt engineering and ethical use of these tools.
Keywords : ChatGPT, GenAI, KPSS, LLM, Prompt Engineering