A Modular Efficiency Determination Formula for Information Retrieval Evaluations and Optimizations

Authors : Veli Özcan Budak

Pages : 209-228

Doi:10.26650/acin.1198925

View : 26 | Download : 42

Publication Date : 2024-01-02

Article Type : Research Paper

Abstract :The notion of efficiency has typically been associated with the efficiency of systems rather than users in Information Retrieval (IR) literature. In the usability literature, on the other hand, this notion is defined from a user-based perspective, corresponding to how long a user accomplishes a task. Despite this, the common aim for both has to do with the efficient use of time. This study examines the efficiency notion in the IR literature from a user-based efficiency window in the usability literature. In the present study, a modular efficiency determination formula (MEDEF) to create different efficiency indicators by focusing on IR system evaluations and optimizations from the usability perspective is proposed. The MEDEF can be thought of as an efficiency indicator creator based on both effectiveness metrics and efficiency indicators already used in IR studies. In the scope of this study, eight MEDEF-based efficiency indicators were created and compared to several baseline efficiency indicators already used in IR studies. While the study’s first aim is to reveal how consistent the MEDEF-based indicators are and whether these indicators are more successful/reliable than the baselines, the second is to set an example of the usage of efficiency indicators in evaluations of IR systems from a usability perspective. General findings from interactive user behaviour for one month show that the MEDEF-based indicators outperform the baseline indicators and further strengthen the reflections in the baseline indicators. Several usage scenarios regarding the potential of the MEDEF are also shared and discussed in the scope of the study.
Keywords : İnsan Bilgisayar Etkileşimi, Verimlilik Göstergesi, Oturum Terk Etme, Etkileşimli Bilgi Erişim