A task category space for user-centric comparative multimedia search evaluations
Lokoc J., Bailer W., Barthel K. U., Gurrin C., Heller S., Jónsson B. Þ., Peska L., Rossetto L., Schoeffmann K., Vadicamo L., Vrochidis S., Wu J.
Evaluation design Task taxonomy Multimedia retrieval
In the last decade, user-centric video search competitions have facilitated the evolution of interactive video search systems. So far, these competitions focused on a small number of search task categories, with few attempts to change task category configurations. Based on our extensive experience with interactive video search contests, we have analyzed the spectrum of possible task categories and propose a list of individual axes that define a large space of possible task categories. Using this concept of category space, new user-centric video search competitions can be designed to benchmark video search systems from different perspectives. We further analyse the three task categories considered so far at the Video Browser Showdown and discuss possible (but sometimes challenging) shifts within the task category space.
Source: MMM 2022 - 28th International Conference on Multi Media Modeling, pp. 193–204, Phu Quoc, Vietnam, 06-10/06/2022
Metrics
Back to previous page
@inproceedings{oai:it.cnr:prodotti:468765,
title = {A task category space for user-centric comparative multimedia search evaluations},
author = {Lokoc J. and Bailer W. and Barthel K. U. and Gurrin C. and Heller S. and Jónsson B. Þ. and Peska L. and Rossetto L. and Schoeffmann K. and Vadicamo L. and Vrochidis S. and Wu J.},
doi = {10.1007/978-3-030-98358-1_16},
booktitle = {MMM 2022 - 28th International Conference on Multi Media Modeling, pp. 193–204, Phu Quoc, Vietnam, 06-10/06/2022},
year = {2022}
}
0000-0001-7182-7038ISTI Repository
DOIAlso available from
