Document - Improving query and assessment quality in text-based interactive video retrieval evaluation

2023

Conference article Restricted

Improving query and assessment quality in text-based interactive video retrieval evaluation

Bailer W, Arnold R, Benz V, Coccomini D, Gkagkas A, Þór Guðmundsson G, Heller S, Þór Jónsson B, Lokoc J, Messina N, Pantelidis N, Wu J

Video retrieval

Different task interpretations are a highly undesired element in interactive video retrieval evaluations. When a participating team focuses partially on a wrong goal, the evaluation results might become partially misleading. In this paper, we propose a process for refining known-item and open-set type queries, and preparing the assessors that judge the correctness of submissions to open-set queries. Our findings from recent years reveal that a proper methodology can lead to objective query quality improvements and subjective participant satisfaction with query clarity.

Metrics

See more details

Back to previous page

Cite as

BibTeX entry

@inproceedings{oai:it.cnr:prodotti:485364,
	title = {Improving query and assessment quality in text-based interactive video retrieval evaluation},
	author = {Bailer W and Arnold R and Benz V and Coccomini D and Gkagkas A and Þór Guðmundsson G and Heller S and Þór Jónsson B and Lokoc J and Messina N and Pantelidis N and Wu J},
	doi = {10.1145/3591106.3592281},
	year = {2023}
}