Document - Leveraging inter-rater agreement for classification in the presence of noisy labels

2023

Conference article Open Access

Leveraging inter-rater agreement for classification in the presence of noisy labels

Bucarelli M. S., Cassano L., Siciliano F., Mantrach A., Silvestri F.

Machine learning

In practical settings, classification datasets are obtained through a labelling process that is usually done by humans. Labels can be noisy as they are obtained by aggregating the different individual labels assigned to the same sample by multiple, and possibly disagreeing, annotators. The interrater agreement on these datasets can be measured while the underlying noise distribution to which the labels are subject is assumed to be unknown. In this work, we: (i) show how to leverage the inter-annotator statistics to estimate the noise distribution to which labels are subject; (ii) introduce methods that use the estimate of the noise distribution to learn from the noisy dataset; and (iii) establish generalization bounds in the empirical risk minimization framework that depend on the estimated quantities. We conclude the paper by providing experiments that illustrate our findings.

Source: CVPR - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3439–3448, Vancouver, CANADA, 17-24/06/2023

Metrics

Back to previous page

Cite as

BibTeX entry

@inproceedings{oai:it.cnr:prodotti:488368,
	title = {Leveraging inter-rater agreement for classification in the presence of noisy labels},
	author = {Bucarelli M. S. and Cassano L. and Siciliano F. and Mantrach A. and Silvestri F.},
	doi = {10.1109/cvpr52729.2023.00335},
	booktitle = {CVPR - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3439–3448, Vancouver, CANADA, 17-24/06/2023},
	year = {2023}
}