Publication details

B. Muscato, P. Bushipaka, G. Gezici, L. Passaro, F. Giannotti and T. Cucinotta. "Embracing Diversity: A Multi-Perspective Approach with Soft Labels," (to appear in) 4th Internatinoal Conference on Hybrid Human-Artificial Intelligence (HHAI 2025), June 9-13, 2025, Pisa, Italy.

Abstract

Prior studies show that adopting the annotation diversity shaped by different backgrounds and life experiences and incorporating them into the model learning, i.e. multi-perspective approach, contribute to the development of more responsible models. Thus, in this paper we propose a new framework for designing and further evaluating perspective-aware models on stance detection task, in which multiple annotators assign stances based on a controversial topic. We also share a new dataset established through obtaining both human and LLM annotations. Results show that the multi-perspective approach yields better classification performance (higher F1-scores), outperforming the traditional approaches that use a single ground-truth, while displaying lower model confidence scores, probably due to the high level of subjectivity of the stance detection task.

Download paper

Last updated on
11 April 2025