CNR Author
operator:
and / or
Typology
operator:
and / or
Language
operator:
and / or
Date
operator:
and / or
Rights
operator:
and / or
2026
Journal article
Open Access
A software architecture for verifiable and explainable classification
Barbosa Raul, Rinzivillo Salvatore, Robin Jacques, Beretta Andrea, Madeira Henrique
In the context of machine learning, classification is the procedure of predicting the class to which each element of a population belongs to. Most classification functions, for real world problems, are imperfect and thus require rigorous analysis for use in safety-critical applications such as health care. This paper proposes a software architecture for improving the trustworthiness and explainability of AI-based classifiers. The architecture combines a search-based approach with machine-learned explanations and satisfiability solving, to provide an indication of classification confidence and counterfactual explanation rules that are deductively verified to be consistent with the classifier. An implementation of the proposed architecture is evaluated on a medical case study of prognosis of Acute Coronary Syndrome (ACS). The evaluation shows that the proposed architecture is consistently able to complement each individual classification with an indication of confidence and an explanation, which is formally verified for consistency with the classifier. This contributes to foster trustworthy and explainable classification.Source: MACHINE LEARNING, vol. 115 (issue 4)
DOI: 10.1007/s10994-026-07006-0
Project(s): Future Artificial Intelligence Research - Spoke 1 “Human-centered AI”, It takes two to tango: a synergistic approach to human-machine decision making, Science and technology for the explanation of AI decision making, Strengthening the Italian RI for Social Mining and Big Data Analytics
Metrics:
Barbosa Raul, Rinzivillo Salvatore, Robin Jacques, Beretta Andrea, Madeira Henrique
In the context of machine learning, classification is the procedure of predicting the class to which each element of a population belongs to. Most classification functions, for real world problems, are imperfect and thus require rigorous analysis for use in safety-critical applications such as health care. This paper proposes a software architecture for improving the trustworthiness and explainability of AI-based classifiers. The architecture combines a search-based approach with machine-learned explanations and satisfiability solving, to provide an indication of classification confidence and counterfactual explanation rules that are deductively verified to be consistent with the classifier. An implementation of the proposed architecture is evaluated on a medical case study of prognosis of Acute Coronary Syndrome (ACS). The evaluation shows that the proposed architecture is consistently able to complement each individual classification with an indication of confidence and an explanation, which is formally verified for consistency with the classifier. This contributes to foster trustworthy and explainable classification.Source: MACHINE LEARNING, vol. 115 (issue 4)
DOI: 10.1007/s10994-026-07006-0
Project(s): Future Artificial Intelligence Research - Spoke 1 “Human-centered AI”, It takes two to tango: a synergistic approach to human-machine decision making, Science and technology for the explanation of AI decision making, Strengthening the Italian RI for Social Mining and Big Data Analytics
Metrics:
See at:
CNR IRIS 
| link.springer.com | CNR IRIS 
2025
Journal article
Open Access
The explanation dialogues: an expert focus study to understand requirements towards explanations within the GDPR
State L., A. Bringas Colmenarejo, Beretta A., Ruggieri S., Turini F., Law S.
Explainable AI (XAI) provides methods to understand non-interpretable machine learning models. However, we have little knowledge about what legal experts expect from these explanations, including their legal compliance with, and value against European Union legislation. To close this gap, we present the Explanation Dialogues, an expert focus study to uncover the expectations, reasoning, and understanding of legal experts and practitioners towards XAI, with a specific focus on the European General Data Protection Regulation. The study consists of an online questionnaire and follow-up interviews, and is centered around a use-case in the credit domain. We extract both a set of hierarchical and interconnected codes using grounded theory, and present the standpoints of the participating experts towards XAI. We find that the presented explanations are hard to understand and lack information, and discuss issues that can arise from the different interests of the data controller and subject. Finally, we present a set of recommendations for developers of XAI methods, and indications of legal areas of discussion. Among others, recommendations address the presentation, choice, and content of an explanation, technical risks as well as the end-user, while we provide legal pointers to the contestability of explanations, transparency thresholds, intellectual property rights as well as the relationship between involved parties.Source: ARTIFICIAL INTELLIGENCE AND LAW
DOI: 10.1007/s10506-024-09430-w
DOI: 10.48550/arxiv.2501.05325
Project(s): NoBIAS
, XAI
Metrics:
State L., A. Bringas Colmenarejo, Beretta A., Ruggieri S., Turini F., Law S.
Explainable AI (XAI) provides methods to understand non-interpretable machine learning models. However, we have little knowledge about what legal experts expect from these explanations, including their legal compliance with, and value against European Union legislation. To close this gap, we present the Explanation Dialogues, an expert focus study to uncover the expectations, reasoning, and understanding of legal experts and practitioners towards XAI, with a specific focus on the European General Data Protection Regulation. The study consists of an online questionnaire and follow-up interviews, and is centered around a use-case in the credit domain. We extract both a set of hierarchical and interconnected codes using grounded theory, and present the standpoints of the participating experts towards XAI. We find that the presented explanations are hard to understand and lack information, and discuss issues that can arise from the different interests of the data controller and subject. Finally, we present a set of recommendations for developers of XAI methods, and indications of legal areas of discussion. Among others, recommendations address the presentation, choice, and content of an explanation, technical risks as well as the end-user, while we provide legal pointers to the contestability of explanations, transparency thresholds, intellectual property rights as well as the relationship between involved parties.Source: ARTIFICIAL INTELLIGENCE AND LAW
DOI: 10.1007/s10506-024-09430-w
DOI: 10.48550/arxiv.2501.05325
Project(s): NoBIAS
, XAI Metrics:
See at:
arXiv.org e-Print Archive | Artificial Intelligence and Law | doi.org | CNR IRIS | CNR IRIS | link.springer.com
2025
Conference article
Restricted
What tugs at your heartstrings? Exploring flow, and affect recognition through HRV, while video gaming
Sajno E., Rossi A., Beretta A., Novielli N., Pappalardo L., Riva G.
This study explores the detection of flow states through affective computing, focusing on physiological measures such as Heart Rate Variability (HRV) during Tetris gameplay. Flow, characterized by a balance between challenge and skill, is a state of complete absorption that is complex to detect. We experimented with different difficulty levels of Tetris to elicit flow states and recorded interbeat intervals using a Polar H10 chest band. We refined the experimental protocol through a pilot study: 5 Tetris levels were chosen so that participants reported a suitable variance in difficulty. During the study, 53 participants' HRV metrics were analyzed alongside self-reported flow and affective states using questionnaires. The data analysis involved classical statistical methods and machine learning algorithms to classify flow and affective states based on HRV features. SVC and RandomForest algorithms achieved high accuracy in predicting flow-related categories (48% for balance, 52% for skill, 49% for challenge), while affective states results were more various (44% for dominance, 44% for valence, 43% for arousal, using respectively BernoulliNB, GaussianNB, and ExtraTrees). The results indicate distinct HRV patterns associated with flow and affective states, suggesting that HRV is a viable indicator for flow detection. Further research is needed to validate these findings with more comprehensive experimental designs and diverse realworld applications.DOI: 10.1109/aciiw63320.2024.00045
Metrics:
Sajno E., Rossi A., Beretta A., Novielli N., Pappalardo L., Riva G.
This study explores the detection of flow states through affective computing, focusing on physiological measures such as Heart Rate Variability (HRV) during Tetris gameplay. Flow, characterized by a balance between challenge and skill, is a state of complete absorption that is complex to detect. We experimented with different difficulty levels of Tetris to elicit flow states and recorded interbeat intervals using a Polar H10 chest band. We refined the experimental protocol through a pilot study: 5 Tetris levels were chosen so that participants reported a suitable variance in difficulty. During the study, 53 participants' HRV metrics were analyzed alongside self-reported flow and affective states using questionnaires. The data analysis involved classical statistical methods and machine learning algorithms to classify flow and affective states based on HRV features. SVC and RandomForest algorithms achieved high accuracy in predicting flow-related categories (48% for balance, 52% for skill, 49% for challenge), while affective states results were more various (44% for dominance, 44% for valence, 43% for arousal, using respectively BernoulliNB, GaussianNB, and ExtraTrees). The results indicate distinct HRV patterns associated with flow and affective states, suggesting that HRV is a viable indicator for flow detection. Further research is needed to validate these findings with more comprehensive experimental designs and diverse realworld applications.DOI: 10.1109/aciiw63320.2024.00045
Metrics:
See at:
doi.org | CNR IRIS | ieeexplore.ieee.org | CNR IRIS
2025
Journal article
Open Access
AI says I’m better: evaluating the effect of AI defer on users. A study protocol
Sajno E., Beretta A., De Gaspari S., Giannotti F., Pedreschi D., Pellungrini R., Pugnana A., Pupillo C., Repetto C., Sansoni M., Villani D., Riva G.
The integration of AI into decision support systems raises concerns about overreliance and distrust. To address this, we propose an experimental protocol combining Learning to Defer (LtD)—where AI delegates decisions to humans when appropriate—and Explainable AI (XAI), which provides users with decision rationales. Our study investigates how these approaches impact human decisionmaking, particularly in high-stakes contexts. Participants will classify noisy images from ImageNet under three between-subjects conditions: Defer (AI defers to user), Defer + XAI (AI provides an explanation), and Hidden Delegation (AI involvement is concealed). Each condition will be tested in neutral and high-stakes scenarios, the latter framed through narratives emphasizing the danger of misclassification. We will assess decision accuracy and reaction times, as well as psychological measures that explore the influence of individual differences (i.e., intolerance to uncertainty and cognitive styles), and emotions (e.g., emotion regulation, and AI-related anxiety). We hypothesize that Defer may prompt more analytical thinking, improving accuracy over Hidden Delegation, while Defer + XAI may further enhance performance. In contrast, Hidden Delegation could promote reliance on intuitive processing. We expect higher accuracy and longer response times in highstakes conditions. Findings will inform the design of human-AI systems that optimize user engagement and reliability, particularly in domains like clinical decision-making.Source: ANNUAL REVIEW OF CYBERTHERAPY AND TELEMEDICINE, vol. 23, pp. 282-288
Sajno E., Beretta A., De Gaspari S., Giannotti F., Pedreschi D., Pellungrini R., Pugnana A., Pupillo C., Repetto C., Sansoni M., Villani D., Riva G.
The integration of AI into decision support systems raises concerns about overreliance and distrust. To address this, we propose an experimental protocol combining Learning to Defer (LtD)—where AI delegates decisions to humans when appropriate—and Explainable AI (XAI), which provides users with decision rationales. Our study investigates how these approaches impact human decisionmaking, particularly in high-stakes contexts. Participants will classify noisy images from ImageNet under three between-subjects conditions: Defer (AI defers to user), Defer + XAI (AI provides an explanation), and Hidden Delegation (AI involvement is concealed). Each condition will be tested in neutral and high-stakes scenarios, the latter framed through narratives emphasizing the danger of misclassification. We will assess decision accuracy and reaction times, as well as psychological measures that explore the influence of individual differences (i.e., intolerance to uncertainty and cognitive styles), and emotions (e.g., emotion regulation, and AI-related anxiety). We hypothesize that Defer may prompt more analytical thinking, improving accuracy over Hidden Delegation, while Defer + XAI may further enhance performance. In contrast, Hidden Delegation could promote reliance on intuitive processing. We expect higher accuracy and longer response times in highstakes conditions. Findings will inform the design of human-AI systems that optimize user engagement and reliability, particularly in domains like clinical decision-making.Source: ANNUAL REVIEW OF CYBERTHERAPY AND TELEMEDICINE, vol. 23, pp. 282-288
See at:
CNR IRIS | www.arctt.info | CNR IRIS
2025
Journal article
Open Access
Beyond rationality: bridging cognitive, emotional, and individual differences in XAI decision-making
Sansoni M., Beretta A., De Gaspari S., Giannotti F., Pedreschi D., Pellungrini R., Pugnana A., Pupillo C., Repetto C., Sajno E., Villani D., Riva G.
eXplainable Artificial Intelligence (XAI) aims to enhance human understanding of AI outputs, particularly in high-stakes domains. While recent research has emphasized the need for more human-centered approaches, many efforts remain technically focused, often overlooking how users’ cognitive, emotional, and dispositional characteristics shape their engagement with AI explanations. This mini-narrative review explores six key psychological factors that influence decision-making in the context of XAI. Our goal is to support the development of more human-centered systems by addressing not only what the system explains, but also how people think, feel, and behave when making decisions. We examine how cognitive elements affect the processing of explanations (such as cognitive biases and cognitive load), and emotional responses (both general and AI-directed) modulate trust and engagement. We also consider individual differences, particularly intolerance of uncertainty and decision-making styles, which shape how users evaluate and respond to AI recommendations. Together, these factors provide a psychologically grounded framework for understanding what people decide, how they process information, and why they trust or reject AI outputs. We propose that adaptive, ethically responsible XAI systems tailored to user traits can improve clarity and usability, ultimately enabling users to navigate, not avoid, uncertainty in AI-supported decision-making. This may have important practical implications for developers, users, and policymakers.Source: ANNUAL REVIEW OF CYBERTHERAPY AND TELEMEDICINE, vol. 23, pp. 12-19
Sansoni M., Beretta A., De Gaspari S., Giannotti F., Pedreschi D., Pellungrini R., Pugnana A., Pupillo C., Repetto C., Sajno E., Villani D., Riva G.
eXplainable Artificial Intelligence (XAI) aims to enhance human understanding of AI outputs, particularly in high-stakes domains. While recent research has emphasized the need for more human-centered approaches, many efforts remain technically focused, often overlooking how users’ cognitive, emotional, and dispositional characteristics shape their engagement with AI explanations. This mini-narrative review explores six key psychological factors that influence decision-making in the context of XAI. Our goal is to support the development of more human-centered systems by addressing not only what the system explains, but also how people think, feel, and behave when making decisions. We examine how cognitive elements affect the processing of explanations (such as cognitive biases and cognitive load), and emotional responses (both general and AI-directed) modulate trust and engagement. We also consider individual differences, particularly intolerance of uncertainty and decision-making styles, which shape how users evaluate and respond to AI recommendations. Together, these factors provide a psychologically grounded framework for understanding what people decide, how they process information, and why they trust or reject AI outputs. We propose that adaptive, ethically responsible XAI systems tailored to user traits can improve clarity and usability, ultimately enabling users to navigate, not avoid, uncertainty in AI-supported decision-making. This may have important practical implications for developers, users, and policymakers.Source: ANNUAL REVIEW OF CYBERTHERAPY AND TELEMEDICINE, vol. 23, pp. 12-19
See at:
CNR IRIS | www.arctt.info | CNR IRIS
2024
Journal article
Open Access
Advancing dermatological diagnostics: interpretable AI for enhanced skin lesion classification
Metta C., Beretta A., Guidotti R., Yin Y., Gallinari P., Rinzivillo S., Giannotti F.
A crucial challenge in critical settings like medical diagnosis is making deep learning models used in decision-making systems interpretable. Efforts in Explainable Artificial Intelligence (XAI) are underway to address this challenge. Yet, many XAI methods are evaluated on broad classifiers and fail to address complex, real-world issues, such as medical diagnosis. In our study, we focus on enhancing user trust and confidence in automated AI decision-making systems, particularly for diagnosing skin lesions, by tailoring an XAI method to explain an AI model’s ability to identify various skin lesion types. We generate explanations using synthetic images of skin lesions as examples and counterexamples, offering a method for practitioners to pinpoint the critical features influencing the classification outcome. A validation survey involving domain experts, novices, and laypersons has demonstrated that explanations increase trust and confidence in the automated decision system. Furthermore, our exploration of the model’s latent space reveals clear separations among the most common skin lesion classes, a distinction that likely arises from the unique characteristics of each class and could assist in correcting frequent misdiagnoses by human professionals.Source: DIAGNOSTICS, vol. 14 (issue 7)
DOI: 10.3390/diagnostics14070753
Project(s): CREXDATA, TAILOR , Future Artificial Intelligence Research, HumanE-AI-Net , XAI , SoBigData-PlusPlus
Metrics:
Metta C., Beretta A., Guidotti R., Yin Y., Gallinari P., Rinzivillo S., Giannotti F.
A crucial challenge in critical settings like medical diagnosis is making deep learning models used in decision-making systems interpretable. Efforts in Explainable Artificial Intelligence (XAI) are underway to address this challenge. Yet, many XAI methods are evaluated on broad classifiers and fail to address complex, real-world issues, such as medical diagnosis. In our study, we focus on enhancing user trust and confidence in automated AI decision-making systems, particularly for diagnosing skin lesions, by tailoring an XAI method to explain an AI model’s ability to identify various skin lesion types. We generate explanations using synthetic images of skin lesions as examples and counterexamples, offering a method for practitioners to pinpoint the critical features influencing the classification outcome. A validation survey involving domain experts, novices, and laypersons has demonstrated that explanations increase trust and confidence in the automated decision system. Furthermore, our exploration of the model’s latent space reveals clear separations among the most common skin lesion classes, a distinction that likely arises from the unique characteristics of each class and could assist in correcting frequent misdiagnoses by human professionals.Source: DIAGNOSTICS, vol. 14 (issue 7)
DOI: 10.3390/diagnostics14070753
Project(s): CREXDATA
, TAILOR , Future Artificial Intelligence Research, HumanE-AI-Net , XAI , SoBigData-PlusPlus Metrics:
See at:
Diagnostics | PubMed Central | Archivio istituzionale della Ricerca - Scuola Normale Superiore | CNR IRIS | www.mdpi.com | Archivio della Ricerca - Università di Pisa | Archivio della Ricerca - Università di Pisa | IRIS Cnr | CNR IRIS
2024
Conference article
Open Access
XAI in healthcare
Gezici G., Metta C, Beretta A., Pellungrini R., Rinzivillo S., Pedreschi D., Giannotti F.
The evolution of Explainable Artificial Intelligence (XAI) within healthcare represents a crucial turn towards more transparent, understandable, and patient-centric AI applications. The main objective is not only to increase the accuracy of AI models but also, and more importantly, to establish user trust in decision support systems through improving their interpretability. This extended abstract outlines the ongoing efforts and advancements of our lab addressing the challenges brought up by complex AI systems in healthcare domain. Currently, there are four main projects: Prostate Imaging Cancer AI, Liver Transplantation & Diabetes, Breast Cancer, and Doctor XAI, and ABELE.Source: CEUR WORKSHOP PROCEEDINGS, vol. 3825, pp. 69-73. Malmö, Sweden, 10-11/06/2024
Project(s): HumanE-AI-Net, XAI
Gezici G., Metta C, Beretta A., Pellungrini R., Rinzivillo S., Pedreschi D., Giannotti F.
The evolution of Explainable Artificial Intelligence (XAI) within healthcare represents a crucial turn towards more transparent, understandable, and patient-centric AI applications. The main objective is not only to increase the accuracy of AI models but also, and more importantly, to establish user trust in decision support systems through improving their interpretability. This extended abstract outlines the ongoing efforts and advancements of our lab addressing the challenges brought up by complex AI systems in healthcare domain. Currently, there are four main projects: Prostate Imaging Cancer AI, Liver Transplantation & Diabetes, Breast Cancer, and Doctor XAI, and ABELE.Source: CEUR WORKSHOP PROCEEDINGS, vol. 3825, pp. 69-73. Malmö, Sweden, 10-11/06/2024
Project(s): HumanE-AI-Net
, XAI
See at:
ceur-ws.org | CNR IRIS | CNR IRIS
2024
Journal article
Open Access
Towards transparent healthcare: advancing local explanation methods in Explainable Artificial Intelligence
Metta C., Beretta A., Pellungrini R., Rinzivillo S., Giannotti F.
This paper focuses on the use of local Explainable Artificial Intelligence (XAI) methods, particularly the Local Rule-Based Explanations (LORE) technique, within healthcare and medical settings. It emphasizes the critical role of interpretability and transparency in AI systems for diagnosing diseases, predicting patient outcomes, and creating personalized treatment plans. While acknowledging the complexities and inherent trade-offs between interpretability and model performance, our work underscores the significance of local XAI methods in enhancing decision-making processes in healthcare. By providing granular, case-specific insights, local XAI methods like LORE enhance physicians’ and patients’ understanding of machine learning models and their outcome. Our paper reviews significant contributions to local XAI in healthcare, highlighting its potential to improve clinical decision making, ensure fairness, and comply with regulatory standards.Source: BIOENGINEERING, vol. 11 (issue 4)
DOI: 10.3390/bioengineering11040369
Project(s): CREXDATA, TAILOR , HumanE-AI-Net , XAI , SoBigData-PlusPlus
Metrics:
Metta C., Beretta A., Pellungrini R., Rinzivillo S., Giannotti F.
This paper focuses on the use of local Explainable Artificial Intelligence (XAI) methods, particularly the Local Rule-Based Explanations (LORE) technique, within healthcare and medical settings. It emphasizes the critical role of interpretability and transparency in AI systems for diagnosing diseases, predicting patient outcomes, and creating personalized treatment plans. While acknowledging the complexities and inherent trade-offs between interpretability and model performance, our work underscores the significance of local XAI methods in enhancing decision-making processes in healthcare. By providing granular, case-specific insights, local XAI methods like LORE enhance physicians’ and patients’ understanding of machine learning models and their outcome. Our paper reviews significant contributions to local XAI in healthcare, highlighting its potential to improve clinical decision making, ensure fairness, and comply with regulatory standards.Source: BIOENGINEERING, vol. 11 (issue 4)
DOI: 10.3390/bioengineering11040369
Project(s): CREXDATA
, TAILOR , HumanE-AI-Net , XAI , SoBigData-PlusPlus Metrics:
See at:
Bioengineering | Bioengineering | CNR IRIS | www.mdpi.com | Software Heritage | IRIS Cnr | GitHub | IRIS Cnr | CNR IRIS
2023
Journal article
Open Access
Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning
Metta C, Beretta A, Guidotti R, Yin Y, Gallinari P, Rinzivillo S, Giannotti F
A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we aim at improving the trust and confidence of users towards automatic AI decision systems in the field of medical skin lesion diagnosis by customizing an existing XAI approach for explaining an AI model able to recognize different types of skin lesions. The explanation is generated through the use of synthetic exemplar and counter-exemplar images of skin lesions and our contribution offers the practitioner a way to highlight the crucial traits responsible for the classification decision. A validation survey with domain experts, beginners, and unskilled people shows that the use of explanations improves trust and confidence in the automatic decision system. Also, an analysis of the latent space adopted by the explainer unveils that some of the most frequent skin lesion classes are distinctly separated. This phenomenon may stem from the intrinsic characteristics of each class and may help resolve common misclassifications made by human experts.Source: INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS
DOI: 10.1007/s41060-023-00401-z
Project(s): TAILOR, HumanE-AI-Net , XAI , SoBigData-PlusPlus
Metrics:
Metta C, Beretta A, Guidotti R, Yin Y, Gallinari P, Rinzivillo S, Giannotti F
A key issue in critical contexts such as medical diagnosis is the interpretability of the deep learning models adopted in decision-making systems. Research in eXplainable Artificial Intelligence (XAI) is trying to solve this issue. However, often XAI approaches are only tested on generalist classifier and do not represent realistic problems such as those of medical diagnosis. In this paper, we aim at improving the trust and confidence of users towards automatic AI decision systems in the field of medical skin lesion diagnosis by customizing an existing XAI approach for explaining an AI model able to recognize different types of skin lesions. The explanation is generated through the use of synthetic exemplar and counter-exemplar images of skin lesions and our contribution offers the practitioner a way to highlight the crucial traits responsible for the classification decision. A validation survey with domain experts, beginners, and unskilled people shows that the use of explanations improves trust and confidence in the automatic decision system. Also, an analysis of the latent space adopted by the explainer unveils that some of the most frequent skin lesion classes are distinctly separated. This phenomenon may stem from the intrinsic characteristics of each class and may help resolve common misclassifications made by human experts.Source: INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS
DOI: 10.1007/s41060-023-00401-z
Project(s): TAILOR
, HumanE-AI-Net , XAI , SoBigData-PlusPlus Metrics:
See at:
International Journal of Data Science and Analytics | CNR IRIS | link.springer.com | ISTI Repository | CNR IRIS
2023
Conference article
Open Access
The explanation dialogues: understanding how legal experts reason about XAI methods
State L, Bringas Colmenarejo A, Beretta A, Ruggieri S, Turini F, Law S
The Explanation Dialogues project is an expert focus study that aims to uncover expectations, reasoning, and rules of legal experts and practitioners towards explainable artificial intelligence (XAI). We examine legal perceptions and disputes that arise in a fictional scenario that resembles a daily life situation - a bank's use of an automated decision-making (ADM) system to decide on credit allocation to individuals. Through this simulation, the study aims to provide insights into the legal value and validity of explanations of ADMs, identify potential gaps and issues that may arise in the context of compliance with European legislation, and provide guidance on how to address these shortcomings.Source: CEUR WORKSHOP PROCEEDINGS. Winterthur, Switzerland, 07-09/06/2023
Project(s): XAI
State L, Bringas Colmenarejo A, Beretta A, Ruggieri S, Turini F, Law S
The Explanation Dialogues project is an expert focus study that aims to uncover expectations, reasoning, and rules of legal experts and practitioners towards explainable artificial intelligence (XAI). We examine legal perceptions and disputes that arise in a fictional scenario that resembles a daily life situation - a bank's use of an automated decision-making (ADM) system to decide on credit allocation to individuals. Through this simulation, the study aims to provide insights into the legal value and validity of explanations of ADMs, identify potential gaps and issues that may arise in the context of compliance with European legislation, and provide guidance on how to address these shortcomings.Source: CEUR WORKSHOP PROCEEDINGS. Winterthur, Switzerland, 07-09/06/2023
Project(s): XAI
See at:
ceur-ws.org | CNR IRIS | ISTI Repository | CNR IRIS
2023
Conference article
Restricted
Interpretable data partitioning through tree-based clustering methods
Guidotti R, Landi C, Beretta A, Fadda D, Nanni M
Interpretable Data Partitioning Through Tree-Based Clustering Methods Riccardo Guidotti, Cristiano Landi, Andrea Beretta, Daniele Fadda & Mirco Nanni Conference paper First Online: 08 October 2023 311 Accesses Part of the Lecture Notes in Computer Science book series (LNAI,volume 14276) The growing interpretable machine learning research field is mainly focusing on the explanation of supervised approaches. However, also unsupervised approaches might benefit from considering interpretability aspects. While existing clustering methods only provide the assignment of records to clusters without justifying the partitioning, we propose tree-based clustering methods that offer interpretable data partitioning through a shallow decision tree. These decision trees enable easy-to-understand explanations of cluster assignments through short and understandable split conditions. The proposed methods are evaluated through experiments on synthetic and real datasets and proved to be more effective than traditional clustering approaches and interpretable ones in terms of standard evaluation measures and runtime. Finally, a case study involving human participation demonstrates the effectiveness of the interpretable clustering trees returned by the proposed method.DOI: 10.1007/978-3-031-45275-8_33
Metrics:
Guidotti R, Landi C, Beretta A, Fadda D, Nanni M
Interpretable Data Partitioning Through Tree-Based Clustering Methods Riccardo Guidotti, Cristiano Landi, Andrea Beretta, Daniele Fadda & Mirco Nanni Conference paper First Online: 08 October 2023 311 Accesses Part of the Lecture Notes in Computer Science book series (LNAI,volume 14276) The growing interpretable machine learning research field is mainly focusing on the explanation of supervised approaches. However, also unsupervised approaches might benefit from considering interpretability aspects. While existing clustering methods only provide the assignment of records to clusters without justifying the partitioning, we propose tree-based clustering methods that offer interpretable data partitioning through a shallow decision tree. These decision trees enable easy-to-understand explanations of cluster assignments through short and understandable split conditions. The proposed methods are evaluated through experiments on synthetic and real datasets and proved to be more effective than traditional clustering approaches and interpretable ones in terms of standard evaluation measures and runtime. Finally, a case study involving human participation demonstrates the effectiveness of the interpretable clustering trees returned by the proposed method.DOI: 10.1007/978-3-031-45275-8_33
Metrics:
See at:
doi.org | CNR IRIS | CNR IRIS | link.springer.com
2023
Journal article
Open Access
Co-design of human-centered, explainable AI for clinical decision support
Panigutti C, Beretta A, Fadda D, Giannotti F, Pedreschi D, Perotti A, Rinzivillo S
eXplainable AI (XAI) involves two intertwined but separate challenges: the development of techniques to extract explanations from black-box AI models and the way such explanations are presented to users, i.e., the explanation user interface. Despite its importance, the second aspect has received limited attention so far in the literature. Effective AI explanation interfaces are fundamental for allowing human decision-makers to take advantage and oversee high-risk AI systems effectively. Following an iterative design approach, we present the first cycle of prototyping-testing-redesigning of an explainable AI technique and its explanation user interface for clinical Decision Support Systems (DSS). We first present an XAI technique that meets the technical requirements of the healthcare domain: sequential, ontology-linked patient data, and multi-label classification tasks. We demonstrate its applicability to explain a clinical DSS, and we design a first prototype of an explanation user interface. Next, we test such a prototype with healthcare providers and collect their feedback with a two-fold outcome: First, we obtain evidence that explanations increase users' trust in the XAI system, and second, we obtain useful insights on the perceived deficiencies of their interaction with the system, so we can re-design a better, more human-centered explanation interface.Source: ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS, vol. 13 (issue 4)
DOI: 10.1145/3587271
Project(s): HumanE-AI-Net, XAI
Metrics:
Panigutti C, Beretta A, Fadda D, Giannotti F, Pedreschi D, Perotti A, Rinzivillo S
eXplainable AI (XAI) involves two intertwined but separate challenges: the development of techniques to extract explanations from black-box AI models and the way such explanations are presented to users, i.e., the explanation user interface. Despite its importance, the second aspect has received limited attention so far in the literature. Effective AI explanation interfaces are fundamental for allowing human decision-makers to take advantage and oversee high-risk AI systems effectively. Following an iterative design approach, we present the first cycle of prototyping-testing-redesigning of an explainable AI technique and its explanation user interface for clinical Decision Support Systems (DSS). We first present an XAI technique that meets the technical requirements of the healthcare domain: sequential, ontology-linked patient data, and multi-label classification tasks. We demonstrate its applicability to explain a clinical DSS, and we design a first prototype of an explanation user interface. Next, we test such a prototype with healthcare providers and collect their feedback with a two-fold outcome: First, we obtain evidence that explanations increase users' trust in the XAI system, and second, we obtain useful insights on the perceived deficiencies of their interaction with the system, so we can re-design a better, more human-centered explanation interface.Source: ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS, vol. 13 (issue 4)
DOI: 10.1145/3587271
Project(s): HumanE-AI-Net
, XAI Metrics:
See at:
dl.acm.org | Archivio istituzionale della Ricerca - Scuola Normale Superiore | CNR IRIS | ISTI Repository | ACM Transactions on Interactive Intelligent Systems | CNR IRIS
2022
Conference article
Open Access
Detecting addiction, anxiety, and depression by users psychometric profiles
Monreale A, Iavarone B, Rossetto E, Beretta A
Detecting and characterizing people with mental disorders is an important task that could help the work of different healthcare professionals. Sometimes, a diagnosis for specific mental disorders requires a long time, possibly causing problems because being diagnosed can give access to support groups, treatment programs, and medications that might help the patients. In this paper, we study the problem of exploiting supervised learning approaches, based on users' psychometric profiles extracted from Reddit posts, to detect users dealing with Addiction, Anxiety, and Depression disorders. The empirical evaluation shows an excellent predictive power of the psychometric profile and that features capturing the post's content are more effective for the classification task than features describing the user writing style. We achieve an accuracy of 96% using the entire psychometric profile and an accuracy of 95% when we exclude from the user profile linguistic features.DOI: 10.1145/3487553.3524918
Project(s): TAILOR, HumanE-AI-Net , SoBigData-PlusPlus
Metrics:
Monreale A, Iavarone B, Rossetto E, Beretta A
Detecting and characterizing people with mental disorders is an important task that could help the work of different healthcare professionals. Sometimes, a diagnosis for specific mental disorders requires a long time, possibly causing problems because being diagnosed can give access to support groups, treatment programs, and medications that might help the patients. In this paper, we study the problem of exploiting supervised learning approaches, based on users' psychometric profiles extracted from Reddit posts, to detect users dealing with Addiction, Anxiety, and Depression disorders. The empirical evaluation shows an excellent predictive power of the psychometric profile and that features capturing the post's content are more effective for the classification task than features describing the user writing style. We achieve an accuracy of 96% using the entire psychometric profile and an accuracy of 95% when we exclude from the user profile linguistic features.DOI: 10.1145/3487553.3524918
Project(s): TAILOR
, HumanE-AI-Net , SoBigData-PlusPlus Metrics:
See at:
ISTI Repository | dl.acm.org | CNR IRIS | CNR IRIS
2022
Conference article
Open Access
Follow the flow: a prospective on the on-line detection of flow mental state through machine learning
Sajno E, Beretta A, Novielli N, Riva G
Flow is a precious mental status for achieving high sports performance. It is defined as an emotional state with high valence and high arousal levels. However, a viable detection system that could provide information about it in real-time is not yet recognized. The prospective work presented here aims to the creation of an online flow detection framework. A supervised machine learning model will be trained to predict valence and arousal levels, both on already existing databases and freshly collected physiological data. As final result, the definition of the minimally expensive (both in terms of sensors and time) amount of data needed to predict a flow status will enable the creation of a real-time detection interface of flow.DOI: 10.1109/metroxraine54828.2022.9967605
DOI: 10.31234/osf.io/9z5pe
Metrics:
Sajno E, Beretta A, Novielli N, Riva G
Flow is a precious mental status for achieving high sports performance. It is defined as an emotional state with high valence and high arousal levels. However, a viable detection system that could provide information about it in real-time is not yet recognized. The prospective work presented here aims to the creation of an online flow detection framework. A supervised machine learning model will be trained to predict valence and arousal levels, both on already existing databases and freshly collected physiological data. As final result, the definition of the minimally expensive (both in terms of sensors and time) amount of data needed to predict a flow status will enable the creation of a real-time detection interface of flow.DOI: 10.1109/metroxraine54828.2022.9967605
DOI: 10.31234/osf.io/9z5pe
Metrics:
See at:
doi.org | CNR IRIS | ieeexplore.ieee.org | ISTI Repository | doi.org | CNR IRIS | CNR IRIS
2022
Journal article
Open Access
Assessing trustworthy AI in times of COVID-19: deep learning for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients
Allahabadi H., Amann J., Balot I., Beretta A., Binkley C., Bozenhard J., Bruneault F., Brusseau J., Candemir S., Cappellini L. A., Chakraborty S., Cherciu N., Cociancig C., Coffee M., Ek I., Espinosa-Leal L., Farina D., Fieux-Castagnet G., Frauenfelder T., Gallucci A., Giuliani G., Golda A., Van Halem I., Hildt E., Holm S., Kararigas G., Krier S. A., Kuhne U., Lizzi F., Madai V. I., Markus A. F., Masis S., Mathez E. W., Mureddu F., Neri E., Osika W., Ozols M., Panigutti C., Parent B., Pratesi F., Moreno-Sanchez P. A., Sartor G., Savardi M., Signoroni A., Sormunen H. M., Spezzatti A., Srivastava A., Stephansen A. F., Theng L. B., Tithi J. J., Tuominen J., Umbrello S., Vaccher F., Vetter D., Westerlund M., Wurth R., Zicari R. V.
This article's main contributions are twofold: 1) to demonstrate how to apply the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does "trustworthy AI" mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of pandemic. The AI system aims to help radiologists to estimate and communicate the severity of damage in a patient's lung from Chest X-rays. It has been experimentally deployed in the radiology department of the ASST Spedali Civili clinic in Brescia, Italy, since December 2020 during pandemic time. The methodology we have applied for our post-hoc assessment, called Z-Inspection®, uses sociotechnical scenarios to identify ethical, technical, and domain-specific issues in the use of the AI system in the context of the pandemic.Source: IEEE TRANSACTIONS ON TECHNOLOGY AND SOCIETY, vol. 3 (issue 4), pp. 272-289
DOI: 10.1109/tts.2022.3195114
DOI: 10.17863/cam.93306
Project(s): CompuLaw, PERISCOPE , PRECISE4Q , XAI , Wellcome Trust Sanger Institute - generic account for deposition of all core- funded research papers.
Metrics:
Allahabadi H., Amann J., Balot I., Beretta A., Binkley C., Bozenhard J., Bruneault F., Brusseau J., Candemir S., Cappellini L. A., Chakraborty S., Cherciu N., Cociancig C., Coffee M., Ek I., Espinosa-Leal L., Farina D., Fieux-Castagnet G., Frauenfelder T., Gallucci A., Giuliani G., Golda A., Van Halem I., Hildt E., Holm S., Kararigas G., Krier S. A., Kuhne U., Lizzi F., Madai V. I., Markus A. F., Masis S., Mathez E. W., Mureddu F., Neri E., Osika W., Ozols M., Panigutti C., Parent B., Pratesi F., Moreno-Sanchez P. A., Sartor G., Savardi M., Signoroni A., Sormunen H. M., Spezzatti A., Srivastava A., Stephansen A. F., Theng L. B., Tithi J. J., Tuominen J., Umbrello S., Vaccher F., Vetter D., Westerlund M., Wurth R., Zicari R. V.
This article's main contributions are twofold: 1) to demonstrate how to apply the general European Union's High-Level Expert Group's (EU HLEG) guidelines for trustworthy AI in practice for the domain of healthcare and 2) to investigate the research question of what does "trustworthy AI" mean at the time of the COVID-19 pandemic. To this end, we present the results of a post-hoc self-assessment to evaluate the trustworthiness of an AI system for predicting a multiregional score conveying the degree of lung compromise in COVID-19 patients, developed and verified by an interdisciplinary team with members from academia, public hospitals, and industry in time of pandemic. The AI system aims to help radiologists to estimate and communicate the severity of damage in a patient's lung from Chest X-rays. It has been experimentally deployed in the radiology department of the ASST Spedali Civili clinic in Brescia, Italy, since December 2020 during pandemic time. The methodology we have applied for our post-hoc assessment, called Z-Inspection®, uses sociotechnical scenarios to identify ethical, technical, and domain-specific issues in the use of the AI system in the context of the pandemic.Source: IEEE TRANSACTIONS ON TECHNOLOGY AND SOCIETY, vol. 3 (issue 4), pp. 272-289
DOI: 10.1109/tts.2022.3195114
DOI: 10.17863/cam.93306
Project(s): CompuLaw
, PERISCOPE , PRECISE4Q , XAI , Wellcome Trust Sanger Institute - generic account for deposition of all core- funded research papers. Metrics:
See at:
Copenhagen University Research Information System | IEEE Transactions on Technology and Society | Apollo | University of Copenhagen: Research | Archivio istituzionale della ricerca - Università di Brescia | CNR IRIS | Archivio Istituzionale della Ricerca (AperTO) - Università di Torino | Archivio Istituzionale della Ricerca (AperTO) - Università di Torino | IEEE Transactions on Technology and Society | IEEE Transactions on Technology and Society | IEEE Transactions on Technology and Society | Apollo | IEEE Transactions on Technology and Society | IEEE Transactions on Technology and Society | CNR IRIS | IEEE Transactions on Technology and Society | TU Delft Repository