An xAI Pipeline for MRI image segmentation for brain tumour identification: from model to GUI through ethics and European regulations

This doctoral thesis presents the design, implementation and validation of an engineering pipeline for explainable artificial intelligence (XAI) in the context of brain tumour segmentation from magnetic resonance imaging (MRI). The proposed system is explicitly developed in accordance with human-centered and regulatory principles—including user technological proficiency, accountability, transparency, completeness, and information format—aligned with current European AI guidelines. The pipeline integrates state-of-the-art segmentation models with explainability tools, notably Grad-CAM, which is operationally embedded to provide visual saliency maps. In addition, the TracIn algorithm was studied and evaluated as an alternative attribution method and has been published in a separate work, although it was not integrated into the pipeline itself. Amodular graphical user interface (GUI) was developed to enable clinicians to interactively explore segmentation results, access algorithmic justifications, and interpret outputs using domain-specific visual tools. Akeyinnovationlies intheintegrationofGPT-4o,amultimodal largelanguagemodel, to generate human-readable textual explanations directly from visual segmentation outputs. The system was evaluated through a qualitative study focused on zero-shot prompting, assessing GPT-4o’s capacity to contextualize visual evidence without relying on traditional classification accuracy or quantitative interpretability metrics, and then tested with minimal modification of prompting to identify critical issues and improvements. Experimental validation was performed using the Br35H and BraTS19 datasets. The findings demonstrate the potential of multimodal LLMs, while also reinforcing the importance of human oversight in medical decision-making. Finally, the thesis addresses legal and ethical considerations for deploying high-risk AI systems in healthcare, o!ering a replicable, regulation-aligned model for transparent and explainable AI in medical imaging.