PICTURE - PRIN PNRR [2022–2026]

PICTURE: Pathological response AI-driven prediCTion after neoadjUvant theRapiEs in NSCLC

Grant number: P2022P3CXJ-PICTURE (CUP C53D23009280001)

Background

Lung cancer is the leading cause of cancer-related deaths worldwide, accounting for the highest mortality rates among both men and women. The most common subtype of lung cancer is non-small cell lung cancer (NSCLC), constituting approximately 85% of lung cancer cases. Currently, surgical treatment remains the mainstay of treatment for early-stage and locally advanced NSCLC. However, a notable post-surgery recurrence rate underscores the need for systemic therapy to improve cure rates. Neoadjuvant therapy (NAT) has shown potential in enhancing overall survival rates and reducing the risk of distant disease recurrence. Achieving a complete pathological response (pCR), hence the absence of tumor cells in all specimens, is a crucial endpoint of NAT. Evaluating pCR before surgical resection could guide the type of treatment, directing the aggressive surgery-based treatment only to patients who would benefit from it.

Hypothesis

The fundamental concept underlying PICTURE is that a multimodal representation of patients’ health status grasped by radiological images, histology images, cytology and molecular data, along with Electronic Health Records (EHRs) aligns consistently with the pCR. Therefore, exploiting the potential of artificial intelligence (AI) to integrate this heterogeneous medical data can provide accurate pCR prediction (Figure 1).

Figure 1. AI Fusion of medical data to predict pathological complete response.

Objectives

PICTURE pursues the following three objectives: (I) Assess the role of baseline radiology imaging, histology imaging, citology and molecular data and EHRs and their combination to predict pCR; (II) Move forward multimodal deep learning (MDL) to better process multimodal data, making the performance of AI resilient and robust to search the quantitative signature for pCR prediction; (III) Provide explanations of the decisions taken by the AI to improve trust and transparency using explainability AI (XAI) models. PICTURE has an exploratory goal as well: after developing AI models using data from patients undergoing chemoradiation or chemotherapy, it will explore how to transfer the models to predict pCR for those patients undergoing chemoimmunotherapy (Figure 2).

Figure 2. Visual abstract of the project.

The precise prediction of pCR marks a significant stride towards personalized medicine. Tailoring treatments based on individual patient characteristics ensures a more targeted and effective approach, leading to better treatment outcomes.

Consortium

• Università degli Studi di Cassino (UniCas)
• Università Campus Bio-Medico di Roma (UCBM)
• Università degli Studi di Torino (UniTo)

List of Journal Articles

• Caruso, C. M., Soda, P., & Guarrasi, V. (2026). Not another imputation method: a transformer-based model for missing values in tabular datasets. AI Open.
• Di Feola, F., Tronchin, L., Guarrasi, V., & Soda, P. (2025). Multi-scale texture loss for CT denoising with GANs. AI Open.
• Caruso, C. M., Soda, P., & Guarrasi, V. (2025). MARIA: A multimodal transformer model for incomplete healthcare data. Computers in Biology and Medicine, 196, 110843.
• Caragliano, A. N., Ruffini, F., Greco, C., Ippolito, E., Fiore, M., Tacconi, C., ... & Guarrasi, V. (2025). Doctor-in-the-Loop: An explainable, multi-view deep learning framework for predicting pathological response in non-small cell lung cancer. Image and Vision Computing, 161, 105630.
• Aksu, F., Gelardi, F., Chiti, A., & Soda, P. (2025). Multi-stage intermediate fusion for multimodal learning to classify non-small cell lung cancer subtypes from CT and PET. Pattern Recognition Letters, 193, 86-93.

List of Peer-Reviewed Conference Proceedings

• Caruso, C. M., Bruni, R., & Guarrasi, V. (2025, September). Mask-Aware Transformers Enable Robust Learning from Incomplete Volumetric Medical Imaging. In International Conference on Image Analysis and Processing (pp. 175-186). Cham: Springer Nature Switzerland.
• Caragliano, A. N., Tacconi, C., Greco, C., Nibid, L., Ippolito, E., Fiore, M., ... & Guarrasi, V. (2025, June). Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer. In 2025 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.
• Guarrasi, V., Di Feola, F., Iannello, G., Iele, I., Shen, L., Mantegna, M., ... & Soda, P. (2025, June). Medicine Without Boundaries: Generative AI for Translating Medical Data Across Modalities. In CEUR Workshop Proceedings (Vol. 4121). CEUR-WS.
• Sicilia, R., Aksu, F., Bria, A., Caragliano, A. N., Caruso, C. M., Cordelli, E., ... & Soda, P. (2025, June). Next-Gen Health: from Multimodal AI to Foundation Models. In CEUR Workshop Proceedings (Vol. 4121). CEUR-WS.
• Ayllón, E. M., Mantegna, M., Shen, L., Soda, P., Guarrasi, V., & Tortora, M. (2025, June). Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging. In 2025 IEEE 38th International Symposium on Computer-Based Medical Systems (CBMS) (pp. 375-380). IEEE.
• Paolo, D., Russo, C., Russo, G., Greco, C., Cortellini, A., Russano, M., ... & Sicilia, R. (2024, December). Pathologic Complete Response Prediction with Machine Learning Using Hierarchical Attention Feature Extraction. In International Conference on Pattern Recognition (pp. 255-267). Cham: Springer Nature Switzerland.
• Aksu, F., Cordelli, E., Gelardi, F., Chiti, A., & Soda, P. (2024, December). Enhancing NSCLC Histological Subtype Classification: A Federated Learning Approach Using Triplet Loss. In International Conference on Pattern Recognition (pp. 154-168). Cham: Springer Nature Switzerland.
• Aksu, F., Gelardi, F., Chiti, A., & Soda, P. (2024, December). Toward a multimodal deep learning approach for histological subtype classification in NSCLC. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 6327-6333). IEEE.
• Aksu, F., Bria, A., Caragliano, A. N., Caruso, C. M., Chen, W., Cordelli, E., ... & Zollo, L. (2024, May). Towards AI-driven next generation personalized healthcare and well-being. In CEUR Workshop Proceedings (Vol. 3762, pp. 360-365). CEUR-WS.
• Adornato, C., Assolito, C., Cordelli, E., Di Feola, F., Guarrasi, V., Iannello, G., ... & Tronchin, L. (2024, May). Virtual scanner: leveraging resilient generative ai for radiological imaging in the era of medical digital twins. In CEUR Workshop Proceedings (Vol. 3762, pp. 12-17). CEUR-WS.