Enhancing Predictive Models in Lung Cancer with Unstructured EHR and Multi-Modal Data Fusion

Non-small cell lung cancer (NSCLC) remains one of the leading causes of cancer-related mortality worldwide, largely due to its biological heterogeneity, late-stage diagnosis, and limited accuracy of current prognostic models. Reliable prediction of survival and treatment response is essential for personalized therapeutic decision. However, traditional approaches mostly rely on structured clinical variables and fail to capture the rich contextual informa- tion embedded in the unstructured components of electronic health records (EHR), such as free-text clinical notes, as well as in medical imaging. In this thesis, we investigate how un- structured clinical text and multimodal data fusion can be leveraged to enhance prognostic modeling in NSCLC and related pulmonary diseases, with particular attention to pulmonary embolism (PE). PE represents a significant contributor to morbidity and mortality among lung cancer patients, and its timely prediction and management are critical for improving clinical outcomes. First, we develop a domain-specific Named Entity Recognition (NER) system tailored to lung cancer, introducing a clinical ontology and fine-tuning a transformer-based model to extract relevant entities from free-text narratives. Building on these structured semantic rep- resentations, we propose a hierarchical attention-based framework that learns patient-level embeddings from clinical notes and demonstrates improved performance in overall survival (OS) and pathologic complete response (pCR) prediction compared to models using only structured data. To incorporate imaging information, we design a slice-level attention mechanism that processes chest CT scans through 2D convolutional encoders and aggregates prognostic fea- tures across slices. This architecture achieves competitive performance on publicly available NSCLC datasets and provides interpretability by highlighting clinically relevant regions. Finally, we propose a multimodal fusion framework integrating unstructured text, imag- ing, and structured EHR data. Our fusion strategies (early, late, and attention-based cross- modal integration) consistently outperform unimodal baselines across multiple prediction horizons, when applied to the INSPECT dataset for PE prognosis. Extensive experiments, ablation studies, and statistical analyses confirm the effectiveness of our approaches and highlight the complementary nature of clinical text and imaging. This thesis demonstrates that integrating unstructured EHR with multimodal deep learning significantly improves prognostic accuracy and model interpretability, paving the way toward more personalized and data-driven decision support systems in lung cancer care. Future directions include large-scale validation, integration of longitudinal and genomic information, and clinical deployment of multimodal AI systems.