Deep Learning-based Radiomics Framework for Multi-Modality PET-CT Images

Abstract

Multimodal positron emission tomography - computed tomography (PET-CT) imaging is widely regarded as the imaging modality of choice for cancer management. This is because PET-CT combines the high sensitivity of PET in detecting regions of abnormal functions and the specificity of CT in depicting the underlying anatomy of where the abnormal functions are occurring.

Radiomics is an emerging research field that enables the extraction and analysis of quantitative features from medical images, providing valuable insights into the underlying pathophysiology that cannot be discerned by the naked eyes. This information is capable of assisting decision-making in clinical practice, leading to better personalised treatment planning, patient outcome prediction, and therapy response assessment.

The aim of this thesis is to propose a new deep learning-based radiomics framework for multimodal PET-CT images. The proposed framework comprises of three methods: 1) a tumour segmentation method via a self-supervision enabled false positive and false negative reduction network; 2) a constrained hierarchical multi-modality feature learning is constructed to predict the patient outcome with multimodal PET-CT images; 3) an automatic neural architecture search method to automatically find the optimal network architecture for both patient outcome prediction and tumour segmentation.

Extensive experiments have been conducted on three datasets, including one public soft-tissue sarcomas dataset, one public challenge dataset, and one in-house lung cancer data. The results demonstrated that the proposed methods obtained better performance in all tasks when compared to the state-of-the-art methods.