Grading the Severity and Progression of Knee Osteoarthritis with Deep Learning Methods

Abstract

Knee osteoarthritis (KOA) is a highly prevalent form of arthritis and a leading cause of physical disability, given the growing aging population. To assist these KOA assessments, there is a demanding interest in computer-aided grading algorithms. To explore better KOA patterns and overcome the dataset limitation, a self-supervised multi-modal method is studied. In addition, current works in predicting the progression of KOA only produce a predicted longitude severity grade, where the visual contents are ignored. To include the predicted visual information for the future period, the generative methods are researched for comprehensive prognosis. To this end, the visual progression trajectory is involved in building a multimodal prediction network. Specifically, the major contributions of this thesis are as follows:Firstly, a novel Self-supervised Multimodal Fusion Network (S-MFN) is proposed for multimodal unsupervised knee OA grading with X-ray and magnetic resonance imaging (MRI) modalities. To this end, multimodal contrastive learning is introduced in a self-supervised manner through modalityspecific and cross-modal modelling. Secondly, an Identity-Consistent Radiographic Diffusion Network (IC-RDN) is introduced for Knee OA prognosis that predicts the X-ray images for longitude medical imaging tests. In particular, an imageto-image diffusion model backbone and an identity consistency component are introduced to generate knee joint X-rays with persisting the patients' identity information. The generated X-rays have been approved that are beneficial to the prognosis. Comprehensive experimental results on the widely used dataset, Osteoarthritis Initiative (OAI), demonstrate the effectiveness of the proposed methods, where the multimodal KOA patterns are analysed and explored under the scenarios of multi-modalities on medical imaging modalities (i.e., Xray and MRI) and clinical diagnosis results (i.e., severity assessment and visual images).