Ubiquitous and Immersive Visualisation and Interaction of Volumetric Medical Images

Abstract

Volumetric medical imaging is an indispensable data in patient management by non-invasively visualising human organ system. A wide range of research has been conducted to provide effective visualisation of medical imaging volumes, by introducing conventional 2D image slice viewing and recent 3D volume rendering techniques. They have been demonstrated in different medical imaging modalities and diverse clinical applications. The recent advances in software platforms / display technologies have been explored for their potentials to bring new capabilities and features to volume visualisation. Web technologies enable ubiquitous computing for complex 3D rendering techniques through standard Web browsing software. Mixed reality (MR) head-mounted display (HMD) technology can deliver an immersive and natural way in which humans perceive and interact with the visualisation. This thesis hypothesises that utilisation of these two emerging technologies can bring new features to medical imaging visualisation. Two medical imaging visualisation systems are proposed. Firstly, a Web-based visualisation system is proposed to allow multi-disciplined clinicians to use personal devices for collaborative visualisation. We developed medical volume visualisation features, such as multi-modal visualisation and visualising personal data, and designed a visualisation sharing mechanism in a clinical context. Secondly, MR-HMD visualisation system is introduced with a new intuitive interaction mechanism where the users can employ a tangible planar object in their physical environment to navigate and interact with virtual 3D visualisation in the same physical coordinates. We developed an algorithm to track and locate the planar object without a marker by exploiting the depth camera in the Microsoft HoloLens. We evaluated two systems by demonstrating them in clinically relevant simulated scenarios with a variety of medical imaging datasets. We also examined their computational performances.