Wearable Augmentative and Alternative Communication (wAAC): A novel solution for people with complex communication needs

Abstract

Cerebral Palsy (CP) is the most common physical disability in childhood. It is a neurodevelopmental disorder that can affect a person's communication ability from a young age. Communication is a vital human right that should not be limited by a person’s disability. However, one in four people with cerebral palsy cannot speak and easily be understood. To support a person's speech and facilitate their communication capabilities, Augmentative and Alternative Communication (AAC) and technology are often used to provide people with CP, who like others with Complex Communication Needs (CCN), a mean to communication. This thesis starts with a systematic review of AAC featuring virtual reality. This, in turn, informed a novel design of an AAC device that utilised Mixed Reality (MR), Eye-Gaze Technology (EGT) and Brain and Computer Interfaces (BCI). The system was based on an MR environment provided by Microsoft HoloLens. The first generation of the prototype, a novel AAC device, adopted Pupil Labs to facilitate eye-tracking when using the device. The second generation was able to utilise an embedded eye-tracking system in HoloLens 2, whilst the first generation prototype was piloted with people with CP, and revised based on their feedback. Additionally, engineering students' feedback on the design was sought in an online survey. A survey of the usability of the first prototype was based on QUEST 2.0 and NASA-TLX and sent to engineers, people with cerebral palsy and/or their guardians. The average result for the QUEST 2.0 yielded an overall satisfaction score of 3.52 out of 5 among the participants. Findings from the NASA-TLX showed that people with cerebral palsy and/or their caregivers perceived a lower workload than the engineers. Based on the collected feedback, an AAC device was developed, that in addition to EGT and MR, also included BCI by capturing an electroencephalograph (EEG). EEG was collected using dry electrode EEG (g.Nautilus) from g.Tec. Although the Signal-to-Noise Ratio (SNR) of such a dry electrode EEG system was lower than expected, feedback from study participants who completed the experiments suggested an overall positive user experience. The result showed a high usability (accuracy=93.30%, ITR=8.55 sels/min) and acceptability (QUEST 2.0=4.30, NASA-TLX=2.14) in Eye-Gaze (EG) interaction. Datasets, from the MEDICON 2019 Scientific Challenge, were used to verify novel online and offline BCI algorithms. Nevertheless, an ongoing time synchronisation problem was identified as the major issue of this system. Furthermore, the additional offline analysis implicated that without time synchronisation errors, the device could achieve an acceptable accuracy level (the best session was 72.73%). This problem was extensively explored and potential solutions proposed for future studies.