Transformer in Seizure Detection

Abstract

This thesis in the School of Biomedical Engineering at The University of Sydney focuses on the automatic epileptic seizure diagnosis of the EEG signals. Epilepsy is the most common neurological system disorder characterized by recurrent seizures. This disease has serious influences on 50 million population all over the world. The manual epileptic diagnosis is a laborious and cost-consuming process. Automatic seizure detection plays a significant role in the solution of this issue. Despite the machine learning method having made progress in this area, there still are a couple of limitations including the lack of large and well-annotated EEG datasets and the high-dimensionality and non-stability of EEG signals leading to a need of a high-performing, robust, and reliable network architecture.

In this study, we focused on automatic seizure detection based on electroencephalography (EEG) signals. Specifically, we proposed a tiny Transformer model (TTM) for this task and validated its performance on the TUH dataset. We achieved an area under the receiver operating characteristic curve (AUROC) of 92.1\% and found that input in the time-frequency domain achieved better results than input in the temporal-only or frequency-only domains. We compared our model with other advanced algorithms and found that TTM outperformed them. In the second part of our study, we evaluated the size and speed of the TTM model. We were able to reduce the model size to 85kB, with a low FLOPS value of 4.1 MFLOPS and 10,673 trainable parameters. These results suggest that the TTM model is suitable for future near-sensor computation and inference use. Overall, our study highlights the potential of TTM for automatic seizure detection and its promising applications in the field of EEG analysis.