Reconstruction of Visual Experiences from Neural Activity

Abstract

Reconstruction of visual experiences from brain activity is emerging as a key research area in cognitive neuroscience and neurotechnology, paving the way for transformative assistive technologies. While recent advances in deep learning have enabled good reconstructions from predominantly Functional Magnetic Resonance Imaging (fMRI), Electroencephalography (EEG)-based image reconstruction (IR) is particularly sought after due to its high temporal resolution and accessibility. However, current research largely relies on high-density EEG and full-bandwidth signals for IR, increasing computational cost, signal transfer requirements, latency, and hardware complexity. Furthermore, existing models show limited cross-subject and cross-dataset generalisation, often failing to transfer across individuals or datasets. This thesis addresses these challenges through a two-stage investigation. First, we examine the feasibility of efficient EEG-based IR using targeted signal processing. Frequency analysis and Generalised Eigenvalue Decomposition (GED) are used to identify frequency bands associated with visual information, followed by evaluation of reduced electrode configurations containing 8, 5, and 3 electrodes. The results show that selected frequency bands and reduced electrode sets of 8 or 5 channels retain reconstruction performance comparable to 63 or 17-electrode configurations, indicating a more resource-efficient approach. Second, we investigate the development of a foundation model for EEG-based IR to improve generalisation. The proposed architecture combines data harmonisation, vision transformers, and a Mixture of Experts to standardise data across different electrode systems and generate visual representations from EEG. While the work does not achieve full IR, it suggests a potential framework for developing generalisable, non-invasive visual decoding systems in future studies.