Research On Neural Signal Decoding Of Handwriting Imagery Based On Spatio-temporal Feature Fusion

Brain-computer interface(BCI)technology is a technique for recording and decoding brain signals.It translates human intentions into instructions that can be understood and processed by computers or embedded devices,thereby achieving human-machine interaction.Language plays a dominant role in human high-level intellectual activities,and language decoding-oriented BCI technology has received considerable attention and become one of the research directions of great significance.Due to the impact of the skull on signal attenuation,scalp electroencephalography(EEG)-based language decoding BCI technology has long been plagued by low decoding accuracy and poor reliability.With the breakthrough progress in BCI signal acquisition technology in recent years,neural signal decoding technology based on microelectrode arrays(MEA)has received widespread attention.By implanting MEA in the cerebral cortex,this technology can obtain neuron-level deep brain information and has been successfully applied to the decoding of handwriting imagery neural signals.This thesis focuses on the decoding of neural signals related to handwriting imagery tasks,by analyzing the MEA signals in the motor cortex area of the human brain during the process of handwriting imagery.The existing techniques typically only consider the temporal decoding of MEA signals,without fully exploiting the high spatial resolution advantage of microelectrode dense arrays.To address this issue,this study conducts research on the decoding of handwriting imagery neural signals by fusing time-domain and channel-domain features,and designs corresponding deep network structures to learn and fuse global and local features on the time and channel dimensions,achieving high-precision decoding of MEA signals for handwriting imagery tasks of English characters and sentences.Specific tasks include:This thesis presents a study on decoding handwriting imagery using high spatial resolution MEA signals through the application of spatiotemporal attention mechanisms.Firstly,a Transformer decoding model based on time series is proposed,with a dense residual Transformer encoder utilized for feature extraction.Then,spatiotemporal features are fused using cross-attention mechanisms to develop a Transformer decoding model based on spatiotemporal attention.Experimental results demonstrate that the spatiotemporal attention-based decoding method effectively leverages the spatiotemporal information in the handwriting imagery neural signals,obtaining richer feature representations through a parallel fusion architecture and significantly improving the accuracy of decoding handwriting imagery.This thesis proposes a novel approach based on spatiotemporal feature cascading for decoding handwriting imagery using MEA signals.To address the issue of the coupled spatiotemporal features in the MEA signals,the study employs the idea of separating and extracting spatiotemporal features and then cascading them back together.Specifically,a Transformer-based encoder with a dense residual architecture is utilized for the separation and extraction of spatiotemporal features.Subsequently,the separated spatiotemporal features are cascaded together to form the final feature representation for the proposed Transformer-based decoder with spatiotemporal feature cascading.Experimental results demonstrate that this approach effectively explores and analyzes the spatiotemporal information in the MEA signals for handwriting imagery,achieving more refined feature extraction and representation through a serial cascading architecture and ultimately improving the decoding accuracy of handwriting imagery.By analyzing and studying the neural signals of handwriting imagery,we can delve into the related neural mechanisms and cognitive processes in the brain,improve the accuracy and robustness of decoding handwriting imagery neural signals,and have important theoretical and practical significance for further development of handwriting imagery brain-computer interface technology,promoting a more intelligent interaction between humans and computers.