融合Transformer网络与卷积神经网络的稳态运动视觉诱发电位解码方法

张焕卿; 谢俊; 杨瀚林; 杜方钊; 金志伟; 陈昱洁

doi:10.7652/xjtuxb202505005

您当前的位置：

首页 >

文章列表页 >

融合Transformer网络与卷积神经网络的稳态运动视觉诱发电位解码方法

专题　医工交叉与人机交互 | 更新时间：2025-08-26

- 融合Transformer网络与卷积神经网络的稳态运动视觉诱发电位解码方法
- Steady-State Motion Visual Evoked Potential Decoding Method Based on Integration of Transformer Network and Convolutional Neural Network
- 西安交通大学学报 2025年59卷第5期页码：44-53
- 作者机构：
  
  西安交通大学机械工程学院， 710049，西安
- 作者简介：
  
  张焕卿（1997—），男，博士生；
  谢俊（通信作者），男，副教授，博士生导师。
- 基金信息：
  
  国家自然科学基金资助项目(52365040)
- DOI：10.7652/xjtuxb202505005
  中图分类号： R318
- 收稿：2024-08-25，
  
  网络首发：2025-01-13，
  
  纸质出版：2025-05-10
- 稿件说明：
移动端阅览
张焕卿, 谢俊, 杨瀚林, 等. 融合Transformer网络与卷积神经网络的稳态运动视觉诱发电位解码方法[J]. 西安交通大学学报, 2025,59(5):44-53.

ZHANG Huanqing, XIE Jun, YANG Hanlin, et al. Steady-State Motion Visual Evoked Potential Decoding Method Based on Integration of Transformer Network and Convolutional Neural Network[J]. Journal of Xi’an Jiaotong University, 2025, 59(5): 44-53.
张焕卿, 谢俊, 杨瀚林, 等. 融合Transformer网络与卷积神经网络的稳态运动视觉诱发电位解码方法[J]. 西安交通大学学报, 2025,59(5):44-53. DOI： 10.7652/xjtuxb202505005.

ZHANG Huanqing, XIE Jun, YANG Hanlin, et al. Steady-State Motion Visual Evoked Potential Decoding Method Based on Integration of Transformer Network and Convolutional Neural Network[J]. Journal of Xi’an Jiaotong University, 2025, 59(5): 44-53. DOI： 10.7652/xjtuxb202505005.

摘要

针对卷积神经网络（CNN）在感受野有限、缺乏对全局信息的有效感知，以及在处理短时稳态运动视觉诱发电位（SSMVEP）信号时分类效果欠佳的问题，提出了一种紧凑EEGNet-Transformer（即EEGNetformer）网络。EEGNetformer网络融合了为脑电（EEG）信号识别任务而设计的通用的卷积神经网络EEGNet网络和Transformer网络的优势，有效地捕捉与处理脑电信号中的局部和全局信息，增强网络对SSMVEP特征的学习，进而实现良好的解码性能。EEGNet网络用于提取SSMVEP的局部时间和空间特征，而Transformer网络用于捕捉脑电时间序列的全局信息。在基于SSMVEP-BCI范式采集的数据基础上，开展了实验以评估EEGNetformer网络的性能。实验结果显示，当在2 s SSMVEP数据条件下，EEGNetformer网络在基于被试者内情况的平均准确率为88.9%±6.6%，在基于跨被试者情况的平均准确率为69.1%±4.3%。与传统的CNN算法相比，EEGNetformer网络的分类性能提升了4.2%~17.4%。研究内容说明，EEGNetformer网络在有效提高SSMVEP-BCI识别准确率方面具有显著优势，为进一步提升SSMVEP-BCI解码性能提供了新的研究思路。

Abstract

Addressing the issues of limited receptive fields and ineffective perception of global information in convolutional neural networks (CNNs)

as well as their suboptimal classification performance when processing short-duration steady-state motion visual evoked potentials (SSMVEP) signals

a compact EEGNet-Transformer (i.e

EEGNetformer) network is proposed. EEGNetformer network integrates the strengths of the EEGNet network (a general-purpose CNN designed for EEG recognition tasks) and the Transformer network. It efficiently captures and processes both local and global information in the electroencephalogram (EEG) signals

enhancing the network's learning of the SSMVEP features

thereby achieving an excellent decoding performance. The EEGNet network is utilized to extracts local temporal and spatial features of SSMVEP

while the Transformer network is utilized to capture global information in EEG time series. Based on data collected using the SSMVEP-BCI paradigm

experiments were conducted to evaluate the performance of the EEGNetformer network. The results showed that under the condition of 2 s SSMVEP data

EEGNetformer achieved an average accuracy of 88.9%±6.6% for within-subject conditions and 69.1%±4.3% for cross-subject conditions. Compared with traditional CNN algorithms

the classification performance of the EEGNetformer network was improved by 4.2%—17.4%. These findings demonstrate the significant advantage of EEGNetformer network in enhancing the recognition accuracy of SSMVEP-BCI systems

providing a new research direction for further enhancing SSMVEP-BCI decoding performance.

关键词

Keywords

references

WOLPAW J R . Brain-computer interfaces [M ] . Amsterdam, Holland : Elsevier , 2013 : 67 - 74 .

姜耿 , 赵春临 . 基于EEG的脑机接口发展综述 [J ] . 计算机测量与控制 , 2022 , 30 ( 7 ): 1 - 8 .

JIANG Geng , ZHAO Chunlin . A review of the development of brain-computer interface based on EEG [J ] . Computer Measurement & Control , 2022 , 30 ( 7 ): 1 - 8 .

VÄRBU K , MUHAMMAD N , MUHAMMAD Y . Past, present, and future of EEG-based BCI applications [J ] . Sensors , 2022 , 22 ( 9 ): 3331 .

BONDRE C , KAPGATE D . Brain computer interfaces using SSVEP: an overview [J ] . International Journal of Engineering & Technology , 2014 , 1 : 9 - 17 .

邱爽 , 杨帮华 , 陈小刚 , 等 . 非侵入式脑-机接口编解码技术研究进展 [J ] . 中国图象图形学报 , 2023 , 28 ( 6 ): 1543 - 1566 .

QIU Shuang , YANG Banghua , CHEN Xiaogang , et al . A survey on encoding and decoding technology of non-invasive brain-computer interface [J ] . Journal of Image and Graphics , 2023 , 28 ( 6 ): 1543 - 1566 .

KAPGATE D , KALBANDE D . A review on visual brain computer interface [C ] // Advancements of Medical Electronics . New Delhi : Springer India , 2015 : 193 - 206 .

梁栗炎 , 孔姝懿 , 张倩 , 等 . 脑机接口的范式、算法与编解码概念探讨与研究 [J ] . 信息通信技术与政策 , 2024 , 50 ( 5 ): 61 - 70 .

LIANG Liyan , KONG Shuyi , ZHANG Qian , et al . Exploration and research on paradigm, algorithm, as well as encoding and decoding concepts of brain-computer interface [J ] . Information and Communications Technology and Policy , 2024 , 50 ( 5 ): 61 - 70 .

李丽 , 陈枭宇 , 随力 . 稳态运动视觉诱发电位的诱发及在脑机接口中的应用进展 [J ] . 上海理工大学学报 , 2022 , 44 ( 1 ): 27 - 33 .

LI Li , CHEN Xiaoyu , SUI Li . Induction of steady-state motion visual evoked potential and its application in brain-computer interface [J ] . Journal of University of Shanghai for Science and Technology , 2022 , 44 ( 1 ): 27 - 33 .

XIE Jun , XU Guanghua , WANG Jing , et al . Steady-state motion visual evoked potentials produced by oscillating Newton's rings: implications for brain-computer interfaces [J ] . PLoS One , 2012 , 7 ( 6 ): e39707 .

刘贵彤 , 张志敏 , 柴晓珂 , 等 . 基于SSMVEP的脑-机接口视觉刺激探究 [J ] . 中国医疗器械杂志 , 2018 , 42 ( 5 ): 313 - 316 .

LIU Guitong , ZHANG Zhimin , CHAI Xiaoke , et al . Study of steady state motion visual evoked potential-based visual stimulation of BCI system [J ] . Chinese Journal of Medical Instrumentation , 2018 , 42 ( 5 ): 313 - 316 .

LIN Zhonglin , ZHANG Changshui , WU Wei , et al . Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs [J ] . IEEE Transactions on Biomedical Engineering , 2006 , 53 ( 12 ): 2610 - 2614 .

CHEN Xiaogang , WANG Yijun , GAO Shangkai , et al . Filter bank canonical correlation analysis for implementing a high-speed SSVEP-based brain-computer interface [J ] . Journal of Neural Engineering , 2015 , 12 ( 4 ): 046008 .

ZHANG Yue , XIE S Q , WANG He , et al . Data analytics in steady-state visual evoked potential-based brain-computer interface: a review [J ] . IEEE Sensors Journal , 2021 , 21 ( 2 ): 1124 - 1138 .

CRAIK A , HE Yongtian , CONTRERAS-VIDAL J L . Deep learning for electroencephalogram (EEG) classification tasks: a review [J ] . Journal of Neural Engineering , 2019 , 16 ( 3 ): 031001 .

XU Dongcen , TANG Fengzhen , LI Yiping , et al . An analysis of deep learning models in SSVEP-based BCI: a survey [J ] . Brain Sciences , 2023 , 13 ( 3 ): 483 .

LAWHERN V J , SOLON A J , WAYTOWICH N R , et al . EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces [J ] . Journal of Neural Engineering , 2018 , 15 ( 5 ): 056013 .

RAVI A , BENI N H , MANUEL J , et al . Comparing user-dependent and user-independent training of CNN for SSVEP BCI [J ] . Journal of Neural Engineering , 2020 , 17 ( 2 ): 026028 .

DING Wenlong , SHAN Jianhua , FANG Bin , et al . Filter bank convolutional neural network for short time-window steady-state visual evoked potential classification [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2021 , 29 : 2615 - 2624 .

PENG Zhiliang , HUANG Wei , GU Shanzhi , et al . Conformer: local features coupling global representations for visual recognition [C ] // 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway, NJ, USA : IEEE , 2021 : 357 - 366 .

HAN Kai , WANG Yunhe , CHEN Hanting , et al . A survey on vision transformer [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023 , 45 ( 1 ): 87 - 110 .

HAN Kai , XIAO An , WU Enhua , et al . Transformer in transformer [C ] // Advances in Neural Information Processing Systems . Red Hook, NY, USA : Curran Associates, Inc , 2021 : 15908 - 15919 .

MAURÍCIO J , DOMINGUES I , BERNARDINO J . Comparing vision transformers and convolutional neural networks for image classification: a literature review [J ] . Applied Sciences , 2023 , 13 ( 9 ): 5521 .

XIE Jin , ZHANG Jie , SUN Jiayao , et al . A transformer-based approach combining deep learning network and spatial-temporal information for raw EEG classification [J ] . IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2022 , 30 : 2126 - 2136 .

SONG Yonghao , JIA Xueyu , YANG Lie , et al . Transformer-based spatial-temporal feature learning for EEG decoding [EB/OL ] . ( 2021-06-11 ) [ 2024-08-02 ] . https://arxiv.org/abs/2106.11170 https://arxiv.org/abs/2106.11170 .

GUO Jiayi , CAI Qing , AN Jianpeng , et al . A Transformer based neural network for emotion recognition and visualizations of crucial EEG channels [J ] . Physica: A Statistical Mechanics and its Applications , 2022 , 603 : 127700 .

戴洋毅 , 何康 , 瑚琦 , 等 . CNN-Transformer混合模型在计算机视觉领域的研究综述 [J ] . 建模与仿真 , 2023 , 12 ( 4 ): 3657 - 3672 .

DAI Yangyi , HE Kang , HU Qi , et al . Review of CNN-Transformer hybrid model in computer vision [J ] . Modeling and Simulation , 2023 , 12 ( 4 ): 3657 - 3672 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

张角凹坑气膜孔对双层壁气膜冷却特性的影响

不同板叠材料热声热机能量转换过程数值模拟研究

扇贝仿生密封腔室内超临界二氧化碳泄漏流湍动能耗散机理研究

水蒸气掺混对氨裂解气旋流燃烧与排放特性的影响研究

EBSILON软件在热电联产系统优化中的应用综述