用于稳态视觉诱发电位目标识别的多尺度特征融合卷积神经网络方法

胡勤伟; 陶庆; 王妮妮; 陈清正; 吴腾辉; 张小栋

doi:10.7652/xjtuxb202204020

您当前的位置：

首页 >

文章列表页 >

用于稳态视觉诱发电位目标识别的多尺度特征融合卷积神经网络方法

更新时间：2025-07-09

- 用于稳态视觉诱发电位目标识别的多尺度特征融合卷积神经网络方法
- A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition
- 西安交通大学学报 2022年56卷第4期页码：185-193+202
- 作者机构：
  
  1. 新疆大学机械工程学院,乌鲁木齐,830047
  2. 西安交通大学机械工程学院,西安,710049
- 作者简介：
  
  : 2021-08-10。作者简介: 胡勤伟(1997—),男,硕士生
  陶庆(通信作者),男,教授,博士生导师。基金项目: 国家自然科学基金资助项目(51865056)
- 基金信息：
  
  : 2021-08-10。作者简介: 胡勤伟(1997—),男,硕士生;陶庆(通信作者),男,教授,博士生导师。基金项目: 国家自然科学基金资助项目(51865056);新疆维吾尔自治区区域协同创新专项(科技援疆计划)资助项目(2020E0259)
- DOI：10.7652/xjtuxb202204020
  中图分类号： TP23
- 网络首发：2022-04-10，
  
  纸质出版：2022
- 稿件说明：
移动端阅览
胡勤伟, 陶庆, 王妮妮, 等. 用于稳态视觉诱发电位目标识别的多尺度特征融合卷积神经网络方法[J]. 西安交通大学学报, 2022,56(4):185-193+202.

HU Qinwei, TAO Qing, WANG Nini, et al. A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition[J]. 2022, 56(4): 185-193+202.
胡勤伟, 陶庆, 王妮妮, 等. 用于稳态视觉诱发电位目标识别的多尺度特征融合卷积神经网络方法[J]. 西安交通大学学报, 2022,56(4):185-193+202. DOI： 10.7652/xjtuxb202204020.

HU Qinwei, TAO Qing, WANG Nini, et al. A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition[J]. 2022, 56(4): 185-193+202. DOI： 10.7652/xjtuxb202204020.

摘要

针对传统稳态视觉诱发电位(SSVEP)脑电信号目标识别方法分类精度低、提取特征不充分、方法复杂且耗时等问题

提出一种基于多尺度特征融合卷积神经网络的SSVEP信号分类识别方法(SSVEP-MF)。利用小波变换将多通道SSVEP信号整合转化为二维图像作为输入样本集; 建立多尺度特征融合卷积神经网络模型(MFCNN)

该模型利用三层二维卷积核实现图像样本不同尺度特征的充分提取

构建多尺度特征融合单元对不同层级特征进行融合

并通过全连接等操作完成模型的训练; 将样本集输入到MFCNN模型中实现脑电信号特征自适应提取及端到端分类。所提SSVEP-MF方法能够充分提取信号各层级特征

实现短时间视觉刺激下SSVEP信号的有效识别

并具有较高的目标识别效率。实验结果表明

在1 s刺激时长时

相比传统功率谱密度分析方法、典型相关分析方法以及普通卷积结构方法

所提方法的识别准确率分别提升了18.57%、20.08%及7.03%

有效提高了基于稳态视觉诱发电位范式下脑机接口的信号识别性能。

Abstract

A multi-scale feature fusion convolutional neural network-based SSVEP signal classification and recognition method is proposed to solve the problems of low classification accuracy

inadequate feature extraction

complex and time-consuming methods of traditional steady-state visual evoked potential(SSVEP-MF)signal target recognition methods. Firstly

the wavelet transform is used to integrate the multi-channel SSVEP signals into two-dimensional images as the input sample set; secondly

a multi-scale feature fusion convolutional neural network model(MFCNN)is established

which uses a three-layer two-dimensional convolutional kernel to achieve sufficient extraction of features at different scales of image samples

constructs multi-scale feature fusion units to fuse features at different levels

and completes the training of the model through operations such as full connectivity; finally

the sample set is input to the MFCNN model to achieve adaptive extraction of EEG signal features and end-to-end classification. The proposed SSVEP-MF method can fully extract the features at each level of the signal

achieve effective recognition of SSVEP signals under short-time visual stimulation

and have high target recognition efficiency. The experimental results show that the recognition accuracy of the proposed method is improved by 18.57%

20.08% and 7.03%

respectively

compared with the traditional power spectral density analysis method

typical correlation analysis method and common convolutional structure method at 1 s stimulus duration

which effectively improves the signal recognition performance of brain-machine interface based on the steady-state visual evoked potential paradigm.

关键词

Keywords

references

ZAVALA S P, LÓPEZ J L M, CHICAIZA K O, et al. Review of steady state visually evoked potential brain-computer interface applications: technological analysis and classification [J]. Journal of Engineering and Applied Sciences, 2020, 15(2): 659-678.

于淑月, 李想, 于功敬, 等. 脑机接口技术的发展与展望 [J]. 计算机测量与控制, 2019, 27(10): 5-12.

YU Shuyue, LI Xiang, YU Gongjing, et al. Development and prospect of brain computer interface technology [J]. Computer Measurement Control, 2019, 27(10): 5-12.

CHAMOLA V, VINEET A, NAYYAR A, et al. Brain-computer interface-based humanoid control: a review [J]. Sensors, 2020, 20(13): 3620.

葛松, 徐晶晶, 赖舜男, 等. 脑机接口: 现状问题与展望 [J]. 生物化学与生物物理进展, 2020, 47(12): 1227-1249.

GE Song, XU Jingjing, LAI Shunnan, et al. Brain computer interface: current situation, problems and prospects [J]. Progress in Biochemistry and Biophysics, 2020, 47(12): 1227-1249.

徐光华, 张锋, 谢俊, 等. 稳态视觉诱发电位的脑机接口范式及其信号处理方法研究 [J]. 西安交通大学学报, 2015, 49(6): 1-7.

XU Guanghua, ZHANG Feng, XIE Jun, et al. Brain-computer interface paradigms and signal processing strategy for steady state visual evoked potential [J]. Journal of Xi'an Jiaotong University, 2015, 49(6): 1-7.

VIALATTE F B, MAURICE M, DAUWELS J, et al. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives [J]. Progress in Neurobiology, 2010, 90(4): 418-438.

张黎明, 张小栋, 陆竹风, 等. 用于稳态视觉诱发电位特征频率提取的同步压缩短时傅里叶变换方法 [J]. 西安交通大学学报, 2017, 51(2): 20-26.

ZHANG Liming, ZHANG Xiaodong, LU Zhufeng, et al. A synchrosqueezing short-time Fourier transform method for characteristic frequency extraction of steady state visual evoked potential [J]. Journal of Xi'an Jiaotong University, 2017, 51(2): 20-26.

杜光景, 谢俊, 张玉彬, 等. 用于稳态视觉诱发电位脑机接口目标识别的深度学习方法 [J]. 西安交通大学学报, 2019, 53(11): 42-48.

DU Guangjing, XIE Jun, ZHANG Yubin, et al. A deep learning method for target recognition in steady-state visual evoked potential-based brain-computer interface [J]. Journal of Xi'an Jiaotong University, 2019, 53(11): 42-48.

ANTELIS J M, RIVERA C A, GALVIS E, et al. Detection of SSVEP based on empirical mode decomposition and power spectrum peaks analysis [J]. Biocybernetics and Biomedical Engineering, 2020, 40(3): 1010-1021.

CHEN Guang, SONG Dandan, LIAO Lejian. A multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis in the frequency domain [J]. Advances in Intelligent Systems and Computing, 2014, 215: 603-616.

WU Ting, YAN Guozheng, YANG Banghua, et al. EEG feature extraction based on wavelet packet decomposition for brain computer interface [J]. Measurement, 2008, 41(6): 618-625.

TABAR Y R, HALICI U. A novel deep learning approach for classification of EEG motor imagery signals [J]. Journal of Neural Engineering, 2017, 14(1): 016003.

SAKHAVI S, GUAN Cuntai, YAN Shuicheng. Parallel convolutional-linear neural network for motor imagery classification [C]∥Proceedings of the 2015 23rd European Signal Processing Conference. Piscataway, NJ, USA: IEEE, 2015: 2736-2740.

CECOTTI H. A time-frequency convolutional neural network for the offline classification of steady-state visual evoked potential responses [J]. Pattern Recognition Letters, 2011, 32(8): 1145-1153.

于鸿伟, 谢俊, 何柳诗, 等. 基于卷积神经网络和支持向量机的事件相关电位识别方法 [J]. 西安交通大学学报, 2021, 55(12): 47-54.

YU Hongwei, XIE Jun, HE Liushi, et al. An event-related potential recognition method based on convolutional neural network and support vector machine [J]. Journal of Xi'an Jiaotong University, 2021, 55(12): 47-54.

李玉花, 柳倩, 韦新, 等. 基于卷积神经网络的脑电信号分类 [J]. 科学技术与工程, 2020, 20(15): 6135-6140.

LI Yuhua, LIU Qian, WEI Xin, et al. Classification of electroencephalogram signals based on convolutional neural network [J]. Science Technology and Engineering, 2020, 20(15): 6135-6140.

LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.

张小栋, 郭晋, 李睿, 等. 表情驱动下脑电信号的建模仿真及分类识别 [J]. 西安交通大学学报, 2016, 50(6): 1-8.

ZHANG Xiaodong, GUO Jin, LI Rui, et al. A simulation model and pattern recognition method of electroencephalogram driven by expression [J]. Journal of Xi'an Jiaotong University, 2016, 50(6): 1-8.

于淑月, 李想, 于功敬, 等. 脑机接口技术的发展与展望 [J]. 计算机测量与控制, 2019, 27(10): 5-12.

YU Shuyue, LI Xiang, YU Gongjing, et al. Development and prospect of brain computer interface technology [J]. Computer Measurement Control, 2019, 27(10): 5-12.

陈保家, 陈学力, 沈保明, 等. CNN-LSTM深度神经网络在滚动轴承故障诊断中的应用 [J]. 西安交通大学学报, 2021, 55(6): 28-36.

CHEN Baojia, CHEN Xueli, SHEN Baoming, et al. An application of convolution neural network and long short-term memory in rolling bearing fault diagnosis [J]. Journal of Xi'an Jiaotong University, 2021, 55(6): 28-36.

张鹏. 基于卷积神经网络的多尺度遥感图像目标检测方法研究 [D]. 西安: 西安电子科技大学, 2019: 37-39.

李睿, 张小栋, 张黎明, 等. 面向假肢的场景动画稳态视觉诱发脑控方法 [J]. 西安交通大学学报, 2017, 51(1): 115-121.

LI Rui, ZHANG Xiaodong, ZHANG Liming, et al. Brain-controlled prosthesis manipulation based on scene graph-SSVEP [J]. Journal of Xi'an Jiaotong University, 2017, 51(1): 115-121.

何柳诗, 谢俊, 于鸿伟, 等. 融合眼动追踪和目标动态可调的稳态视觉诱发电位脑机接口系统设计 [J]. 西安交通大学学报, 2021, 55(10): 87-95.

HE Liushi, XIE Jun, YU Hongwei, et al. A design of brain-computer interface system for steady-state visual evoked potential integrating eye movement tracking and target dynamical adjustment [J]. Journal of Xi'an Jiaotong University, 2021, 55(10): 87-95.

MANSHOURI N, MALEKI M, KAYIKCIOGLU T. An EEG-based stereoscopic research of the PSD differences in pre and post 2D3D movies watching [J]. Biomedical Signal Processing and Control, 2020, 55: 101642.

PALM R B. Prediction as a candidate for learning deep hierarchical models of data [D]. Lyngby: Technical University of Denmark, 2012: 46-51.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

蜉蝣优化双通道网络在齿轮箱故障诊断中的应用

一种适用于混合三端直流输电线路的故障定位方法

采用格拉姆角场-卷积神经网络-时序卷积网络混合模型的锂离子电池健康状态估计

输电线路超高速保护中小波算法的研究

广义证据理论中的基本概率指派生成方法