A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition

HU Qinwei; TAO Qing; WANG Nini; CHEN Qingzheng; WU Tenghui; ZHANG Xiaodong

doi:10.7652/xjtuxb202204020

您当前的位置：

首页 >

文章列表页 >

A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition

更新时间：2025-07-09

- A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition
- Vol. 56, Issue 4, Pages: 185-193+202(2022)
- 作者机构：
  
  1. 新疆大学机械工程学院,乌鲁木齐,830047
  2. 西安交通大学机械工程学院,西安,710049
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202204020
  CLC： TP23
- Online First：10 April 2022，
  
  Published：2022
- 稿件说明：
移动端阅览
HU Qinwei, TAO Qing, WANG Nini, et al. A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition[J]. 2022, 56(4): 185-193+202.
DOI：

HU Qinwei, TAO Qing, WANG Nini, et al. A Multi-Scale Feature Fusion Convolutional Neural Network Approach for Steady-State Visual Evoked Potential Target Recognition[J]. 2022, 56(4): 185-193+202. DOI： 10.7652/xjtuxb202204020.

摘要

针对传统稳态视觉诱发电位(SSVEP)脑电信号目标识别方法分类精度低、提取特征不充分、方法复杂且耗时等问题

提出一种基于多尺度特征融合卷积神经网络的SSVEP信号分类识别方法(SSVEP-MF)。利用小波变换将多通道SSVEP信号整合转化为二维图像作为输入样本集; 建立多尺度特征融合卷积神经网络模型(MFCNN)

该模型利用三层二维卷积核实现图像样本不同尺度特征的充分提取

构建多尺度特征融合单元对不同层级特征进行融合

并通过全连接等操作完成模型的训练; 将样本集输入到MFCNN模型中实现脑电信号特征自适应提取及端到端分类。所提SSVEP-MF方法能够充分提取信号各层级特征

实现短时间视觉刺激下SSVEP信号的有效识别

并具有较高的目标识别效率。实验结果表明

在1 s刺激时长时

相比传统功率谱密度分析方法、典型相关分析方法以及普通卷积结构方法

所提方法的识别准确率分别提升了18.57%、20.08%及7.03%

有效提高了基于稳态视觉诱发电位范式下脑机接口的信号识别性能。

Abstract

A multi-scale feature fusion convolutional neural network-based SSVEP signal classification and recognition method is proposed to solve the problems of low classification accuracy

inadequate feature extraction

complex and time-consuming methods of traditional steady-state visual evoked potential(SSVEP-MF)signal target recognition methods. Firstly

the wavelet transform is used to integrate the multi-channel SSVEP signals into two-dimensional images as the input sample set; secondly

a multi-scale feature fusion convolutional neural network model(MFCNN)is established

which uses a three-layer two-dimensional convolutional kernel to achieve sufficient extraction of features at different scales of image samples

constructs multi-scale feature fusion units to fuse features at different levels

and completes the training of the model through operations such as full connectivity; finally

the sample set is input to the MFCNN model to achieve adaptive extraction of EEG signal features and end-to-end classification. The proposed SSVEP-MF method can fully extract the features at each level of the signal

achieve effective recognition of SSVEP signals under short-time visual stimulation

and have high target recognition efficiency. The experimental results show that the recognition accuracy of the proposed method is improved by 18.57%

20.08% and 7.03%

respectively

compared with the traditional power spectral density analysis method

typical correlation analysis method and common convolutional structure method at 1 s stimulus duration

which effectively improves the signal recognition performance of brain-machine interface based on the steady-state visual evoked potential paradigm.

关键词

Keywords

references

ZAVALA S P, LÓPEZ J L M, CHICAIZA K O, et al. Review of steady state visually evoked potential brain-computer interface applications: technological analysis and classification [J]. Journal of Engineering and Applied Sciences, 2020, 15(2): 659-678.

于淑月, 李想, 于功敬, 等. 脑机接口技术的发展与展望 [J]. 计算机测量与控制, 2019, 27(10): 5-12.

YU Shuyue, LI Xiang, YU Gongjing, et al. Development and prospect of brain computer interface technology [J]. Computer Measurement Control, 2019, 27(10): 5-12.

CHAMOLA V, VINEET A, NAYYAR A, et al. Brain-computer interface-based humanoid control: a review [J]. Sensors, 2020, 20(13): 3620.

葛松, 徐晶晶, 赖舜男, 等. 脑机接口: 现状问题与展望 [J]. 生物化学与生物物理进展, 2020, 47(12): 1227-1249.

GE Song, XU Jingjing, LAI Shunnan, et al. Brain computer interface: current situation, problems and prospects [J]. Progress in Biochemistry and Biophysics, 2020, 47(12): 1227-1249.

徐光华, 张锋, 谢俊, 等. 稳态视觉诱发电位的脑机接口范式及其信号处理方法研究 [J]. 西安交通大学学报, 2015, 49(6): 1-7.

XU Guanghua, ZHANG Feng, XIE Jun, et al. Brain-computer interface paradigms and signal processing strategy for steady state visual evoked potential [J]. Journal of Xi'an Jiaotong University, 2015, 49(6): 1-7.

VIALATTE F B, MAURICE M, DAUWELS J, et al. Steady-state visually evoked potentials: focus on essential paradigms and future perspectives [J]. Progress in Neurobiology, 2010, 90(4): 418-438.

张黎明, 张小栋, 陆竹风, 等. 用于稳态视觉诱发电位特征频率提取的同步压缩短时傅里叶变换方法 [J]. 西安交通大学学报, 2017, 51(2): 20-26.

ZHANG Liming, ZHANG Xiaodong, LU Zhufeng, et al. A synchrosqueezing short-time Fourier transform method for characteristic frequency extraction of steady state visual evoked potential [J]. Journal of Xi'an Jiaotong University, 2017, 51(2): 20-26.

杜光景, 谢俊, 张玉彬, 等. 用于稳态视觉诱发电位脑机接口目标识别的深度学习方法 [J]. 西安交通大学学报, 2019, 53(11): 42-48.

DU Guangjing, XIE Jun, ZHANG Yubin, et al. A deep learning method for target recognition in steady-state visual evoked potential-based brain-computer interface [J]. Journal of Xi'an Jiaotong University, 2019, 53(11): 42-48.

ANTELIS J M, RIVERA C A, GALVIS E, et al. Detection of SSVEP based on empirical mode decomposition and power spectrum peaks analysis [J]. Biocybernetics and Biomedical Engineering, 2020, 40(3): 1010-1021.

CHEN Guang, SONG Dandan, LIAO Lejian. A multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis in the frequency domain [J]. Advances in Intelligent Systems and Computing, 2014, 215: 603-616.

WU Ting, YAN Guozheng, YANG Banghua, et al. EEG feature extraction based on wavelet packet decomposition for brain computer interface [J]. Measurement, 2008, 41(6): 618-625.

TABAR Y R, HALICI U. A novel deep learning approach for classification of EEG motor imagery signals [J]. Journal of Neural Engineering, 2017, 14(1): 016003.

SAKHAVI S, GUAN Cuntai, YAN Shuicheng. Parallel convolutional-linear neural network for motor imagery classification [C]∥Proceedings of the 2015 23rd European Signal Processing Conference. Piscataway, NJ, USA: IEEE, 2015: 2736-2740.

CECOTTI H. A time-frequency convolutional neural network for the offline classification of steady-state visual evoked potential responses [J]. Pattern Recognition Letters, 2011, 32(8): 1145-1153.

于鸿伟, 谢俊, 何柳诗, 等. 基于卷积神经网络和支持向量机的事件相关电位识别方法 [J]. 西安交通大学学报, 2021, 55(12): 47-54.

YU Hongwei, XIE Jun, HE Liushi, et al. An event-related potential recognition method based on convolutional neural network and support vector machine [J]. Journal of Xi'an Jiaotong University, 2021, 55(12): 47-54.

李玉花, 柳倩, 韦新, 等. 基于卷积神经网络的脑电信号分类 [J]. 科学技术与工程, 2020, 20(15): 6135-6140.

LI Yuhua, LIU Qian, WEI Xin, et al. Classification of electroencephalogram signals based on convolutional neural network [J]. Science Technology and Engineering, 2020, 20(15): 6135-6140.

LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.

张小栋, 郭晋, 李睿, 等. 表情驱动下脑电信号的建模仿真及分类识别 [J]. 西安交通大学学报, 2016, 50(6): 1-8.

ZHANG Xiaodong, GUO Jin, LI Rui, et al. A simulation model and pattern recognition method of electroencephalogram driven by expression [J]. Journal of Xi'an Jiaotong University, 2016, 50(6): 1-8.

于淑月, 李想, 于功敬, 等. 脑机接口技术的发展与展望 [J]. 计算机测量与控制, 2019, 27(10): 5-12.

YU Shuyue, LI Xiang, YU Gongjing, et al. Development and prospect of brain computer interface technology [J]. Computer Measurement Control, 2019, 27(10): 5-12.

陈保家, 陈学力, 沈保明, 等. CNN-LSTM深度神经网络在滚动轴承故障诊断中的应用 [J]. 西安交通大学学报, 2021, 55(6): 28-36.

CHEN Baojia, CHEN Xueli, SHEN Baoming, et al. An application of convolution neural network and long short-term memory in rolling bearing fault diagnosis [J]. Journal of Xi'an Jiaotong University, 2021, 55(6): 28-36.

张鹏. 基于卷积神经网络的多尺度遥感图像目标检测方法研究 [D]. 西安: 西安电子科技大学, 2019: 37-39.

李睿, 张小栋, 张黎明, 等. 面向假肢的场景动画稳态视觉诱发脑控方法 [J]. 西安交通大学学报, 2017, 51(1): 115-121.

LI Rui, ZHANG Xiaodong, ZHANG Liming, et al. Brain-controlled prosthesis manipulation based on scene graph-SSVEP [J]. Journal of Xi'an Jiaotong University, 2017, 51(1): 115-121.

何柳诗, 谢俊, 于鸿伟, 等. 融合眼动追踪和目标动态可调的稳态视觉诱发电位脑机接口系统设计 [J]. 西安交通大学学报, 2021, 55(10): 87-95.

HE Liushi, XIE Jun, YU Hongwei, et al. A design of brain-computer interface system for steady-state visual evoked potential integrating eye movement tracking and target dynamical adjustment [J]. Journal of Xi'an Jiaotong University, 2021, 55(10): 87-95.

MANSHOURI N, MALEKI M, KAYIKCIOGLU T. An EEG-based stereoscopic research of the PSD differences in pre and post 2D3D movies watching [J]. Biomedical Signal Processing and Control, 2020, 55: 101642.

PALM R B. Prediction as a candidate for learning deep hierarchical models of data [D]. Lyngby: Technical University of Denmark, 2012: 46-51.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Application of Mayfly Algorithm Optimized Dual-Channel Network in Gearbox Fault Diagnosis

A Fault Location Method for Hybrid Three-Terminal High Voltage Direct Current Transmission Lines

State of Health Estimation of Lithium-Ion Batteries Using a Gramian Angle Field-Convolutional Neural Network-Temporal Convolution Network Hybrid Model

Algorithms of Wavelet Transform in Ultra-High-Speed Protection for Transmission Line

Methods to Determine Generalized Basic Probability Assignment in Generalized Evidence Theory

Related Author

YU Ning

WEI Chenqian

TIAN Liyong

ZHAO Jianjun

YU Xiaohan

GAO Shuping

YANG Lili

WU Xinyu

Related Institution

College of Mechanical Engineering, Liaoning Technical University

School of Electrical Engineering, Xi'an Jiaotong University

Xi'an Key Laboratory of Electrical Equipment Condition Monitoring and Power Supply Security

College of Electrical and Control Engineering, Xi'an University of Science and Technology

Automotive Engineering Research Institute, Shaanxi Heavy Vehicle Co., Ltd.

⁰