引入深度强化学习思想的脑-机协作精密操控方法

张腾 1; 张小栋 1; 张英杰 1; 2; 陆竹风 1; 朱文静 1; 蒋永玉 1

doi:10.7652/xjtuxb202102001

您当前的位置：

首页 >

文章列表页 >

引入深度强化学习思想的脑-机协作精密操控方法

更新时间：2025-07-09

- 引入深度强化学习思想的脑-机协作精密操控方法
- A Precise Control Method for Brain-Computer Cooperation with Deep Reinforcement Learning
- 西安交通大学学报 2021年55卷第2期页码：1-9
- 作者机构：
  
  西安交通大学机械工程学院,西安,710049
- 作者简介：
- 基金信息：
  
  : 2020-07-19。作者简介: 张腾(1990—),男,博士生;张小栋(通信作者),男,教授,博士生导师。基金项目: 国家重点研发计划资助项目(2017YFB1300303)
- DOI：10.7652/xjtuxb202102001
  中图分类号： TP242
- 网络首发：2021-02-10，
  
  纸质出版：2021
- 稿件说明：
移动端阅览
张腾 1, 张小栋 1, 张英杰 1, 等. 引入深度强化学习思想的脑-机协作精密操控方法[J]. 西安交通大学学报, 2021,55(2):1-9.

A Precise Control Method for Brain-Computer Cooperation with Deep Reinforcement Learning[J]. 2021, 55(2): 1-9.
张腾 1, 张小栋 1, 张英杰 1, 等. 引入深度强化学习思想的脑-机协作精密操控方法[J]. 西安交通大学学报, 2021,55(2):1-9. DOI： 10.7652/xjtuxb202102001.

A Precise Control Method for Brain-Computer Cooperation with Deep Reinforcement Learning[J]. 2021, 55(2): 1-9. DOI： 10.7652/xjtuxb202102001.

摘要

针对精密操控中人-机之间缺乏信息双向交互

以及人的精神状态变化会严重影响肢体操控的精度和安全性等问题

提出了一种引入深度强化学习思想的脑-机协作精密操控方法。首先

结合人在上层规划与机器在精细控制上的各自优势

建立由主动操控和被动调控组成的双环路人-机信息交互机制; 其次

引入深度强化学习思想

从蒙特卡罗采样原理出发

以表征精神状态的脑电信号(EEG)作为模型的输入

以机器人速度指令作为模型的输出

推导出脑-机协作方法的数学模型; 再次

建立具有3个全连接层的精神状态感知网络

从脑-机接口系统的实时监测计算机内存中提取最后1 000 ms的EEG作为输入信号

设计开发脑-机协作精密操控算法。最后

创建轨迹跟踪虚拟环境和任务场景

对脑-机协作精密操控方法进行了实验验证

结果表明:该方法在轨迹跟踪任务的控制精度和完成时间指标上均得到了提高; 相较于传统方法

该方法的平均轨迹跟踪精度和完成时间指标分别提高了36.55%和22.81%。

Abstract

A precise control method for brain-computer cooperation with deep reinforcement learning is proposed to solve the problem that the lack of bidirectional information interaction between human and computer and the change of mental state in precise control seriously affect the precision and safety of limb control. First of all

combining the advantages of human in global planning and machine in fine control

a ‘double-loop’ information interaction mechanism composed of active control loop and passive control loop is established. Secondly

the idea of deep reinforcement learning is introduced

and a mathematical model of brain-computer cooperation is derived based on the Monte Carlo sampling principle

with the electroencephalogram(EEG)representing mental state feature as the input of the model and the robot speed instruction as the output. Thirdly

a mental state perception network with three fully connected layers is established

and the EEG of the last 1 000 ms in the real-time monitoring computer memory of brain-machine interface system is extracted as input signal

then a precise brain-computer cooperation algorithm is designed and developed. Finally

a virtual environment and task scene for trajectory tracking is created

and the precise brain-computer cooperation method is experimentally verified. Results and a comparison with a traditional method show that the proposed method improves both the accuracy and completion time of trajectory tracking control task by 36.55% and 22.81%

respectively.

关键词

Keywords

references

BI Luzheng, WANG Huikang, TENG Teng, et al. A novel method of emergency situation detection for a brain-controlled vehicle by combining EEG signals with surrounding information [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(10): 1926-1934.

DUAN Xu, XIE Songyun, XIE Xinzhou, et al. Quadcopter flight control using a non-invasive multi-modal brain computer interface [J]. Frontiers in Neurorobotics, 2019, 13: 23.

唐智川, 张克俊, 李超, 等. 基于深度卷积神经网络的运动想象分类及其在脑控外骨骼中的应用 [J]. 计算机学报, 2017, 40(6): 1367-1378.

TANG Zhichuan, ZHANG Kejun, LI Chao, et al. Motor imagery classification based on deep convolutional neural network and its application in exoskeleton controlled by EEG [J]. Chinese Journal of Computers, 2017, 40(6): 1367-1378.

徐光华, 张锋, 谢俊, 等. 稳态视觉诱发电位的脑-机接口范式及其信号处理方法研究 [J]. 西安交通大学学报, 2015, 49(6): 1-7.

XU Guanghua, ZHANG Feng, XIE Jun, et al. Brain-computer interface paradigms and signal processing strategy for steady state visual evoked potential [J]. Journal of Xi’an Jiaotong University, 2015, 49(6): 1-7.

张小栋, 李睿, 李耀楠. 脑控技术的研究与展望 [J]. 振动、测试与诊断, 2014, 34(2): 205-211.

ZHANG Xiaodong, LI Rui, LI Yaonan. Research on brain control technology [J]. Journal of Vibration, Measurement Diagnosis, 2014, 34(2): 205-211.

ZHANG Jinhua, WANG Baozeng, ZHANG Cheng, et al. An EEG/EMG/EOG-Based multimodal human-machine interface to real-time control of a soft robot hand [J]. Frontiers Neurorobotics, 2019, 13: 7.

李治非, 杨阳, 苏月, 等. 我国外科手术机器人研究应用现状与思考 [J]. 中国医学装备, 2019, 16(11): 177-181.

LI Zhifei, YANG Yang, SU Yue, et al. Current status and thinking of research and application of surgical robots in China [J]. China Medical Equipment, 2019, 16(11): 177-181.

陈涛. 遥操作排爆机器人研制及其运动控制研究 [D]. 济南: 山东大学, 2019: 6-12.

唐道光, 张公平, 杜肖, 等. 人在回路对导弹制导性能的影响研究 [J]. 航空兵器, 2018(3): 24-30.

TANG Daoguang, ZHANG Gongping, DU Xiao, et al. Research on missile guidance performance of soldier-in-the-loop [J]. Aero Weaponry, 2018(3): 24-30.

BURNS J O, MELLINKOFF B, SPYDELL M, et al. Science on the lunar surface facilitated by low latency telerobotics from a lunar orbital platform-gateway [J]. Acta Astronautica, 2019, 154: 195-203.

DIJKSTERHUIS C, WAARD D D, BROOKHUIS K A, et al. Classifying visuomotor workload in a driving simulator using subject specific spatial brain patterns [J]. Frontiers in Neuroscience, 2013, 7: 149.

朱成杰, 陈继龙, 施斌斌. 飞行疲劳检测技术的研究进展 [J]. 航空航天医学杂志, 2019, 30(5): 543-546.

ZHU Chengjie, CHEN Jilong, SHI Binbin. Research progress of flight fatigue detection technology [J]. Journal of Aerospace Medicine, 2019, 30(5): 543-546.

WANG Ziheng, HOPE R M, WANG Zuoguan, et al. Cross-subject workload classification with a hierarchical Bayes model [J]. NeuroImage, 2012, 59(1): 64-69.

PARASURAMAN R, WILSON G F. Putting the brain to work: neuroergonomics past, present, and future [J]. Human Factors, 2008, 50(3): 468-474.

JIA Y, XI N, LIU S, et al. Quality of teleoperator adaptive control for telerobotic operations [J]. The International Journal of Robotics Research, 2014, 33(14): 1765-1781.

杨少增. 基于多源生理数据与模糊建模方法的操作员功能状态预测与调节 [D]. 上海: 华东理工大学, 2014: 72-134.

赵星宇, 丁世飞. 深度强化学习研究综述 [J]. 计算机科学, 2018, 45(7): 1-6.

ZHAO Xingyu, DING Shifei. Research on deep reinforcement learning [J]. Computer Science, 2018, 45(7): 1-6.

刘雪晨, 李宝瑜, 张晰. 蒙特卡洛特征样本采样方法研究 [J]. 统计与信息论坛, 2019, 34(1): 3-12.

LIU Xuechen, LI Baoyu, ZHANG Xin. The research on Monte Carlo feature sample generation method [J]. Statistics Information Forum, 2019, 34(1): 3-12.

ZHANG T, ZHANG X, ZHANG Y, et al. Effects of user fatigue mental state on the facial-expression paradigm of BCI [C]∥2nd WRC Symposium on Advanced Robotics and Automation. Piscataway, NJ, USA: IEEE, 2019: 394-400.

石立臣. 基于脑电信号的警觉度估计研究 [D]. 上海: 上海交通大学, 2012: 45-48.

刘坤华, 钟佩思, 徐东方, 等. 基于双曲正切函数的修正线性单元 [J]. 计算机集成制造系统, 2020, 26(1): 145-151.

LIU Kunhua, ZHONG Peisi, XU Dongfang, et al. Tangent-based rectified linear unit [J]. Computer Integrated Manufacturing Systems, 2020, 26(1): 145-151.

章东平, 陈思瑶, 李建超, 等. 基于改进型加性余弦间隔损失函数的深度学习人脸识别 [J]. 传感技术学报, 2019, 32(12): 1830-1835.

ZHANG Dongping, CHEN Siyao, LI Jianchao, et al. Deep learning face recognition based on improved additive cosine margin softmax loss function [J]. Chinese Journal of Sensors and Actuators, 2019, 32(12): 1830-1835.

KINGMA D P, BA J L. Adam: a method for stochastic optimization [C]∥3rd International Conference on Learning Representations. Montreal, Canada: ICLR, 2015: 20193407343622.

仲文远,李大海,张进华,等.可穿戴式干电极脑机接口系统设计.2020,54(6):066-74.[doi:10.7652/xjtuxb202006009]

杨磊,杨帆,何艳.采用样本熵自适应噪声完备经验模态分解的脑电信号眼电伪迹去除算法.2020,54(8):177-184.[doi:10.7652/xjtuxb202008023]

张克旭,杜昌旺,赵浩淇,等.针对局灶性癫痫患者的脑电微状态分析.2020,54(9):157-163.[doi:10.7652/xjtuxb2020 09018]

张彦军,谢俊,薛涛,等.稳态视觉诱发电位脑机接口的现场可编程逻辑门阵列实现.2020,54(2):158-165.[doi:10.7652/xjtuxb202002020]

杜光景,谢俊,张玉彬,等.用于稳态视觉诱发电位脑机接口目标识别的深度学习方法.2019,53(11):042-48.[doi:10.7652/xjtuxb201911006]

吴敏,王化宁,张羿阳,等.情绪面孔识别状态下抑郁症患者大脑微状态的差异性研究.2019,53(4):143-149.[doi:10.7652/xjtuxb201904021]

王栋,李扩,刘晓芳,等.利用高频脑电的局灶性癫痫患者癫痫发作检测.2018,52(2):148-154.[doi:10.7652/xjtuxb2018 02023]

赵广宾,王会友,秦歌,等.脑深部刺激电极表面多壁碳纳米管的电泳沉积工艺研究.2019,53(2):142-149.[doi:10.7652/xjtuxb201902019]

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向变形滑翔飞行器的智能变形决策方法

联合联邦学习与深度强化学习的并行服务功能链部署算法

机械臂非奇异快速终端滑模迭代学习轨迹跟踪控制研究

基于深度强化学习的变形滑翔飞行器智能变形决策方法

采用改进快速变幂次趋近律的滑模控制方法