Human-Robot Collaborative Assembly Intention Recognition Method for Complex Environments

HE Jiawei; ZHANG Chaoyang; YE Zijian; SHI Tianyou; ZHENG Kunming

doi:10.7652/xjtuxb202512004

您当前的位置：

首页 >

文章列表页 >

Human-Robot Collaborative Assembly Intention Recognition Method for Complex Environments

Special Topic Industrial Internet and Manufacturing Services | 更新时间：2025-12-08

- Human-Robot Collaborative Assembly Intention Recognition Method for Complex Environments
- Journal of Xi'an Jiaotong University Vol. 59, Issue 12, Pages: 44-57(2025)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202512004
  CLC： TH166
- Received：28 April 2025，
  
  Published：10 December 2025
- 稿件说明：
移动端阅览
HE Jiawei, ZHANG Chaoyang, YE Zijian, et al. Human-Robot Collaborative Assembly Intention Recognition Method for Complex Environments[J]. Journal of Xi'an Jiaotong University, 2025, 59(12): 44-57.
DOI：

HE Jiawei, ZHANG Chaoyang, YE Zijian, et al. Human-Robot Collaborative Assembly Intention Recognition Method for Complex Environments[J]. Journal of Xi'an Jiaotong University, 2025, 59(12): 44-57. DOI： 10.7652/xjtuxb202512004.

摘要

针对复杂环境中人机协作装配系统存在的识别装配动作准确率较低、动作相似度高时无法准确感知装配者意图，以及由此导致的机器人配合操作者的装配效率较低的问题，从装配动作信息和装配零件信息之间的联系出发，建立了面向复杂环境的人机协作装配意图识别融合模型。基于骨架特征，采用双流自适应图卷积神经网络（2 S-AGCN）模型针对装配进行动作识别；提出改进的YOLOV8模型，以提高装配零件在无序和遮挡环境下的识别准确率；结合当前装配任务，设计了包含装配动作和装配零件信息的装配推理规则，规划装配顺序。在装配动作数据集以及装配泵体零件数据集上对所提方法进行了验证。实验结果表明：交并比为0.50～0.95时，YOLOV8模型对零件识别的平均准确率达到88.361%；所提出的融合模型对于操作者装配意图的识别准确率达到96.66%，验证了人机协作系统识别操作者装配意图的可行性和有效性。将装配动作信息与装配零件信息相结合，有助于在复杂的人机协作环境中准确地识别出操作者的装配意图。

Abstract

To address the issues of low accuracy in recognizing assembly actions and the inability to accurately perceive human operators'intention under high action similarity in human-robot collaborative assembly systems within complex environments

as well as the resulting inefficiency in robot-operator collaboration

this study establishes an intention recognition fusion model for human-robot collaborative assembly in complex environments by integrating assembly action information and assembly part information.Based on skeletal features

a two-stream adaptive graph convolutional neural network（2 S-AGCN）model is employed for action recognition in assembly tasks.An improved YOLOV8 model is proposed to enhance the recognition accuracy of assembly parts in disordered and occluded environments.Considering with the current assembly task

assembly reasoning rules incorporating both assembl

y actions and part information are designed to plan the assembly sequence.The proposed method is validated on an assembly action dataset and an assembly pump part dataset.The experimental results show that when the intersection over union（

）threshold ranges from 0.50 to 0.95

the improved YOLOV8 model achieves a mean average precision of 88.361% for part recognition.The proposed fusion model achieves an intention recognition accuracy of 96.66%

demonstrating the feasibility and effectiveness of the human-robot collaborative system in recognizing operator assembly intention.Integrating assembly action information with assembly part information contributes to accurately identifying operator assembly intention in complex human-robot collaborative environments.

关键词

Keywords

references

戴汝为 . 人-机结合的智能科学和智能工程 [J ] . 中国工程科学 , 2004 ( 5 ): 24 - 28 .

DAI Ruwei . Man-computer cooperative intelligent science and intelligent technology [J ] . Strategic Study of CAE , 2004 ( 5 ): 24 - 28 .

戴汝为 . 人-机结合的智能工程系统：处理开放的复杂巨系统的可操作平台 [J ] . 模式识别与人工智能 , 2004 , 17 ( 3 ): 257 - 261 .

DAI Ruwei . Man-computer cooperated intelligent engineering system:a management platform for dealing withOCGS [J ] . Pattern Recognition and Artificial Intelligence , 2004 , 17 ( 3 ): 257 - 261 .

MATH ESON E , MINTO R , ZAMPIERII E G G , et al . Human-robot collaboration in manufacturing applications:a review [J ] . Robotics , 2019 , 8 ( 4 ): 100 - 124 .

李梓响 . 人机协同双边装配线平衡建模及智能算法研究 [D ] . 武汉 : 武汉科技大学 , 2018 : 1 - 10 .

ZHENG Xiao , FU Mengyin , YANG Yi , et al . 3D human postures recognition using kinect [C ] // 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics . Piscataway, NJ, USA : IEEE , 2012 : 344 - 347 .

王政伟 , 甘亚辉 , 戴先中 . 协作环境下的人机距离建模与最小距离迭代算法 [J ] . 机器人 , 2018 , 40 ( 4 ): 413 - 422 .

WANG Zhengwei , GAN Yahui , DAI Xianzhong . Human-robot distance modeling and minimum distance iterative algorithm in collaboration environment [J ] . Robot , 2018 , 40 ( 4 ): 413 - 422 .

VIEIRA A W , NASCIMENTO E R , OLIVEIRA G L , et al . Stop:space-time occupancy patterns for 3d action recognition from depth map sequences [C ] // Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications . Berlin, Germany : Springer , 2012 : 252 - 259 .

XIAO Jiongen , TIAN Wenchun , DING Liping . Basketball action recognition method of deep neural network based on dynamic residual attention mechanism [J ] . Information , 2022 , 14 ( 1 ): 13 - 27 .

姚冬安 , 徐文君 , 姚碧涛 , 等 . 基于上下文感知与图注意力网络的人机协作装配人员作业意图识别方法 [J ] . 计算机集成制造系统 , 2024 , 30 ( 6 ): 2005 - 2013 .

YAO Dongan , XU Wenjun , YAO Bitao , et al . Human intention recognition method based on context awareness and graph attention network for human-robot collaborative assembly [J ] . Computer Integrated Manufacturing Systems , 2024 , 30 ( 6 ): 2005 - 2013 .

ZHU Shaoping , XIA Limin . Human action recognition based on fusion features extraction of adaptive background subtraction and optical flow model [J ] . Mathematical Problems in Engineering , 2015 , 2015 ( 1 ): 387464 .

URGO M , TARABINI M , TOLIO T . Ahuman modelling and monitoring approach to support the execution of manufacturing operations [J ] . CIRP Annals , 2019 , 68 ( 1 ): 5 - 8 .

BERG J , RECKORDT T , RICHTER C , et al . Action recognition in assembly for human-robot-cooperation using hidden Markov models [J ] . Procedia CIRP , 2018 , 76 : 205 - 210 .

Al-AMIN M , QIN R , MONIRUZZAMAN M , et al . An individualized system of skeletal data-based CNN classifiers for action recognition in manufacturing assembly [J ] . Journal of Intelligent Manufacturing , 2023 , 34 ( 2 ): 633 - 649 .

YAN Sijie , XIONG Yuanjun , LIN Dahua . Spatial temporal graph convolutional networks for skeletonbased action recognition [C ] // Proceedings of the 32nd AAAI Conference on Artificial Intelligence . Palo Alto, C A,USA : AAAI , 2018 : 7444 - 7452 .

SHI Lei , ZHANG Yifan , CHENG Jian , et al . Twostream adaptive graph convolutional networks for skeleton-based action recognition [C ] // Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway, NJ, USA : IEEE , 2019 : 12018 - 12027 .

ANDRIANAKOS G , DIMITROPOULOS N , MICHA-LOS G , et al . An approach for monitoring the execution of human based assembly operations using machine learning [J ] . Procedia Cirp , 2019 , 86 : 198 - 203 .

宋栓军 , 侯中原 , 王启宇 , 等 . 改进YOLOV3算法在零件识别中的应用 [J ] . 机械科学与技术 , 2022 , 41 ( 10 ): 1608 - 1614 .

SONG Shuanjun , HOU Zhongyuan , WANG Qiyu , et al . Application of improved YOLOV3 algorithm in part recognition [J ] . Mechanical Science and Technology for Aerospace Engineering , 2022 , 41 ( 10 ): 1608 - 1614 .

GAO Zenggui , YANG Ruining , ZHAO Kai , et al . Hybrid convolutional neural network approaches for recognizing collaborative actions in human-robot assembly tasks [J ] . Sustainability , 2024 , 16 ( 1 ): 139 - 146 .

ZHANG Yaqian , DING Kai , HUI Jizhuang , et al . Human-object integrated assembly intention recognition for context-aware human-robot collaborative assembly [J ] . Advanced Engineering Informatics , 2022 , 54 : 101792 .

FAN Junming , ZHENG Pai , LEE C K M . A visionbased human digital twin modeling approach for adaptive human-robot collaboration [J ] . Journal of Manufacturing Science and Engineering , 2023 , 145 ( 12 ): 121002 .

LI Chengxi , ZHENG Pai , ZHOU Peng , et al . Unleashing mixed-reality capability in deep reinforcement learning-based robot motion generation towards safe human-robot collaboration [J ] . Journal of Manufacturing Systems , 2024 , 74 : 411 - 421 .

ZHENG Pai , LI Chengxi , FAN Junming , et al . A vision-language-guided and deep reinforcement learning-enabled approach for unstructured human-robot collaborative manufacturing task fulfilment [J ] . CIRP Annals , 2024 , 73 ( 1 ): 341 - 344 .

ZHANG Yaqian , DING Kai , HUI Jizhuang , et al . Skeleton-RGB integrated highly similar human action prediction in human-robot collaborative assembly [J ] . Robotics and Computer-Integrated Manufacturing , 2024 , 86 : 102659 .

WANG Tian , FAN Junming , ZHENG Pai . An LLMbased vision and language cobot navigation approach for human-centric smart manufacturing [J ] . Journal of Manufacturing Systems , 2024 , 75 : 299 - 305 .

黄思翰 , 陈建鹏 , 徐哲 , 等 . 基于大语言模型和机器视觉的智能制造系统人机自主协同作业方法 [J ] . 机械工程学报 , 2025 , 61 ( 3 ): 130 - 141 .

HUANG Sihan , CHEN Jianpeng , XU Zhe , et al . Human-robot autonomous collaboration method of smart manufacturing system based on large language model and machine vision [J ] . Journal of Mechanical Engineering , 2025 , 61 ( 3 ): 130 - 141 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Assembly Intention Recognition Method in Human-Robot Collaboration for Complex Environments

Research on the Electrohydrodynamic Effects of Ionic Liquid-Modified Fuels

A Fast Calculation Method for Thermal Flow Fields in Transformer Windings Based on PI-DeepONet

Influence of Structural Parameters on the Summer Operational Characteristics of a Three-Tower-in-One Indirect Air-Cooling System

Diffusion Characteristics and Influencing Factors in Case of Hydrogen Leakage from Fuel Cell Vehicles Running in Tunnels

Related Author

HE Jiawei

ZHANG Chaoyang

YE Zijian

SHI Tianyou

Related Institution

School of Mechanical Engineering, Jiangnan University

Jiangsu Key Laboratory of Advanced Food Manufacturing Equipment and Technology

江南大学

⁰