Assembly Intention Recognition Method in Human-Robot Collaboration for Complex Environments

HE Jiawei; ZHANG Chaoyang; YE Zijian; SHI Tianyou

doi:null

您当前的位置：

首页 >

文章列表页 >

Assembly Intention Recognition Method in Human-Robot Collaboration for Complex Environments

更新时间：2025-12-01

- Assembly Intention Recognition Method in Human-Robot Collaboration for Complex Environments
- Moren Journal (2025)
- 作者机构：
  
  1.江南大学机械工程学院,214122
  2.江苏省食品先进制造装备技术重点实验室,214122
  3.江南大学,江苏省无锡市,中国
- 作者简介：
- 基金信息：
- DOI：
  CLC： TH166
- Received：28 April 2025，
  
  Revised：2025-07-27，
  
  Accepted：29 August 2025，
- 稿件说明：
移动端阅览
HE Jiawei, ZHANG Chaoyang, YE Zijian, et al. Assembly Intention Recognition Method in Human-Robot Collaboration for Complex Environments[J/OL]. Moren Journal, 2025.
DOI：

HE Jiawei, ZHANG Chaoyang, YE Zijian, et al. Assembly Intention Recognition Method in Human-Robot Collaboration for Complex Environments[J/OL]. Moren Journal, 2025. DOI：

摘要

针对复杂环境中人机协作装配系统识别装配动作准确率较低，并且动作相似度高情况下无法准确感知装配者意图，导致机器人配合操作者的装配效率较低的问题，从装配动作信息和装配零件信息之间的联系分析建立了面向复杂环境的人机协作装配意图识别融合模型。通过基于骨架特征使用一种双流自适应图卷机神经网络（2S-AGCN）模型，针对装配动作进行识别；提出一种改进的YOLOV8模型，以提高网络模型对于装配零件的无序和遮挡环境下装配零件识别的准确率；结合当前装配任务设计了一种包含装配动作和装配零件信息的装配推理规则，规划装配顺序。在装配动作数据集以及装配泵体零件数据集上对所提方法进行了验证。实验结果表明：所提出的改进的YOLOV8模型对于零件识别的mAP50-90的平均准确率达到88.361%；提出的人机协作装配意图识别融合模型对于操作员装配意图识别的准确率达到96.66%，验证了人机协作系统用于识别操作员装配意图的可行性和有效性，将装配动作信息与装配零件信息相结合，有助于在复杂的人机协作环境中准确地识别出操作员的装配意图。

Abstract

Aiming at the problem that the human-robot cooperative assembly system in complex environment has low accuracy in identifying assembly actions

and can not accurately perceive the assembler's intention when the action similarity is high

which leads to low assembly efficiency of robot cooperation with the operator

a recognition fusion model of human-robot collaboration assembly intention in complex environment is established from the relationship between assembly action information and assembly part information. By using a dual stream adaptive graph convolutional neural network (2S-AGCN) model based on skeleton features

assembly actions are recognized; Propose an improved YOLOV8 model to enhance the accuracy of the network model in identifying assembly parts in disordered and occluded environments; A assembly inference rule containing assembly actions and assembly part information was designed based on the current assembly task to plan the assembly sequence. The proposed method was validated on the assembly action dataset and the assembly pump body parts dataset. The experimental results show that the proposed improved YOLOV8 model achieves an average accuracy of 88.361% for mAP50-90 in part recognition; The proposed human-robot collaboration assembly intent recognition fusion model has an accuracy rate of 96.66% for operator assembly intent recognition

which verifies the feasibility and effectiveness of human-robot collaboration system used to identify operator assembly intent. The combination of assembly action information and assembly part information is helpful to accurately identify operator assembly intention in the complex human-robot collaboration environment.

关键词

Keywords

references

戴汝为 . 人-机结合的智能科学和智能工程 [J ] . 中国工程科学 , 2004 , ( 05 ): 24 - 28 .

DAI Ruwei . Man-computer cooperative intelligent science and intelligent technology [J ] . Strategic Study of CAE , 2004 , ( 05 ): 24 - 28 .

戴汝为 . 人-机结合的智能工程系统——处理开放的复杂巨系统的可操作平台 [J ] . 模式识别与人工智能 , 2004 , 17 ( 03 ): 257 - 261 .

DAI Ruwei . Man-computer cooperated intelligent engineering system—a management platform for dealing with ocgs [J ] . Pattern Recognition and Artificial Intelligence , 2004 , 17 ( 03 ): 257 - 261 .

MATHESON E , MINTO R , ZAMPIERII E G G , et al . Human–robot collaboration in manufacturing applications: a review [J ] . Robotics , 2019 , 8 ( 4 ): 100 .

李梓响 . 人机协同双边装配线平衡建模及智能算法研究 [D ] . 武汉 : 武汉科技大学 , 2018 : 1 - 10 .

LI Zixiang . Mathematical modeling and intelligent algorithm for man-robot cooperative two-sided assembly line balanceing problems [D ] . Wuhan : Wuhan University of Science and Technology , 2018 : 1 - 10 .

Lv M , Feng Z , Yang X , et al . A method for human–robot complementary collaborative assembly based on knowledge graph [J ] . CCF Transactions on Pervasive Computing and Interaction , 2025 : 1 - 17 .

喻惟刚 , 文思捷 , 李建军 , 等 . 基于可供性推理的人机协作拆卸序列规划方法 [J ] . 机械工程学报 , 2024 , 60 ( 17 ): 297 - 309 .

YU Weigang , WEN Sijie , Li Jianjun , et al . Affordance reasoning-based sequence planning manner for human-robot collaborative disassembly [J ] . Journal of Mechanical Engineering , 2024 , 60 ( 17 ): 297 - 309 .

姚碧涛 , 向宇光 , 徐文君 , 等 . 基于人体肌肉激活疲劳模型的人机协作装配序列重规划研究 [J/OL ] . 机械工程学报 , 1 - 12 [ 2025-07-27 ] .

YAO Bitao , XIANG Yuguang , XU Wenjun , et al . Research on human-robot collaborative assembly sequence replanning based on the human muscle activation fatigue model . [J/OL ] . Journal of Mechanical Engineering , 1 - 12 [ 2025-07-27 ] .

XIAO J , TIAN W , DING L . Basketball action recognition method of deep neural network based on dynamic residual attention mechanism [J ] . Information , 2022 , 14 ( 1 ): 13 .

姚冬安 , 徐文君 , 姚碧涛 , 等 . 基于上下文感知与图注意力网络的人机协作装配人员作业意图识别方法 [J ] . 计算机集成制造系统 , 2024 , 30 ( 06 ): 2005 - 2013 .

YAO Dongan , XU Wenjun , YAO Bitao , et al . Human intention recognition method based on context awareness and graph attention network for human-robot collaborative assembly [J ] . Computer Integrated Manufacturing Systems , 2024 , 30 ( 06 ): 2005 - 2013 .

ZHOU Y , LUO L , LI P . An Accurate Prediction Method of Human Assembly Motion for Human–Robot Collaboration [J ] . Symmetry , 2024 , 16 ( 1 ): 118 .

URGO M , TARABINI M , TOLIO T . A human modelling and monitoring approach to support the execution of manufacturing operations [J ] . CIRP Annals , 2019 , 68 ( 1 ): 5 - 8 .

WANG Z , YAN J , YAN G , et al . Multi-scale control and action recognition based human-robot collaboration framework facing new generation intelligent manufacturing [J ] . Robotics and Computer-Integrated Manufacturing , 2025 , 91 : 102847 .

Al-AMIN M , QIN R , MONIRUZZAMAN M , et al . An individualized system of skeletal data-based CNN classifiers for action recognition in manufacturing assembly [J ] . Journal of Intelligent Manufacturing , 2023 : 1 - 17 .

YAN S , XIONG Y , LIN D . Spatial temporal graph convolutional networks for skeleton-based action recognition [C ] //Proceedings of the AAAI conference on artificial intelligence. Washington, D.C: AAAI , 2018 , 32(1) .

SHI L , ZHANG Y , CHENG J , et al . Two-stream adaptive graph convolutional networks for skeleton-based action recognition [C ] // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . Long Beach, CA, USA : 2019 : 12026 - 12035 .

WANG T , LIU Z , WANG L , et al . Data-efficient multimodal human action recognition for proactive human–robot collaborative assembly: A cross-domain few-shot learning approach [J ] . Robotics and Computer-Integrated Manufacturing , 2024 , 89 : 102785 .

MOUTINHO D , ROCHA L F , COSTA C M , et al . Deep learning-based human action recognition to leverage context awareness in collaborative assembly [J ] . Robotics and Computer-Integrated Manufacturing , 2023 , 80 : 102449 .

GAO Z , YANG R , ZHAO K , et al . Hybrid convolutional neural network approaches for recognizing collaborative actions in human–robot assembly tasks [J ] . Sustainability , 2024 , 16 ( 1 ): 139 .

ZHANG Y , DING K , HUI J , et al . Human-object integrated assembly intention recognition for context-aware human-robot collaborative assembly [ J ] . Advanced Engineering Informatics , 2022 , 54 : 101792 .

FAN J , ZHENG P , LEE C K M . A vision-based human digital twin modeling approach for adaptive human–robot collaboration [J ] . Journal of Manufacturing Science and Engineering , 2023 , 145 ( 12 ): 121002 .

LI C , ZHENG P , ZHOU P , et al . Unleashing mixed-reality capability in Deep Reinforcement Learning-based robot motion generation towards safe human–robot collaboration [J ] . Journal of Manufacturing Systems , 2024 , 74 : 411 - 421 .

ZHENG P , LI C , FAN J , et al . A vision-language-guided and deep reinforcement learning-enabled approach for unstructured human-robot collaborative manufacturing task fulfilment [J ] . CIRP annals , 2024 , 73 ( 1 ): 341 - 344 .

ZHANG Y , DING K , HUI J , et al . Skeleton-RGB integrated highly similar human action prediction in human–robot collaborative assembly [J ] . Robotics and Computer-Integrated Manufacturing , 2024 , 86 : 102659 .

WANG T , FAN J , ZHENG P . An LLM-based vision and language cobot navigation approach for Human-centric Smart Manufacturing [J ] . Journal of Manufacturing Systems , 2024 , 75 : 299 - 305 .

黄思翰 , 陈建鹏 , 徐哲 , 等 . 基于大语言模型和机器视觉的智能制造系统人机自主协同作业方法 [J ] . 机械工程学报 , 2025 , 61 ( 03 ): 130 - 141 .

HUANG Sihan , CHEN Jianpeng , XU Zhe , et al . Human-robot autonomous collaboration method of smart manufacturing system based on large language model and machine vision [J ] . Journal of Mechanical Engineering , 2025 , 61 ( 03 ): 130 - 141 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Human-Robot Collaborative Assembly Intention Recognition Method for Complex Environments

Influence of Structural Parameters on the Summer Operational Characteristics of a Three-Tower-in-One Indirect Air-Cooling System

Integrated and Joint Design Optimization of Centrifugal Compressor Impeller and Casing Treatment at Multiple Working Points

Numerical Simulation of the Energy Conversion Process in Thermoacoustic Engines with Different Stack Materials

Research on the Electrohydrodynamic Effects of Ionic Liquid-Modified Fuels

Related Author

HE Jiawei

ZHANG Chaoyang

YE Zijian

SHI Tianyou

ZHENG Kunming

Related Institution

江南大学机械工程学院，214122，江苏无锡School of Mechanical Engineering, Jiangnan University, Wuxi, Jiangsu 214122, China

江苏省食品先进制造装备技术重点实验室，214122，江苏无锡Jiangsu Key Laboratory of Advanced Food Manufacturing Equipment and Technology, Wuxi, Jiangsu 214122, China

江南大学智能制造学院，214122，江苏无锡School of Intelligent Manufacturing, Jiangnan University, Wuxi, Jiangsu 214122, China

⁰