基于大语言模型和机器视觉的智能制造系统人机自主协同作业方法

In Industry 4.0, the emerging technologies such as artificial intelligence, big data, and the Internet of Things are appearing endlessly, accelerating the transformation and upgrading of the manufacturing industry. In this process, industry robot plays an increasingly important role, which also lays a solid foundation for the high-quality development of intelligent/smart manufacturing. With the proposal of Industry 5.0, human centricity concept becomes popular, which has given birth to the emerging field of human-centric smart manufacturing. The boundary between human and robot in the smart manufacturing systems gets blurred, and the research on human-robot autonomous collaboration has attracted more and more attentions. Therefore, proposes a human-robot autonomous collaboration method based on large language model and machine vision to improve the intelligence level of human-robot collaboration. First, dynamic perception of the working process for human-robot collaboration is carried out by the fusion of machine vision and deep learning, where the fusion of YOLO and transfer learning is adopted to accurately identify the operate progress and the long short-term memory network and attention mechanism are combined to recognize the actions of operator. Second, the large language model is fine-tuned for human-robot collaboration to realize autonomous operating decision for smart robot during the dynamic work process. Finally, a reducer assembly case is used to verify the effectiveness of the proposed method.