Modeling of Multi-Modal Knowledge Graph for Assembly Process of Wind Turbines with Multi-Source Heterogeneous Data

The assembly process information of wind turbines is usually scattered in process documents consisting of multi-modal information, such as 3D models, natural texts, and images. Therefore, the cost of maintaining data and extracting process knowledge is high while the efficiency is low. To solve this problem, a multi-modal knowledge graph-based modeling method for the assembly process knowledge of wind turbines is proposed with multi-source heterogeneous data. First, the concepts in multi-modal process knowledge graph of wind turbine (MPKG-WT) are defined by analyzing the process characteristics of wind turbines to complete the construction of ontology. Then, based on the characteristics of multi-source heterogeneous data and multi-modal information, data analysis, knowledge extraction, and semantic similarity calculation are leveraged to realize the automatic instantiation of the graph. Finally, taking the process data of a wind turbine enterprise as an example, MPKG-WT is constructed and verified by implementing an auxiliary system for process design. The results show that MPKG-WT is more informative than the single-modal graph, and the data in different modals can complement each other, which leads to significant improvements in the efficiency of process design. © 2024 Shanghai Jiaotong University. All rights reserved.