Research On Engineering Oriented Cross Modal Mechanical Parts Retrieval Method

Mechanical parts are the foundation of the mechanical industry,and the design,research,development,and application management of product parts play a very important role in the development of the mechanical industry and social progress.The long-term industrial development and mechanical production have produced a large amount of part information,which has condensed a large amount of experience and knowledge of workers,and is of extremely important value.With the continuous increase in the number of parts,many excellent information is submerged in huge part data.Part designers or managers need to spend a lot of time and energy searching for high-quality part resources,which seriously affects the development process and application circulation of part products.This topic introduces cross modal retrieval technology into the relevant retrieval work of mechanical parts,breaking the retrieval barrier between part images and text,and achieving rapid and accurate retrieval of mechanical part information within the same mode and between different modes,which is conducive to shortening the design cycle,improving product quality,and reducing maintenance management costs.The main research contents are as follows:(1)Data acquisition of mechanical parts images and text.Build an offline shooting platform,analyze and select key shooting equipment such as cameras,lenses,and light sources,and complete the offline shooting process for some parts;Query and search the image information of parts through the network;Add a text description to the image of the part to form the text data of the mechanical part.(2)Similarity matching of part related text descriptions.Calculate surface feature similarity between different texts based on surface information such as word character distance and weight hash feature differences between different texts.Word vectors of different words in the text are extracted through Word2 vec,and then weight coefficients of different words are obtained through TF-IDF.Each word vector is weighted to synthesize high-level semantic feature vectors of the text,and the similarity of high-level semantic features between different parts of the text is calculated.Finally,surface similarity and high-level similarity are fused to bring their respective advantages into play and achieve accurate part text matching.(3)Multi feature fusion matches part image data.In order to increase the rotation invariance and scale invariance of part image matching,a combined hash is used to encode the gray scale image of the part.In order to effectively identify the color features of different part drawings,color histograms are used to match the similarity of different part drawings.In order to distinguish the brightness,structure,and other information of different part drawings,SSIM structure similarity coefficients are added to the part drawing matching process.Obtain high-level semantic features of part images through MobileNet2 to conduct deeper level similarity comparison of part images.Integrate multiple features,take advantage of different features in part drawing matching,and complete accurate calculation of part drawing similarity.(4)Research on cross modal retrieval of improved graph convolution.In order to increase the local consistency within the respective modes of the part image and text,a graph convolution network is used to extract the part image and text features.To solve the problem of over smoothing in deep graph convolution networks,a method of adding initial residual connection and unit weight matrix to each layer graph convolution is proposed.By adding a fully connected network parallel to the graph convolution network,the fitting ability of the network model is increased.(5)Integrating attention into part image and text retrieval.In order to assign different weights to each neighbor node of the central node and highlight the importance,an attention mechanism is introduced into graph convolution to calculate the weight coefficients between different nodes.To solve the problem of insufficient feature mining between nodes,multiple groups of attention heads are added to fully focus on multiple groups of related features between each node.By using intra modal semantic constraints and inter modal invariant constraints,the distance between similar graphic and textual feature vectors in the common subspace is narrowed,effectively improving the accuracy of cross modal mechanical parts retrieval.