Research On Image Retrieval Methods For Woven Fabrics Based On Feature Engineering

Fabric retrieval is an important link in the fabric production and management of textile enterprises.According to the provided samples,existing similar products can be retrieved to invoke the process parameters to facilitate the production,so as to save the heavy work of sample analysis and repeated weaving proofing.Moreover,the digital and intelligent management of fabric products can be carried out,so fabric retrieval has strong practical significance for fabric enterprises.At present,fabric manufacturers generally adopt the real sample storage method and text-based image retrieval(TBIR)method.However,the real sample storage method occupies the resources of preserving and managing fabric real samples,and the artificial comparison for fabric search is subjective and with low accuracy.Although the TBIR method improves the retrieval accuracy and efficiency to a certain extent,a large number of text annotations are needed to label fabric images.The annotating process is timeconsuming and laborious,and also has strong subjectivity.Thus,studying a fast,effective and objective fabric retrieval method is an urgent need for fabric manufacturers.The existing image retrieval technologies based on feature engineering include contentbased image retrieval(CBIR)and multimodal fusion retrieval.They mainly focus on nature image retrieval,being difficult to distinguish the fine features such as fabric density,fabric weave and yarn fineness contained in fabric images.Moreover,there is a lack of a standard fabric image dataset for algorithm evaluation,being difficult to obtain satisfactory retrieval results by directly adopting the existing retrieval technologies.At present,there are four retrieval demands for fabric manufacturers according to users' different emphases on fabric attributes during imitation production.The retrieval demands include:(1)emphasizing texture similarity;(2)emphasizing color similarity;(3)emphasizing the joint similarity of texture and color;(4)providing the similar sample when the user cannot provide samples that match exactly with expectations and the text description is needed to assist in making changes to fabric attributes.In the existing researches of fabric retrieval,CBIR methods are mostly used for fabric image retrieval by the joint similarity of texture and color,being difficult to characterize the fine features of fabrics.The retrieval fineness cannot meet the requirements of industrial applications.Moreover,the retrieval method is single and cannot meet the demands of enterprises emphasizing on different properties of fabrics.Based on the above problems and the four demands of manufacturers,this paper establishes a dataset for woven fabric retrieval with many varieties and categories as data support and validation basis.The retrieval methods for woven fabric image retrieval which emphasize texture similarity,color similarity,texture and color joint similarity and multimodal fusion retrieval methods were studied based on feature engineering to meet different retrieval demands and improve the retrieval accuracy and efficiency.An online retrieval system for woven fabrics was established to realize efficient management and production of enterprises.The specific research works of this paper are as follows:(1)According to the periodic characteristics of fabrics,the macroscopic and fine texture features of fabric images were explored from the global and local perspectives to propose a fabric retrieval method emphasizing texture similarity.Firstly,the characterizability of texture features of fabric images was improved by texture enhancement,and then the frequency differential features were designed and extracted based on the circular partition of the Fourier spectrum to obtain the macroscopic and fine texture features of fabrics.Meanwhile,local fine texture features based on local binary patterns were explored and extracted to assist in characterizing fabric images.The advantages of the two features were integrated by the proposed approach of similarity product to achieve fabric image retrieval emphasizing texture similarity.Experimental results show that the proposed method can effectively retrieve fabric images with similar textures,and the average retrieval precision(P@8)and recall(R@8)of the top 8 images reach 83.1% and 51.9%,respectively.The mean Average Precision(m AP)is up to 0.766 and the elapsed time reaches 3.2 seconds.The experimental comparison of different texture characterization methods verified the better adaptability of the proposed extraction method for fabric texture features in this paper than other methods.(2)For the problem that the colors of different kinds of fabrics are different and the proportion of some colors in the embedded fabrics is small,a fabric retrieval method emphasizing color similarity was proposed based on the dominant color and color moment features by image partition.First,the fabric image was scaled to reduce the computing time.Then a fast quantization method was used to extract the dominant colors of the image.A weighted aggregation average method was used to aggregate the set of non-dominant colors to characterize macroscopic and fine color features.The aggregation operator can improve the characterization performance of fabrics with a small percentage of colors.Meanwhile,local features characterizing the information of color local position were extracted by partitioned color moments to assist in distinguishing different categories of fabric images.The similarity product was also used to fuse the advantages of the two features.The experimental results show that the average P@8,R@8,m AP and retrieval elapsed time reach 84.5%,42.6%,0.754 and2.8 seconds,respectively.Compared with other color characterization methods,the performances verify that the proposed method is more applicable to characterize the color information of fabrics.(3)A woven fabric retrieval method emphasizing the joint similarity of texture and color was proposed based on the idea of transfer learning.First,the salient activated regions of the feature maps in different module layers were located based on the hierarchical characterization mechanism of the convolutional neural network(CNN)model.Then,the global average pooling was used for regional aggregation to construct deep aggregated features which contain macroscopic and fine features.The characterization abilities of different module layers were explored for fabric images.To improve the timeliness,the Annoy algorithm in the approximate nearest neighbor search was used for similarity measurement.The proposed method can jointly characterize the texture and color of the fabric with the average P@8,R@8,m AP and retrieval elapsed time of 89.1%,56.0%,0.840 and 51.5 milliseconds,respectively.Different feature extraction methods and approximate nearest neighbor search algorithms were compared to highlight the applicability of the methods in fabric retrieval emphasizing the joint similarity of texture and color.(4)From the perspective of fabric image understanding and attribute mining,a multimodal fusion-based woven fabric retrieval method was proposed by incorporating the user's change requirements for the query image.Image visual features with separable macroscopic and fine features were extracted by modifying the pre-trained CNN model.Semantic features of text descriptions were extracted by the long and short-term memory network.In the multi-modal feature fusion space,the selective expression of image macroscopic and fine features was controlled by the designed gated and residual structure.Meanwhile,the image visual features of the query fabric and the semantic features of the text description were fused to realize the flexible control of fabric image retrieval results by the text description.The experimental results show that the average recall of the user-desired target image in the top 1,5,10 and 20 images are up to 35.8%,67.6%,79.9% and 89.9%,respectively.The average precision of the top 5images reaches 69.8%.The average elapsed time is 13.2 milliseconds.The results show that the proposed method is feasible and efficient,which can realize the flexible control of retrieval results.The experimentally comparison with different image characterization methods and multimodal feature fusion methods demonstrated that the proposed method is more suitable for multimodal fusion retrieval of woven fabrics.Combined with the above research findings for four retrieval demands,this paper constructed a set of industrial woven fabric online retrieval system to meet the actual demands of fabric manufacturers.The established system is much better than the existing methods in retrieval accuracy and timeliness.The system can quickly and effectively realize woven fabric retrieval emphasizing texture similarity,color similarity,the joint similarity of texture and color,and multimodal fusion retrieval,having broad application prospects.