Automatic Labeling Method For Mural Paint Loss Disease Based On Hyperspectral And Deep Learning

Mural painting is an important part of Chinese cultural heritage.Various diseases have appeared in ancient murals affected by the natural environment.Disease labeling is a record of the location and size of the diseases of the murals.It is the basis for the investigation and protection and restoration of the mural status.Disease labeling plays an important role in the protection of cultural relics.The paint loss is a typical disease of murals.At present,the paint loss disease labeling of murals is mainly done by manual labeling or interactive labeling combined with machine learning algorithms.Manual labeling consumes a lot of time,the accuracy of machine learning methods needs to be improved.At the same time,the existing marking method is mainly based on the human eye visual marking in the data collected by the general digital camera.The image contains a limited amount of information,which makes it difficult to find some diseased areas,it is easy to miss the mark.Hyperspectral data contains a large amount of spectral spatial information.It is a non-destructive,non-contact imaging method,which is a research hotspot in the field of cultural relic’s protection.In the research of mural disease labeling,because hyperspectral images contain more band information and the spectral curves of various pigments and diseases in murals are different,so this paper is based on the hyperspectral data to do research on the labeling of paint loss diseases.The main work of this paper is as follows:(1)We proposed an automatic labeling framework for the paint loss disease of ancient murals based on hyperspectral image classification and segmentation technology.This method adopted a disease labeling strategy that effectively combines the results of deep learning-based hyperspectral classification and segmentation on hyperspectral images to obtain the final labeling result.The experimental results show that this method obtains better labeling results.It is an effective labeling method for paint loss disease.(2)We proposed a multi-scale U-Net hyperspectral image segmentation and heterogeneous transfer learning to label diseases.First,a mapping network was designed to map hyperspectral data to three channels.This is done to make it consistent with the number of channels of data captured by a general digital camera.Second,we built a hyperspectral image segmentation network based on U-Net.Then,we fine-tune the hyperspectral data based on the U-Net network parameters trained on images taken by general digital cameras.On this basis,we introduced a multi-scale module to extract features of different scales while retaining low-level features,improving the accuracy of marking.The experimental results show that this method can effectively use the prior knowledge that has been annotated in the images taken by general digital cameras,guide the disease marking on the hyperspectral data.It gave better annotation results.