The Detection Method Of Dish Waste Degree Based On Computer Vision

Nowadays,with the improvement of people’s living standards,more and more people choose to eat out,and the phenomenon of food waste is becoming increasingly serious.Food waste is closely related to food security,so it is necessary to reduce food waste.At present,the main way to reduce food waste is to increase publicity or introduce related policy measures,but there are few reports on the technical methods to reduce food waste.At present,food waste also lacks a corresponding measure,making it difficult to qualitatively or quantitatively describe the degree of waste.Starting from the detection of food waste on the table,this paper studies two methods to measure food waste,which are based on convolutional neural network and mass measurement,and image segmentation.The corresponding waste degree was defined,and the waste degree level of dishes on the dining table was detected by computer vision technology.Detecting the degree of dish waste on the table can help understand the popularity of each type of food,and then eliminate the more wasteful food from the menu,keep the less wasted food,and reduce waste from the source.In addition,it can optimize the restaurant’s purchase inventory based on the corresponding waste situation of dishes,and improve the economic efficiency of the restaurant.The main research contents and conclusions of this paper are as follows:(1)A method for detecting dish waste degree based on convolutional neural network and mass measurement was proposed.13762 images of dishes were collected from a college canteen,including 5 categories of dishes.During the collection process,the dishes were placed on an electronic scale,and a fixed camera was used to record the complete change process from the presence to absence of the dishes and the corresponding mass information on the electronic scale.Then,the dishes were divided into 6 categories according to the interval where the mass ratio of the dishes before and after consumption is located,which serves as a sample label.Based on the convolutional neural network model,two images before and after food consumption are stacked in the direction of color channel information as the input of the network,and the experiments are carried out on three network structures,Inception V3,Xception and Res Net18.The experimental results show that,compared with the traditional single image as the input of the network,the method proposed in this paper can significantly improve the recognition accuracy,and the training process is more stable.In addition,in view of the low accuracy of sample label recognition for each model,but the identified incorrect label is adjacent to the actual label,this paper proposes a soft waste level definition method.Using the probability vector distribution of the output from the Softmax layer of the trained network model,we find the two adjacent labels with the highest probability.According to the location of the two labels,the degree of waste is defined as five levels: severe waste,very waste,general waste,mild waste and no waste.The corresponding accuracy measurement standard was also given.Finally,the best performing Inception V3 model achieves a waste degree level detection accuracy of 98.47% on test set data.(2)To solve the problem that mass measurement-based detection methods are difficult to collect training samples and can only detect a single dish,this paper further proposes the detection of dish waste degree method based on image segmentation technology.A total of 4589 images were collected from a university canteen,with a total of 27 categories of dishes.Each image contains 1 to 5 dish samples,and all images contain a total of 13024 dish samples.During the collection process,a fixed camera was used to capture the change process of dishes from presence to absence.Label the collected data with labelme software,mark the edible food area,and make the dish image segmentation data set.Using Swin Transformer and Uper Net as the segmentation framework,and the Uper Net was improved by adding a transpose convolution layer before the bilinear interpolation layer of the Uper Net.Finally,the average intersection over union of the improved model on the test set can reach 93.56%,which is 1.26% higher than the original model.When there are multiple samples of the same category of dishes in an image,the semantic segmentation model can not distinguish the different sample individuals very well,so this paper introduces the instance segmentation model Mask R-CNN to solve this problem effectively.Meanwhile,this paper also established the level of waste degree according to the statistical information of the segmentation area ratio before and after dish consumption.According to statistical information,regression line was calculated,and then according to the transformation relationship between ideal line and regression line,division threshold of each waste level is calculated.According to this dividing threshold,the level of dish waste degree was divided into 5 levels: serious waste,very waste,general waste,mild waste and no waste.Testing were carried out on 1978 dish samples in the test set data.Finally,the best performing semantic segmentation model,Swin-T+M-Uper Net,achieved 94.84% accuracy in waste degree level detection.The level of waste degree identification accuracy of the instance segmentation model Mask R-CNN can reach91.76%.(3)Comparing and analyzing the two detection methods of dish waste degree,we find that each method has its own advantages and disadvantages and is suitable for different detection scenarios.In view of this,based on Py Qt5,this paper develops the detection software of dish waste degree,encapsulates the two detection methods,and presents them in the form of application software,which can realize the functions of calling the camera to take photos to detect and selecting folder pictures to detect,so that users can use them easily.