Image-based Chinese Dish Segmentation And Recognition

Food image recognition is one of the research hotspots in the field of computer vision in recent years.Vision-based food recognition technology not only helps people quickly understand unfamiliar foods,but also can be used in a range of artificial intelligence applications such as dietary preference analysis and recommendation,nutrient analysis and calorie measurement,dietary health assessment and recommendations,which has the broad application prospects.However,current research on food recognition is mostly limited to Western and Japanese dish.There are many varieties of Chinese dish,and the shape of the same dish tends to be diverse,while the dishes with similar appearance may belong to different categories,so the visual recognition of multi-target Chinese dishes is still an unresolved challenge.This paper takes the school canteen as a typical application scene to study the recognition of multi-target Chinese dishes.In this paper,the following technical route is adopted:for the image of the canteen tray for cooking,firstly,the unsupervised image segmentation algorithm is used to extract each Chinese dish region,and then the extracted target regions are sent to the deep convolutional neural network one by one for classification.The main work content and innovative achievements include the following three aspects.(1)In the absence of a large,open data set for the detection of Chinese dish,a traditional non-learning multi-target Chinese dish image segmentation algorithm based on local variation and region analysis was proposed.The segmentation algorithm requires no training and can directly locate and extract dishes targets.After the input image is preprocessed by the guided filtering,the SLIC algorithm is used to produce superpixels,and then the superpixels are used as the vertices,and the Euclidean distance of the LAB feature between the superpixels is used as the edge,and the relaxation graph model is constructed.The idea of local variation further aggregates supeipixels to form different regions,and finally extracts the target regions of each dish through the region analysis algorithm.In this paper,the boundaries of 664 dishes in 162 tray images are manually labeled as reference segmentation.The experimental results show that the segmentation performance of the proposed algorithm is better than the existing algorithms,which can effectively extract the target regions of each dish.(2)A dual-stream convolutional neural network based on bilinear structure was proposed to recognize single-target Chinese dish images.The dual-stream design with different fields of view is used to extract the image representation of Chinese dishes with higher confidence.The translation invariance of texture features is realized by bilinear structure,and the complexity of network model is reduced by compact bilinear pooling.The stream connection method uses additive fusion to achieve the fusion of dual stream feature information.Aiming at the problem that the number of pictures in each category of the data set is not balanced during the training process,the weighted cross entropy function is proposed,and the network over-fitting phenomenon is reduced by calculating the confusion loss.The test results on the open data set containing 208 Chinese dishes show that the Top-1 accuracy rate of the proposed network is 81.16%,and the Top-5 accuracy rate is 96.32%,which is significantly better than the recognition accuracy using the existing classic network.(3)According to the canteen application scene,our campus canteen dish data set was constructed,and a multi-target Chinese dish recognition algorithm was realized.Taking into account the individualization of the school canteen dishes in the category and practice,this paper developed a special data set for our campus canteen dishes,including 50 kinds of dishes,a total of 6,891 pictures,basically covering the common dishes in our campus canteen.The dual-stream neural network proposed in this paper is pre-trained by ImageNet and ChineseFoodNet,and then migrated to the canteen dish data set for fine tuning,which optimizes recognition performance.The experimental results show that the Top-1 accuracy rate is 95.28%,and the Top-5 accuracy rate is 99.86%.Combined with the Chinese dish segmentation algorithm and recognition network proposed in this paper,the extraction and classification of Chinese dish targets in the canteen tray image are realized one by one.