Research On Automatic In-field Rice Panicle Counting Based On Object Detection

Rice is the major grain crop in China,ensuring the security of rice yield has a great significance to maintain social stability and sustainable economic development.In the process of rice production,Estimated output is essential and can provide an important decision basis for the scientific production and economic regulation.When we estimate the rice yield,number of grain per unit area,setting rate and 1000-grain weight are the three main indexes we need to consider.Meanwhile,the number of grain per unit area is the most relevant agronomic traits of yield,generally calculated by the panicle number per unit area and grain number per panicle.Therefore,obtaining the rice panicles per unit area accurately is the key to realize the rice yield estimation.However,at present,the number of rice panicles per unit area mainly depends on manual investigation,which is time-consuming and laborious.In view of this,it is particularly necessary to develop a fast and accurate method for the automatic rice panicle counting.With the rapid development of object detection algorithm,the counting method based on object detection and deep learning has gradually become the mainstream of the automatic rice panicle counting method.However,there are some limitations in practical applications due to the complexity of field scene and the constraints of general object detection algorithms’ generalization ability,especially for small-sized panicles and panicles locally covered by rice leaf.Therefore,in this study,we commit to improving the general object detection algorithm for the small-sized object,occluded object and occluded small-sized object,and significantly improve the counting accuracy of small-sized panicles,occluded panicles and occluded small-sized panicles.The main innovations of this study include:（1）Collected and constructed 4 different rice panicle detect data sets.In order to count the small-sized and occluded rice panicles automatic,we screened rice panicles in the actual field scenes.In this study,the Nanjing 46 was used as the research object,and on this basis 4 different data sets of PANICLE2017,PANICLE_MHW,PANICLE_OCCLUDED and PANICLE_SAO were collected and produced.In the data set production,we carry out effective cutting and occlusion simulation,and manually annotate all training images.（2）To solve the problem of insufficient feature information of small objects,we propose a detection algorithm for small-sized objects based on the multi-scale hybrid window（MHW-PD）.The background of the in-field panicle images is complex.For the small-sized rice panicles,its limited original information will constantly lose in the abstract process of feature learning,which leads to the very low detection and counting precision.Therefore,in this research,we study the enhancement effect on small object’s features by constructing multi-scale hybrid window,and propose a detection algorithm named MHW-PD for small-sized objects based on the multi-scale hybrid window,to detect and count the small-sized rice panicles from the in-field images.The hybrid window provides rich and detailed multi-scale feature expression for the tiny rice panicles,which effectively improves the coverage of the proposal regions to the small target in the detection process.For the PANICLE2017 dataset,the precision of MHW-PD is 95.4%;while the counting accuracy of MHW-PD is 87.2%,which is about 50%higher than that of the classic Faster-RCNN.The results show that the MHW-PD could significantly improve the counting accuracy of small-sized rice panicles in the field.（3）To remove the noise in the feature of occluded objects,we propose a detection algorithm for occluded objects based on the sample generation and feature completing（GFC-PD）.In the actual field,rice grows densely and the panicles are often covered by the leaves more or less.In the process of feature learning,the local leaf noise will gradually spread to the global feature with high semantics,which inhibits the feature quality of occluded rice panicles,resulting in a very low detection and counting accuracy.Therefore,in this research,we study the enhancement effect on occluded object’s features by generating occluded samples and completing feature noise,and propose a detection algorithm named GFC-PD for occluded objects based on the sample generation and feature completing,to detect and count the rice panicles occluded by leaves from the in-field images.Firstly,we design the occluded sample generation module to expand the richness of the occlusion scene in the model training data.Secondly,we design the feature noise completing module to repair the occlusion noise as the target feature.By improving the feature quality of the occluded object,the detection error can be effectively suppressed.We validate the GFC-PD on the VOC2007,VOC2012,COCO and PANICLE_OCCLUDED dataset.For VOC2007,the mAPs of GFC-PD are 50.5%,61.1%and 65.1%for the large,medium and small occlusion scales respectively.For PANICLE_OCCLUDED,the precision and counting accuracy of GFC-PD are 84.8%and 99.1%respectively,which are 10.6%and 5.2%higher than the Faster-RCNN and about 2%and 4%higher than similar counting algorithms.The results show that GFC-PD can significantly improve the counting accuracy of occluded rice panicles in the field.（4）On the basis of the above two works,we propose a detection algorithm for occluded small-sized objects based on the feature pyramid completing network（FPCN-PD）.The actual field rice panicle is a kind of complex target with the characteristics of small and occlusion.Compared with the previous two works,the interaction between feature noise and feature information loss reduces the proportion of the panicle features in the global feature map,and increases the proportion of the noise in the panicle features,which resulting in a lower detection and counting accuracy of the occluded small-sized rice panicles.Therefore,in this research,we study the enhancement effect on occluded small-sized object’s features by generating and completing feature pyramid,and propose a detection algorithm named FPCN-PD for occluded small-sized objects based on the feature pyramid completing network,to improve the rice panicles automatic counting level further.By introducing the feature pyramid completing network,the noise caused by occlusion can be effectively cleaned while the feature richness of the small-sized target can be effectively improved.For Bottle and Potted Plant datasets,the mAP of FPCN-PD are 48.3%and 42.4%respectively,which is more than 10%higher than the Faster-RCNN.For PANICLE2017 dataset,the precision and counting accuracy of FPCN-PD is 90.8%and 98.3%respectively.The results show that FPCN-PD can be applied to the automatic counting task of small-sized rice panicles in the field which are partially occluded by the leaves.