Research On Behavior Recognition Of Post Officer Based On Depth Features

In recent years,the courier industry has boomed along with the advent of the e-commerce era,and the safety issues in the process of courier consignment,collection,and transportation have become a growing concern.Hazardous checking of items in express boxes in the express scene is a basic inspection,and relevant departments require that during the collection process,staff must open and check packages before they are sealed.As a typical abnormal behaviour in the courier scene,collection operations without completed opening and inspection need to be strictly controlled.However,there are many interfering behaviours in the courier scene(e.g.use of mobile phones,mobile couriers and acts of consignment,etc.),complex item semantic information(e.g.a wide range of items,different sizes,etc.),environmental light changes and other factors,which bring great difficulties and challenges to the intelligent analysis and identification of abnormal behaviour of personnel.With the development of video surveillance hardware systems,image analysis,video analysis,human behaviour detection and recognition,and other technologies,intelligent video analysis systems have been widely used on many occasions,such as city roads,petrol stations,airports,and other key security areas.However,for the special abnormal behaviours in complex courier scenarios,general intelligent video surveillance cannot achieve the desired results.The establishment and identification of humanobject interaction behaviour models in courier scenarios is an urgent problem to be solved.Therefore,it is of high value in theoretical research and practical application to carry out research on human-object interaction behaviour recognition algorithms and systems for courier personnel.This paper takes the problem of personnel behaviour recognition in the courier scene as the research object.The main objective is to complete the stable recognition of the opening and inspection behaviour of personnel at courier stations.Combining colour map,depth map,and skeleton point features,this paper will start the research on human-object interaction behaviour recognition from several painful problems such as target segmentation in complex background,human-object interaction model establishment by fusing depth features,and suppression of multi-person-multi-object semantic interference.The main research elements are as follows.(1)According to the technical requirements of the subject,the characteristics of the courier scene and the target behaviour are analysed through an overall design of courier personnel behaviour recognition.Considering that there are multiple courier outlets,a distributed monitoring network is designed,while the hardware selection of image acquisition and analysis equipment is completed based on the characteristics of the courier station environment and the analysis of algorithm complexity.The process of the courier personnel behaviour recognition algorithm was designed based on a comprehensive analysis of complex factors such as the categories of people and items in the courier scene,behavioural semantic information,and changes in the light field of view.(2)Research on semantic segmentation algorithms for courier scenes in complex contexts.To address the problem of target segmentation errors caused by ambient light changes,similar background interference,and different sizes of target objects(e.g.courier boxes)in courier outlets,an improved semantic segmentation network AM-UNet combining multi-scale convolution and integrated attention gate mechanism is proposed.In this paper,we use Multiscale Convolutional Unit to replace the traditional convolutional units in the UNet model,which improves the multi-scale generalization ability of the model and can accurately segment and locate targets of different sizes and fuzzy overlap.The problem of apparently similar targets(e.g.similarly colored hands and boxes)is effectively solved by feeding the feature maps of each layer and its neighbouring layers into the AGs module;to address the problem of missing semantic information of character interaction in the segmentation model,a character interaction relevance prediction algorithm HOIC is proposed to supplement the character interaction information.The experiments demonstrate that the improved AM-UNet semantic segmentation method can achieve accurate segmentation of multi-scale target objects(boxes and human hands)in complex courier scenes,laying a good foundation for the subsequent extraction of features required for behavioural recognition.(3)A character interaction behaviour recognition method based on deep Spatio-temporal features.Aiming at the problem of difficult modelling of human-object interaction models in the process of opening and inspecting boxes,a new feature extraction method incorporating deep Spatio-temporal information is proposed to extract features for boxes and human hands separately.First,semantic segmentation is completed using the UNet colour map to obtain the segmentation mask of the target object,and the extraction of the target object(human hand and box candidates)in the depth map is completed according to the mask.Then,to enhance the spatial information,the depth variance is calculated to dynamically describe the box deformation features;to enhance the time-domain information,Deep Motion Maps(DMMs)are constructed based on the depth map of the human hand,and the LBP features of the DMMs are extracted and fused with the colour map of the human hand motion trajectory to describe the human hand motion features.Finally,the Spatio-temporal feature vector of human-object unpacking behaviour is used as input,and the support vector machine(SVM)algorithm is used to achieve efficient and accurate determination of unpacking behaviour.(4)Human-object interaction recognition based on key poses in courier scenes.To address the problems of other interfering behaviours and objects in the courier scene(e.g.sender opening parcels,solicitor using a mobile phone),a human-object interaction recognition method based on human pose estimation and target detection is proposed by establishing the semantic information of objects and the human-object interaction relationship model.Firstly,we use the skeleton point information extracted from Open Pose to estimate the human posture and identify the initial behaviour category;then,we use the YOLOv5 algorithm to detect the object category and location of interest to address the problem that the semantic information of objects is lost in conventional behaviour recognition methods;we propose a multi-personmulti-object optimal allocation algorithm based on the Auction algorithm to construct the human-object interaction relationship Finally,the initial behavioural labels and the humanobject interaction feature descriptors are fused to obtain the final recognition results,which can accurately identify similar targets and open-box inspection behaviour in a multi-people interference scenario.To sum up,according to the technical requirements of personnel abnormal behaviour recognition in courier scenes,this topic researches the behaviour recognition method based on depth Spatio-temporal features and key pose based behaviour recognition methods and designs a set of fused depth features for human-object interaction behaviour recognition system in courier stations,which contains four parts: user login module,online operation module,offline test module and data management module.Finally,through the experimental verification of video data taken from real courier scenes,it is proved that the courier station staff behaviour recognition system with fused depth features researched in this project can achieve effective recognition of abnormal behaviour of opening and inspection under complex environments,and the accuracy rate of opening and inspection behaviour recognition can reach 96.3% and the recall rate can reach 96.9%,achieving accurate and efficient human-object interaction behaviour in courier station scenes The recognition plays an important role in promoting the prosperous and healthy development of the courier industry.