Research On Limb Movement Recognition Algorithm For Neonates In Incubators

With advances in medical technology,the survival rate of neonates born at less than full term has greatly improved,but they are still at risk for developing developmental neurological disorders.Therefore,real-time monitoring of neonates in the incubator is critical for the diagnosis of short-and long-term complications.Currently,the neurological development and physical condition of newborns is still assessed clinically by physician observation of the whole body movements of newborns in the incubator.However,this manual approach relies on the clinician’s empirical knowledge,is limited by the physician’s subjective judgment,and does not have a uniform standard.In addition,this approach cannot achieve real-time monitoring.With the development of deep learning,it is a practical solution to build an intelligent recognition model of neonatal limb movements based on deep learning to realize real-time detection of neonates in incubators 24 hours a day.However,the difficulties in building an intelligent recognition model for neonatal limb movements are: 1)lack of reliable real video data samples of clinical neonates; 2)blurring of edges and limb contours caused by the incomplete unfolding and overall flexion of neonatal limbs; 3)many false feature points intermingled in video frames due to the occlusion of limbs by medical equipment in the incubator; 4)involuntary subtle movements of neonates causing limb movements The recognition distinction threshold is blurred.To address these problems,this paper first constructs the JMCH database of neonatal limb movements in incubators,and around this database,the following research work is carried out:(1)The text proposes a 3D ResNet network model based on edge information enhancement for the problem of neonatal limb flexion and curl and self-obscuration of medical devices.First,Laplacian convolutional layers are proposed for image frame sequences to perform edge enhancement,highlight contour detail information,and filter out spurious feature points.Then,a 3D residual convolutional network is used to focus on extracting spatio-temporal features of neonatal limb contours to improve the model’s ability to spatially model neonatal limb information,and then optimize the model’s performance for limb action recognition with high feature extraction requirements.Finally,the effectiveness of the analyzed scheme is verified by edge enhancement operators,ablation experiments of operator templates and comparison experiments of network models for different neonatal limb action recognition,which achieves83.44%,85.34% accuracy and F1-score in the limb action classification task.(2)To address the problem of recognition confusion caused by involuntary subtle movements of neonates,NLMNet,a neonatal limb movement recognition network model based on multi-task multi-label learning,is proposed.the model output is improved by using multilabel soft coding to map neonatal limb movements into an ordered arrangement matrix of head and limbs for more accurate classification of multiple limb movements.A feature extraction network based on R(2+1)D separated spatio-temporal convolution was designed for the neonatal limb movement type classification assistance task as a way to provide information on the number of limb movements and inject them into the underlying feature map.The redefined feature map encodes features that contribute to the multi-label classification of neonatal limb movements,reducing the model’s misclassification rate for involuntary subtle movements of neonates.Finally,the effectiveness of the scheme was verified by the selection of network locations for the auxiliary task,the ablation experiments of the joint task loss ratio,and the comparison experiments of network models implemented for different neonatal limb movement recognition,which could achieve 93.7%,90.00%,88.04% of the average accuracy,the average F1-score per class,and the average overall F1-scores.(3)A video-based intelligent video monitoring system for neonates is developed,based on the GUI graphics tool in PyQt,and the multi-task multi-label learning network model for neonatal limb movement recognition proposed in this paper is applied to the system.The system has the functions of video stream input,action recognition and log record export,and provides both offline video and online monitoring to identify the specific moving limb parts through neonatal limb action video,which can assist clinical medical personnel to diagnose the whole body movement status of neonates and carry out next step analysis.