Diver's Sign Language Recognition Method Based On Deep Learning

Divers are a dangerous and important job.They will work on construction,surveying,demolition,etc.and they will work in 300 meters or deeper below the surface of the ocean.To do those jobs,saturated divers will have to work for weeks in highpressure environments.With the development of marine robot technology,a large number of marine robots began to assist divers with the work.Sign language has also become the only way for divers to interact with marine robots.Therefore,the robot can quickly understand the diver's sign language movements,which is of great significance to assist the diver in his work.In order to realize the diver's sign language recognition,this paper first identifies the position of the diver's gloves,extracts the sign language image,and then compares the sign language image with the sign language library to realize the diver's sign language recognition.The Diver Sign Language Set uses the open source data set CADDY Underwater Gestures Dataset(CUGD).In this paper,we selected 36 sign language actions from the CUGD data set,and due to the limitation of sign language action samples,this paper uses the Generative Adversarial Networks to enhance data and supplement some sign language samples.In order to generate high-definition sign language images,this paper adopts the method of Generative Adversarial Networks training based on the gradual growth,which effectively solves the problem of generating high-definition sample generation in the Generative Adversarial Networks.The experimental results show that the Generative Adversarial Networks can generate clear images of diver sign language,and greatly expand the number of experimental samples.For the sign language recognition,in order to reduce the workload of sign language comparison and the training calculations of deep learning model,this paper first uses the Mask R-CNN algorithm to extract the diver's sign language position.The Mask R-CNN algorithm can obtain the outer outline of the diver's glove through the key point of the diver's glove.In turn,we got the regions of interest in sign language.Then,Siamese Neural Network compared the target image with the knowledge in the sign language recognition library,to obtain specific sign language semantics.The sign language comparison method based on Siamese Neural Network is not only highly accurate,but also scalable.That makes it easy for the diver's opponent to expand and modify,which is adapted to the more complex underwater work.Experiments show that the Mask R-CNN algorithm can predict the outer border of the diver's glove,and the recognition accuracy of the Siamese Neural Network can reach the standard classifier as VGG,which can be competent for sign language semantic recognition.In this paper,we proposed an underwater behavior prediction algorithm of divers based on deep learning.In addition,the marine robot system can identify the diver's sign language action effectively and accurately.The algorithm is helpful to the intelligent control and assistance of marine robot system with divers.