Research And Application Of 3D Shape Recognition Based On Multi-modal Feature Fusion

In recent years,with the development of computer hardware and 3D scanning technology,the research for 3D shape feature extraction and recognition has received more and more attention.3D shapes and 3D scenes have a stronger sense of realism and are widely used in many fields.Due to the complexity and diversity of 3D shape representation,3D shape recognition has become a hot issue in computer vision.Most current recognition methods are based on a single data mode of 3D shapes,and the extracted features cannot fully characterize 3D shapes.Research on 3D shape recognition methods is gradually shifting towards methods based on multimodal feature fusion.The existing recognition methods based on the fusion of multimodal features have the problems of inadequate expression of modal saliency features and inadequate extraction of relationships among the modalities.In response to the above issues,this paper proposes a 3D shape recognition method based on multimodal feature fusion,including three research elements:each modal feature extraction,multimodal feature fusion,and 3D shape recognition.The experimental results show that the proposed method can more fully extract the salient features of each mode and the relational features between modes,and obtain more comprehensive feature information,thus improving the accuracy of 3D shape recognition.The main research works of this paper are as follows:(1)In the feature extraction stage,the 3D point cloud feature extraction method of Point-View Relation Network(PVRNet)is improved to address the problem of losing 3D shape saliency features due to uniform point extraction.Firstly,the enhanced 3D point cloud is generated by using the adversarial generation learning strategy,and the enhanced 3D point cloud pays more attention to the salient features of 3D shapes,and finally,the 3D point cloud modal features are extracted.Meanwhile,for the problem that features extraction may be inaccurate due to different 3D shape multi-views collected from fixed viewpoints,the multi-view feature extraction method of PVRNet is improved,and the multi-view feature extraction method based on adaptive viewpoints is proposed,which generates 3D shape multi-views by adaptively changing the viewpoint position.Finally,extracts multi-view modal features.(2)In the multimodal feature fusion stage,the PVRNet multimodal feature fusion method is improved to address the problem of inadequate extraction of multimodal relationships in the PVRNet model.Firstly,the loss function of each modality is introduced to preserve the information of each modality.Secondly,the step of deleting multi-view features in the original model is optimized,and the relationship features between and within each modality are extracted at the same time.Finally,the relationship features,3D point cloud modal features,and multi-view modal features are fused to form a comprehensive and rich global feature for 3D shape recognition.(3)To meet the user’s demand for 3D shape data management,a 3D shape recognition system is designed and developed based on the algorithm proposed in this paper,which mainly includes functions such as login,3D shape file operation,modification,feature extraction,recognition,and result display,and exit from the system.