| As the supplement of video monitoring,urban sound monitoring can remedy for video monitoring deficiencies affected by weather,occlusion and so on.Effective urban sound monitoring can improve people’s acoustic comfort,give warning to various places in the city,and provide a lot of valuable guidance information for city managers.Urban sound recognition(USR)is the core content of urban sound monitoring.For a long time,sound signal processing with complex noise environment has been the focus and difficulty of researchers.In this study,typical urban sounds are taken as the research object,and the research on its feature extraction and recognition are carried out.A method of USR based on convolutional neural network(CNN)is discussed,which provides the core theory and technical basis for urban sound monitoring.Therefore,a deformable feature map residual network(DFM-Res Net)is proposed to solve the problem of typical USR performance degradation caused by the variable scale,complex geometry and irregular contour edge of the energy concentration region in the logarithmic Mel spectrogram(Log-Mel-spec).Meanwhile,transfer learning,sound data enhancement and feature re-weighting are proposed to further improve the urban sound recognition performance.The main work of this paper is as follows:(1)Fully investigate the research status of USR methods at home and abroad,analyze time-frequency characteristics of typical urban sounds,and deeply study the relevant methods of sound time-frequency image feature extraction and recognition based on CNN.Through characteristics analysis,time and frequency domain characteristics can better describe the urban sound features.Because of the strong background noise of urban sounds,Log-Mel-spec is a good choice to show the time-frequency characteristics,which can enhance the display of the frequency component with lower energy.However,there are some problems in the energy concentration region of Log-Mel-spec,such as variable scale,complex geometry and irregular contour edge.Compared with traditional methods,the time-frequency image recognition based on CNN is better,but convolution kernel often sample in uninterested image region in the traditional CNN.Therefore,further improvement of CNN is needed to solve the problem of USR with background noise.(2)A multi-level feature fusion network is proposed for typical USR.In order to adapt to variable scale of the energy concentration area in urban sound Log-Mel-spec and reduce the loss of feature information caused by the process of pooling and so on,a multi-level feature fusion network(MFFNet)is designed.MFFNet adapts to the scale change of the energy concentration area through Inception block composed of convolution with different size convolution kernels.In order to reduce the information loss in feature extraction,a shortcut is designed to fuse the feature map not pooled with the higher level features of the network.MFFNet is used in the typical USR.Meanwhile,data enhancement and parameter transfer are proposed to reduce the impact of small sample problem of urban sounds on recognition performance.(3)A deformable feature map residual network is proposed for typical USR.Because the convolution kernel size,shape,sampling location and sampling points of MFFNet proposed in(2)are fixed,it can not adapt to the changes of the geometric structure and contour of the energy concentration area in the urban sound Log-Mel-spec.For this reason,DFM-Res Net is proposed.The core of DFM-Res Net is a deformable feature map residual block,which mainly includes offset layer and convolution.In the offset layer,the pixels of the input feature map are shifted,and the shifted feature map is overlapped with the feature map extracted from the convolution through shortcut,so that DFM-Res Net can focus on the interested area of the feature map for sampling,and transfer the information of the shifted feature map to the higher network.DFM-Res Net is used to the feature extraction and recognition of typical urban sounds.In the same way as in(2),data enhancement and parameter transfer are also carried out for typical urban sounds,and re-weighting block is used to solve the problem of feature weight distribution in each channel.(4)Design and carry out relevant verification experiments to verify the effectiveness of MFFNet and DFM-Res Net and the advantages of typical USR methods based on MFFNet and DFM-Res Net.The results show that MFFNet and DFM-Res Net are effective for the typical USR,and DFM-Res Net has better performance.Compared with the optimal methods in published literatures,the results of typical USR based on MFFNet and DFM-Res Net are improved,and the result of typical USR based on DFM-Res Net is better.It is proved that the research in this paper is effective and has certain reference value for the sound signal processing in noisy environment. |
