Research On Space Time Image Velocimetry For River Surface Velocity Measurement Based On 3D Vision And Deep Learning

Timely and accurate acquisition of river surface flow velocity is essential for flow measurement,flood warning,hydraulic engineering planning and design,and navigation.Accurate measurement of river surface flow velocity is not only a frontier problem in the field of hydrological science and technology,but also a key research direction and a challenge in the field of water resources.At present,the river flow measurement includes two methods,the traditional sensor-based measurement and image-based measurement.Among them,the traditional sensor-based measurement method has many shortcomings such as low automation,low measurement efficiency,poor anti-interference ability and easy damage of instruments.Especially during floods,image-based flow velocity measurement methods have great advantages.The emerging Fast Fourier Transform-based Space-Time Image Velocimetry(FFT-STIV)technology,as a typical representative of the image-based flow velocity measurement method,has demonstrated excellent performance in terms of measurement efficiency and accuracy under the normal operating conditions.The core of this method lies in the recognition of the main direction of the spectral image and the measurement of the actual length of the velocimetry line,which directly affects the accuracy of the flow velocity measurement process.Camera calibration is an indispensable step before solving the actual length of the velocimetry line,and the existing Direct Linear Transform(DLT)algorithm calibration ignores the nonlinear aberrations,has limited accuracy,and requires on-site measurement of multiple 2D/3D point pairs,which is difficult to implement.Existing spectral image main direction recognition method is polar coordinate system method,the method can show good performance under good imaging conditions,but it is difficult to adapt to the complex river environment,anti-interference is not strong,once there is a more complex field environment,the method may fail,appearing similar to the outlier results.In order to cope with the above challenges,this study,relying on the Beijing Science and Technology planning project,"Research and Development and Demonstration Application of the Grand Canal Intelligent River Patrol System"(Project No.Z201100001820022),and takes FFT-STIV technology as the research object,and accurately determines the surface flow velocity of the river as the research objective,we have carried out research on 3D vision-based STIV velocimetry line length measurement method,pixel coordinate back projection research considering the effect of water level change,deep learning-based spectral image principal direction recognition research,river surface flow velocity correction research considering the effect of wind,and the design and development of a river surface flow velocity measurement system based on 3D vision and deep learning,to realize non-contact and fast acquisition of flow velocity data.The feasibility and accuracy of the method proposed in this paper were initially verified through theoretical validation in a laboratory environment.Subsequently,an application demonstration was carried out in an outdoor research river section to further validate the practicality,reliability and stability of the proposed method.Through in-depth research,the results achieved are as follows:1.Aiming at the shortcomings of the existing DLT algorithm in camera calibration,such as limited accuracy and difficult implementation,this paper designs a more fine and flexible "Riverbank intrinsic parameters calibration-Water surface extrinsic parameters calibration" scheme.That is,the camera calibration is carried out on the riverbank based on the Zhang’s calibration method to obtain the intrinsic parameters and distortion coefficients of the camera.Then,a calibration plate is placed horizontally on the water surface,which is used to establish a 3D spatial coordinate system on the water surface.Then,based on the Pn P(Perspective-n-Point)problem to solve the attitude(extrinsic parameters)of the fixed-point camera on the riverbank relative to the water surface calibration plate,and finally realize more refined calibration of water surface camera parameters.Finally,a high-precision mapping from the image plane to the object plane is established to solve for the length of the STIV velocimetry line.2.Aiming at the problem that the change of water level causes the original 3D spatial coordinates of pixel points to shift,this paper proposed a pixel coordinate back-projection method considering the influence of the change of water level.The method calculates the scaling factor Δs of the planar spatial coordinates relative to the camera’s calibration plane after a change in water level by solving for the camera’s attitude angle and utilizing principles such as perspective projection,thus solving for the 3D spatial coordinates corresponding to the pixel points after a change in water level.Meanwhile,this paper gives a detailed mathematical derivation process,which effectively solves the problem of deflation of pixel size in 3D spatial coordinate system due to water level change.3.Aiming at the shortcomings of the existing method of recognizing the main direction of the spectrum image through the polar coordinate system,such as weak anti-interference and difficult to adapt to the changing imaging environment,we constructed a set of technical routes for recognizing the main direction of the spectrum image by integrating deep learning.The introduction of deep learning provides new ideas and tools to convert the spectrum main direction recognition into an image classification task,which effectively improves the recognition accuracy and robustness.The order of accuracies of different models is as follows:YOLOv5s>YOLOv8s>Goog Le Net>VGG-16>Res Net-50>Res Net-18>Le Net-5>Alex-Net>Polar coordinate system method,with the MSE of 0.13、0.14、25.01、25.03、25.15、25.20、95.22、99.85、4024.76,respectively.4.That the observed surface flow velocity of rivers during windy weather conditions incorporate wind-induced effects.Compared to the windless condition,when the wind is downwind(wind force is in the same direction as the water flow),the surface flow velocity is larger,and thus the virtual flow rate is larger;when the wind is upwind,the surface flow velocity is smaller,and thus the virtual flow rate is smaller.In order to obtain more accurate flow measurements under windy conditions,the surface flow rate needs to be determined by taking into account the effect of wind on the water surface to accurately calculate the virtual flow rate.In this paper,the relationship between wind force and surface flow rate is quantitatively analyzed by constructing a multi-model combination correction model of CART,XGBoost,and KNN,which realizes the effective correction of surface flow rate under windy environment,and provides accurate inputs for the calculation of the subsequent virtual flow rate and the flow test.5.We designed and developed a 3D vision and deep learning based water reporting system,and constructed a high-definition,intelligent video monitoring network with the help of SRS(Simple Realtime Server),which is an open-source streaming service that supports various real-time streaming protocols.Based on Spring Boot back-end framework and Vue3 front-end framework,we build backend services and Web front-end visualization pages to provide services such as real-time video monitoring,event management,data chart visualization and system management,and realize real-time and stable acquisition of flow rate data.This research is expected to achieve technological and application innovation,technical breakthroughs in image method speed measurement,the formation of a complete set of products with promotional value,the engineering application of image method speed measurement in China has a strong reference significance,the scientific and practical value is outstanding.