Machine Vision-based Railway Rail Fastener Detection And Recogniton

Rail fasteners are very important railway infrastructures for keeping the gauge unchanged.However,the lateral force will impact the rail fastener when the train passing,which will result in the loosing,damage and even missing,and further lead to major safety accidents,such as train derailment.Therefore,we should regularly inspect the status of rail fasteners in the railway lines to ensure the safety of railway operation.In this paper,we take the rail fastener as the main research object,and thoroughly study the application mode of machine vision-based automatic inspection technology in rail fastener inspection task.The research contents include object detection and image recognition in the complex scene image.Our proposed methods not only can improve the effectiveness and realtime of the rail fastener inspection system,but also can enhance the generalization ability in different environmental railway lines.More importantly,the proposed methods realize the purpose of inspecting the status of rail fasteners efficiently and accurately,which have important theoretical significance and practical value.The contributions of the dissertation are as follows:1)The existing rail fastener regions detection methods have some shortcomings,such as poor accuracy,slow speed and insufficient generalization ability.Therefore,we propose a rail fastener regions detection method based on online learning strategy.Firstly,we detect rail fastener candidate regions by using the prior information of rail infrastructures.Secondly,we calculate similarity between each sub-window of rail fastener candidate regions and each rail fastener template by using K nearest neighbor algorithm and template libraries.Subsequently,we infer rail fastener regions according to the similarity score and the location distribution of rail fasteners.Finally,we dynamically update the online template library by using online learning strategy,which can make the method to adapt different lighting conditions,different image backgrounds and different types of rail fasteners.The experimental results show that our proposed method has achieved good performance on the rail fastener region detection datasets on different kinds of railway lines.Moreover,it does not need training process,and the detection speed is very fast,which can meet the demands of the rail fastener inspection task.2)Due to the location distribution of rail fasteners in the turnout section is irregular and multi-directional,the above proposed method may not be able to detect rail fastener regions in the turnout section.At present,the object detection methods based on deep convolution neural network can detect the object at any position in the image.However,they can not detect the missing or covered rail fasteners.Therefore,we propose a real-time detection method of multi-directional rail fastener based on scene context-aware network.Specifically,we first propose a multi-scale residual neural network,which not only can enhance the feature learning ability,but also can reduce the complexity of the deep network.Secondly,we propose a real-time detection method of multi-directional object based on multi-scale residual neural network,which can predict the bounding box of the target on the multi-level fusion feature maps by using multi-directional anchor boxes.Finally,we propose a scene context-aware network,which can train multi-scale residual neural network to learn the scene context information by utilizing the differences between the track image with missing or covered rail fasteners and the track image without missing or covered rail fasteners.So,the network model can detect the missing or covered rail fasteners.The experimental results show that our proposed method has achieved high accuracy and generalization ability in different railway lines,which not only can detect visible rail fasteners in turnout sections,but also can detect the missing or covered rail fasteners accurately.In addition,it also has achieved competitive performance in object detection dataset and natural scene text detection dataset.3)When performing rail fastener inspection task in different environmental railway lines,the existing rail fastener defects recognition methods need to collect and annotate a large number of rail fasteners,and re-train classifiers.Even more unfortunately,the training set is often unbalanced because the number of defective rail fasteners is very small,which will result in poor accuracy of the classifier.In this paper,we propose a rail fastener defects recognition method based on visual similarity network.Firstly,we use the rail fastener regions detection method based on online learning strategy to collect and annotate a large number of rail fasteners automatically.And then,we match normal fasteners and defective fasteners randomly to obtain a large number of rail fastener pairs.Secondly,we construct the visual similarity network to learn the visual similarity of rail fastener pairs to obtain the pre-trained model.When performing the rail fastener inspection task on a railway line,we only use a small number of rail fasteners to fine-tune the pre-trained model,and the rail fastener defects recognition model can be obtained.The experimental results show that the classification performance of our proposed method is better than that of the existing methods,which improves the accuracy of rail fastener defects recognition.Moreover,it can apply to different environmental railway lines by using fine-tune strategy.4)Rail fastener loosing is a very small detail change,and the forms of that are diverse.The existing image classification methods can not accurately identify the rail fasteners loosing.Thereforce,we propose a rail fastener loosing inspection method by learning image representative features.Firstly,we analyze the principle of loss functions of current image classification methods in detail.And then,we propose a novelty loss function,namely constrictive annular loss,which can train the deep convolutional neural network to learn more representative features.In practical application,we extract the features from one normal fastener and an input fastener,and calculate the cosine similarity between two features to identify whether the input fastener is loosing or not.The experimental results show that our proposed method has achieved high accuracy and good generalization on identifying rail fastener loosing.Moreover,it not only can apply to face verification task,but also can improve the classification performance of deep network on imbalanced and large-scale datasets.