The Design And Implementation Of Cross-platform Test Script Record And Replay System Based On Image Recognition

Automated testing is characterized by high efficiency and high reliability,and has gradually become the mainstream testing method in the development of mobile applications.Mobile applications often run on multiple models and multiple systems.The "fragmentation" of devices and systems makes it difficult to implement cross-platform execution of test scripts in existing automated testing frameworks,resulting in high cost of test scripts build and maintenance.This thesis designs and implements an automated testing script record and replay system based on image recognition to solve the problems of high cost of test scripts build and maintenance.The system simulates the process of manual testing.On the one hand,it provides remote control of the device,and the development process of test scripts is simplified by the script record and replay mode,which reduces the techni-cal threshold of test scripts development.On the other hand,the script record process generates scripts by adding information such as screenshot and layouts position of the component.The script replay process uses the image recognition technology to locate the component on the new device,which realizes cross-device and cross-platform re-play of the script,thereby avoiding scripts of repeating builds and maintenance.The system is mainly divided into a device management module,a script record module and a script replay module.In design,the services are developed separately,distribut-ed deployment,and RESTful interfaces are used for communication between modules,which reduces the coupling between modules and improves the scalability of the sys-tem.Through the encapsulation of ADB(Android Debug Bridge)and WDA(Web Driver Agent)tools,unified management of Android and iOS devices is realized.The system makes full use of the MiniCap tool to obtain screenshots of the device page at high speed,and establishes a TCP stable transmission channel between the web and the device through the Netty framework to realize real-time synchronization of the in-terfaces at both ends.The system makes full use of SIFT and OCR(Optical Character Recognition)technology to process pictures and layout information to complete com-ponent positioning and achieve cross-platform replay of scripts.This thesis selects 10mainstream mobile devices and 5 mobile applications to an-alyze and verify the system availability.The experiment proves that the record process of this system has better support for different devices under the Android platform,and the script cross-device replay success rate is over 80%.The cross-platform replay mod-e has a success rate of about 60%.The system has been deployed for internal mobile application testing.Through a single record of test scripts and cross-platform replay of scripts,it effectively reduces the number of test script maintenance and the cost of automated testing.