Design And Implementation Of Automatic Acquisition And Management System For Digital Notices To Mariners

Chart is the basic tool of modern navigation.It is of great significance to ensure the navigation safety of ships that cartographers can quickly and accurately obtain the information of notices to Mariners and use it to correct and update the chart products in time.Notice to Mariners is an important data source for chart production and chart correction.With the development of marine mapping at home and abroad,it has entered the digital stage driven by database from pure manual operation.In recent years,based on the rapid development of Internet technology,notices to Mariners have also changed greatly in the way of release and application.Relying on the network and digital media,compared with the traditional carriers such as paper and telegraph,it has the advantages of fast publishing speed,high efficiency of chart correction application,and labor cost saving.Although at present,according to relevant international treaties,Chinese chart production departments can obtain paper notices from domestic and foreign notices issuing agencies irregularly,but in view of the long cycle of obtaining paper documents by mail,which is not conducive to the protection of the current situation of chart products,it is usually used as data backup or historical document preservation.Therefore,it is an important problem for the chart production department to improve the efficiency of charting personnel by using the computer intelligent technology to accurately,quickly and timely collect the notice data in view of the current digital marine notice release mode mainly issued by the Internet.This paper studies the digital chart correction,content and structure,editing and distribution mode,computer crawler technology,database technology,and so on,according to the specific needs and applications of digital notices to the chart production departments in China,an automatic acquisition and management system for digital notices to Mariners is designed,it includes six main modules that advertisement search strategy,page download,URL duplication,page data extraction,data storage of notices to Mariners,intelligent monitoring and management.According to the structure of the web page,the search strategy module adopts different search strategies,mainly according to the content structure of the announcement web page and the different loading mode of the web page content,adopts different combination modes,and adopts breadth first search and depth first search.The page download module consists of three parts that dynamic web page rendering,automatic navigation interaction of web pages and multi-threaded parallel download,the purpose of multi thread parallel download is to improve the crawling efficiency of the system,the interaction between dynamic web page rendering and automatic web page navigation is based on the Google browser kernel package,which uses the compiled navigation script to simulate the user's operation sequence for single or multiple automatic click loading.The URL duplication module adopts two-layer de duplication method to improve the ability of URL duplication and prevent repeated access to the same URL.Page data extraction is divided into two parts: page preprocessing and data extraction.Preprocessing is mainly to eliminate noise information,and transform language coding into unified coding,then the DOM(document object model)tree is constructed.Data extraction is to extract valuable announcement data according to the corresponding extraction rules according to the hierarchical structure of the web page.According to the requirements of the design of the marine notice database,the marine notice data storage module can persist the marine notice data to the database.The intelligent monitoring and management module is one of the keys to realize the intelligent system.Through real-time monitoring of the operation of different links,it can automatically resume processing or send out manual intervention prompt in time.The practical application in chart production department shows that the research results of this paper can automatically download and incrementally backup the digital notices to Mariners in real time,provide high-quality basic data sources for the integrated chart production system,improve the efficiency of chart cartographers in correcting and updating chart products.