Network File Download Based On Multi-threaded

Linux as the Internet era of the most popular free operating system, more and morepeople favor it. But at the same time under the platform of the download software is rarely,and supports of multi-threaded breakpoint continuingly download little. The system notonly supports the breakpoint continuingly functionality, multi-threaded download, HTTPtunnelandinaccordancewiththeprincipleofthenatureofthefirewallwithapenetratingDownload tools, the primary task is accurately, a byte can not go wrong. Forsingle-threaded to complete it is relativelysimple, but multi-threadingis taken into accountthe relatively more on, for example, distribution of tasks between threads, the contents ofthe paper splicing, mutually exclusive ways to access files and so on. And the thread to beable to record accurately the breakpoint information for the next resume. Followed by thefast, if the network is smooth, so fast, single-threaded and multi-threading is not muchdifference, but the present situation is especiallycrowded network, multi-threaded so it canplay a strong point, if a thread is blocked in several other were also normal, butnoninterference. Multi-threaded download line to take full advantage of the bandwidth andimprovetheefficiencyofdownload.The design is under Linux systems POSIX (Portable Operating System Interface) as astandard multi-threaded. The origin of multi-threaded multi-CPU machines, developed intoa more advanced programming ideas. It is mainly characterized by the concept of parallelprocessing. Thread is included in the process of an entity; it has run its own clues, tocompletecertaintasks,andprocesses toachievea similarconcurrencywithotherthreads tocompletetheprocessofcollaborativetasks.Processoftenreferredtoaslightweightthreads.Useoneofthereasonsformulti-threadedprocessiscompared,itisavery"thrifty"modeofoperation of the multi-tasking. We know that inthe Linux system, start a new process mustbeallocatedtoitsseparateaddressspace,theestablishmentofalargenumberofdatatablesto safeguard its code segment, stack and data segment, which is an "expensive"multi-tasking work. And a process running on multiple threads, among them use the sameaddress space, sharing most of the data, start a thread has spent far less than the space ittakes to start a process space, and thread switch between the time required for each other isfar less than the process of switching between the time required. Of course, in the specific system, the data may have a greater difference. The reason for the use of multi-threadedthread between the two is a convenient communication mechanism. Different process, theyhave separate data space, it is necessary to transfer data only through the means ofcommunication, not only in this way is time-consuming and inconvenient. Thread is not,under the same process of sharing data between threads of space, so a data thread can bedirectly used for other threads, not only of speed, convenience, and improved downloadperformance.The beginning of this article describes some of the associated technical, threadcreation and exit, thread synchronization and mutual exclusion, the security thread, and itsrelated API call, which is the basis for the realization of parallel downloads. In this paper,based on the request / response model of HTTP protocol, in the download prior to theHTTPservertosendtheHEADrequest,theserverreceivedarequest,theconcurrentreturnon the resources in response to download message header, through the analysis of theresponse messages, the Content Analysis -Length entity header of the content domain, thesize of the file can be downloaded on the file partition, for each thread a piece of paper todownload the distribution of tasks, this process is sent through the GET request to theserver to achieve, before that, by calculating each start a thread to download the locationand end location, a number of threads to download each receiving different tasks, do notrepeat each other before, so that the whole document can be divided into N blocks,respectively, can be downloaded, after the completion of each task thread , the recyclingofresources,duringwhichtheuseofwrittendocumentstoavoidduplicationwriteMutex.Breakpoint Continuingly prior to the realization of large files in accordance withpre-divided into N size block of transmission at the same time to create N threads toconnect the target server-side. When the server received a connection termination request,inform the client can transfer files. Termination when a customer can receive the news oftransferring files, the first transmission to the server sending data block (including the firstfew in the beginning of block) request termination when the server received the request,send data to the client transmit information, client data and then transmit the informationspecified in the data block to the server-side, server-side data transfer to update theinformation block. In the margins of the download to check the local information todetermine the numbers of bytes have been downloaded. We can create a temporary file fordownload information, such as the number of bytes has been downloaded and so on. Tocheck the output file exists, if it exists, and then get its size, and has been downloaded as apartof.Eachtransmissionisstopped,ifnotwrittenintothewillofitstemporaryfile.Inthis way, even if the network has been broken, it does not matter because we have alreadydownloaded content and thelocationhas beenrecorded,the next downloadfrom the recordstart position of the breakpoint to continue the download, this is the BreakpointContinuingly. Therefore, users do not have to worry about sub-large files, downloadmulti-thread synchronization, if the half-way network, part of the file would have causedlossoftimetowasteandtheInternettodownloadthecostofwaste.It also conducted a study on the firewall, through the firewall on the way to interceptpackets, found weaknesses in the firewall and to make use of HTTP tunneling technology,the network packet in the process of encapsulation, so that it will have to penetrate thenature of the firewall. HTTP tunnel that can penetrate not only penetrate the personalfirewall, but also penetrate the network firewall. Here the main advantage of the credibilityofcrediblecommunicationportuserstopenetratefirewalls.In short, the design of multi-threading technology, can be very good in the Linuxsystem for file downloads, in parallel to achieve a multi-threaded download, breakpointcontinuinglyfunctionality,andtopenetratethenatureofthefirewall.