Research On Buffer Overflow Detection And Protection Improvement

With the rapid development of computer technology,network security has been an unprecedented challenge.Buffer overflow attack is one of the most serious threats to computer security.Because of the large number of such attacks and serious consequence,buffer overflow attack detection and protection are of great importance.In recent years,the research of buffer overflow is mainly focused on two aspects: how to quickly confirm the existence of such vulnerabilities and the location if existed in target program,and how to protect the procedures having buffer overflow vulnerabilities.The former focused on Fuzzing technology,currently with the problems like strong randomness、high rate of false and long detection time.StackGuard,one of the popular dynamic protection technologies in the latter aspect,was found with the problem that canary value may be guessed successfully and then overwritten.In view of the above problems,this paper has done some research from detection and protection:(1)In view of the current situation of buffer overflow vulnerability mining,a new improved method isproposed in this paper.In order to solve the problems of strong randomness and blindness in Fuzzing test,this paper proposes a detection method for buffer overflow attack using Fuzzing technology with genetic algorithms,basing on the characteristic of a buffer overflow exploit,combing with the flow control theory about static analysis,designing a corresponding fitness function of GAs,intelligently guiding program data to reach the vulnerability of dangerous code snippet.Compared with SA using simulation,experimental results demonstrate that new method has higher convergence speed and completeness.(2)According to the research status of buffer overflow protection,this paper focuses on the improvement of StackGuard model.StackGuard model use XOR operation algorithm to increase the complexity of the canary generation.Once canary value which is still stored in a local function stack is calculated,Stack Guard protection can be overwritten and bypassed.For this problem,an improved model named A-GS is proposed in this paper.To avoid speculation and rewritten by overwriting attack,canary is set above the local variable space;new canary is set as the original return address.Abandon of XOR generation algorithm shortens the detection time;in terms of multi-level calls,the canary of parent function was placed at the top of the sub-level function’s stack space to avoid the possibility that canary can be overwritten and bypassed again.The experimental results show that the code execution efficiency of the A-GS model is similar to that of the StackGuard model.But In A-GS model,it is prevented such overwriting and bypass attacks by calculating the canary in StackGuard.