A Universal Method Of Linear Approximation For The Efficient Implementation Of Nonlinear Functions

As a special type of transformation function,nonlinear functions are widely used in neural network algorithms.They are used to add nonlinear factors to neural network models and solve problems that the linear model cannot solve,such as sigmoid,tanh,softmax and so on.In addition,nonlinear functions are also widely used in digital signal processing and communication algorithms,such as logarithm and hyperbolic tangent functions commonly used in communication algorithms.However,due to the nonlinear nature of nonlinear functions,many challenges are encountered in the circuit implementation of nonlinear functions.Some of the existing algorithms cannot control the approximate accuracy in advance,or cannot accurately and efficiently approximate the target nonlinear function at the same time.Some other algorithms can only approximate some special functions and are not universal.Therefore,in order to improve the above defects,a general error controlled piecewise linear approximation method is designed in this paper.This paper presents a general piecewise linear approximation method with controllable error,which is based on the common look-up table and piecewise linear approximation method.In view of the problem that the existing algorithm cannot control approximate precision independently,the algorithm in this paper has adaptability,which uses the maximum absolute value as the approximate precision to control the linear segment,that is,to achieve the prior control approximate precision;in view of the existing algorithm cannot effectively and accurately approximate the target nonlinear function,the parallel segment number comparison mechanism is used to reduce the calculation period,and the symbol bit as the comparison result to reduce the circuit area of the segment number generator.Moreover,the actual calculation period of the hardware architecture of this paper is a clock cycle,the calculation delay is small,and for the existing algorithm only to approximate the specific nonlinear function,depending on the disadvantages of the special nature of the target function,the algorithm does not utilize any nature of the target function,the algorithm has universality,and does not need to make secondary development when approximate other nonlinear functions.The proposed general error controlled piecewise approximation method realizes sigmoid,tanh,logarithm and inverse hyperbolic tangent functions in 65 nm CMOS process library.Compared with the results in the existing works,the approximation accuracy is increased by 4.5,2.1,1.3 and 4.6 times respectively,and the circuit area is reduced by 49.9%,22.45%,28.5% and 50% respectively.In the actual implementation of LSTM network,when using the same neural network model,the recognition rate of the whole model can be increased by 17% by using the method proposed in this paper.