Distributed BFGS Algorithm Design And Statistical Properties For Parameter Estimation

In the environment of big data,traditional statistical estimation methods may no longer be feasible due to the storage,computing power and security privacy of computers.Therefore,the advantage of multiple machines can be utilized to conduct distributed statistical inference and calculation on data sets.Under a distributed setting,in order to reduce the computational complexity,this paper solves the corresponding distributed algorithm design and statistical inference problems based on quasi Newton’s method—BFGS.Specifically,it includes the following two parts:First,for a distributed environment with normal communication,a fast general distributed BFGS algorithm is established with lower communication cost.The key of this algorithm is to perform a distributed approximate calculation of the step size.It is theoretically proved that when the number of iterations meets some conditions,the BFGS estimator is consistent and asymptotically normal.And we propose a estimation formula of variance.The experiment verifies the correctness of the relevant basic theory.At the same time it is showed that the estimation accuracy of the distributed BFGS method is very close to that of the centralized method,which further illustrates the effectiveness of the method.Second,for a distributed environment with Byzantine failures of machines,under lower communication cost,a robust median-based distributed BFGS algorithm is established.The main idea is aggregating local gradients of all local machines by taking the median and getting the global gradient.Theoretically,we prove the consistency and statistical error rate of the obtained estimator under certain conditions.The results of numerical experiments also show the robustness and effectiveness of the proposed method.