Research On A Class Of Distributed Second-Order Optimization Methods In Low And High Dimensions

In the information age,the rapid development of data generation and data collection capabilities has led to a serious shortage of computing power and storage resources.Researchers have designed a variety of modern parallel or distributed computing architectures.Existing research shows that in distributed cases,in order to improve the efficiency of the algorithm,reducing the communication between the central machine and the local machine or between two nodes is a very effective method.This dissertation mainly studies the distributed second-order local Newton method from different perspectives in two directions under the general linear model:for reducing the communication cost,a Trimmed Local Newton(TLN)method is proposed;for the problem of the curse of dimensionality,a Local Newton Iterative Hard Thresholding(LNIHT)method is proposed?In the case of low dimensions,the Trimmed LocalNewton method synchronizes the model every selected L times.Before synchronization,the 1-norm of the local Newton direction of each local machine pkt=Hkt(?kt)-1gkt(?kt)is arranged from small to large,the largest ?×100%local Newton direction is removed,and the first(1-?)×100%samples are selected to synchronize and update the model.It saves communication costs in two ways,reducing the number of iterations and communication rounds overall.In the case of high dimensions,the Local Newton Iterative Hard Thresholding combines the distributed second-order method with the iterative hard threshold method in the gradient compression method,and only needs to select the sparse order p as a hyperparameter.Experiments show that LNIHT can quickly solve the curse of dimensionality problem in a distributed case.