Distributed Quantile Regression Algorithms And Applications

Quantile regression,as a statistical tool complementary to mean regression,is able to reveal the relationships between the conditional quantiles of the response variable and its covariates in a regression model.Since the conditional quantiles contain probabilistic information,quantile regression can provide a complete depiction of the conditional distribution of the response variable by estimating a series of its conditional quantiles.Besides,quantile regression does not require any distributional assumption on the random term in a regression model,and it is robust to outliers.Due to these advantages,quantile regression has been extensively and deeply studied in both the academic and the industrial fields.In most of the existing research works,quantile regression was studied in a centralized scenario,and the corresponding centralized quantile regression algorithms were developed.However,with the rapid development of sensor technology,embedded systems and mobile communication technology,and the advent of the era of big data,a variety of practical applications are based on networked scenarios,such as wireless sensor networks(WSN),the Internet of Things(Io T)and distributed computing systems.In the circumstances,distributed information processing has become a popular research topic.It mainly studies fully decentralized methods for information mining in the situation where data are distributed on different nodes in the network,and are hard to gather.Due to the highly-centralized nature,the existing centralized quantile regression algorithms are not applicable to solving the quantile regression problem over networks.Thus,it is of great significance to study and develop fully distributed quantile regression algorithms.In this thesis,we mainly concentrate on the linear and the nonlinear quantile regression problems when taking the restrictions in networked scenarios into consideration,and we systematically develop several distributed quantile regression algorithms which are of global sense.Similar work is rare in the existing literature.The main contents and originalities of this thesis are summarized as follows:First,for the linear quantile regression problem over wireless sensor networks,we develop two distributed algorithms utilizing the subgradient method and the diffusion strategy,namely,the non-sparse and the sparse distributed linear quantile regression algorithms.The latter is specially developed for utilizing the sparsity of the regression model to achieve better performance.We provide the convergence analyses of the proposed algorithms.The numerical simulation results show that the proposed distributed algorithms can achieve very similar performances to their centralized counterparts,and the sparse algorithm outperforms the non-sparse one on sparse models.Second,for the online linear quantile regression problem over networks with quantized communication,by employing an encoding-decoding system,the online subgradient method and the consensus strategy,we develop the non-sparse and the sparse quantized-communication based distributed online linear quantile regression algorithms.Besides,we theoretically analyze the convergences of the proposed algorithms,prove that the proposed algorithms can make the local estimates at all nodes reach a consensus at the true parameter vector without quantization error,and provide the required conditions.The simulation results indicate that the proposed algorithms can finally eliminate the quantization error,and achieve very similar performances to their corresponding nonquantized centralized and distributed counterparts.The sparse algorithm is shown to outperform the non-sparse one on sparse models.Third,we study the nonlinear quantile regression problem over networks.The existing nonlinear quantile regression algorithms,such as the kernel quantile regression(KQR)algorithm and the quantile regression neural network(QRNN)algorithm,are all centralized,and it is hard to directly extend these algorithms to the distributed framework.Besides,these centralized algorithms suffer from high computational complexities and low efficiencies,especially for large-scale problems.To address these issues,we develop the distributed quantile random-features kernel machine algorithm by utilizing the random-feature kernel approximation method and the ADMM method,and provide the convergence analyses of the proposed algorithm.The simulations on both a synthetic data set and a real wind power data set indicate that compared with the KQR algorithm and the QRNN algorithm,the proposed algorithm can not only achieves comparable or even better accuracy,but also possesses much higher computational efficiency.