Key Technology Research On Personalized Federated Learning For Urban Sensing

Urban sensing is one of the most important areas of the Internet of Things(IoT),which enables environment sensing of the urban areas and facilities through a range of sensing techniques and provides decision suggestions for users by adopting intelligent models.Urban sensing tasks usually involve a large amount of sensing data collected by decentralized devices,and the traditional centralized learning approach based on data centers can hardly meet their requirements for data privacy,communication efficiency and computational performance.Federated learning,an emerging distributed machine learning technique,has gained significant attention in urban sensing applications because it can train models without aggregating data from decentralized sensing devices.However,applying general federated learning frameworks directly to urban sensing applications faces unique challenges,i.e.,the lack of individuality in the global model,coarse-grained fusion of local models and the difficult training procedure of heterogeneous data.This thesis introduces personalized federated learning methods to the urban sensing applications,proposes a series of method to establish a framework named FedSensing with a three-tier architecture consisting of a central server,regional servers,and local devices,and conducts extensive experiments to evaluate the performance of the framework on urban sensing applications.The main research contributions of this thesis are summarized as follows.(1)A federated regionalized learning method based on feature correlation of sensory data is proposed.In urban sensing applications,the sensing data collected by local devices in different geographical locations have varying feature distributions.As a result,a single global model may not adequately meet the sensing requirements of urban sensing applications in complex urban scenarios.To address this issue,firstly,the region partitioning algorithm is proposed that groups local devices with similar features into the same region and sets up a region server.Secondly,the regionalized modeling of urban sensing applications is achieved through collaborative training between the regional server and the central server.Thirdly,to addresses the limitations of model updating strategies in existing federated learning frameworks,the model flexibility updating algorithm based on model differences is proposed to achieve flexible regional model updating by setting specific weighting ratios according to the weight divergence of global and regional models.The experimental results demonstrate that the proposed federated regionalized learning method improves the accuracy by 1.0%and reduces the mean absolute error by 16.8%,respectively,compared with the existing optimal method in two real-world urban sensing datasets.(2)A global model fusion method based on neural network decoupling is proposed.In the collaborative training process,the fusion and transmission of global models are key to achieving knowledge transfer.However,existing model fusion algorithms may lead to the loss of global model performance due to the alignment invariance of neural networks and the imbalance in data distribution may result in weight drift,which impacts the effectiveness of the regional models that are fused into the global model.To address the aforementioned issues,the regional model decoupling algorithm and the global model fusion algorithm are proposed.These algorithms aim to filter out the neurons that are not activated in the inference process of the regional model during the global model fusion process,thereby reducing the interference of confusing information.Additionally,to alleviate the increased transmission overhead between the central server and the regional server caused by the decoupling of neural networks,a mask encoding scheme is proposed.The experimental results indicate that the proposed method improves the performance of model fusion and achieved an improvement in accuracy on four datasets compared to existing optimal methods by 0.3%,2.5%,2.1%,and 1.3%.(3)A meta-learning based local personalization model training method is proposed.As the amount of data collected by local devices increases,it has become feasible to construct personalized models on a device-level.However,the heterogeneity of sensing data collected by various local devices leads to the negative transfer problems,which impair the effectiveness of personalized models.To address these issues,a meta-learning network has been designed and deployed on the regional server.This network automatically analyzes the similarities among local devices to fuse parameters of global,regional,and local models.As a result,it enables the output of local model parameters.Meanwhile,an alternate training strategy is proposed to train both the meta-model and the local model in an end-to-end manner,as the regional server needs to train both models simultaneously.The experimental results show that the proposed method improves the average accuracy of personalized models by 3.52%and 2.93%in the two datasets,respectively.(4)A prototype system for urban environmental monitoring based on personalized federated learning framework is designed and implemented.This thesis establishes a systematic implementation of the FedSensing framework that integrates the proposed methods.FedSensing adopts deep learning models such as convolutional neural networks and recurrent neural networks,to learn knowledge from various environmental sensing data,including weather data,air quality data,outdoor environmental image data,and urban point-of-interest data,to achieve weather prediction and air quality prediction tasks.In conclusion,this thesis delves into the key challenges of federated learning for urban sensing applications,namely regional model training,global model fusion,and personalized model training,and provides theoretical methods to address the challenges.Additionally,a prototype system named FedSensing is established to evaluate its performance on two general tasks of urban sensing applications to illustrate the effectiveness of the proposed framework.