| Choosing a suitable model is the core of all data statistics.In practical problems:people often use different model selection criteria to determine a model that can best meet the research needs for analysis according to the nature and purpose of the research problem,such as the BIC criterion with consistency,Cq criterion proposed from the perspective of prediction,etc.But in practical problems,people often get limited data information.Because of the lack of data information,Sometimes the model determined by the model selection criteria is not significantly superior to other models.Based on this problem,one of the research ideas of statisticians at present is to construct model confidence set,that is,to construct a set of candidate models with a given level to contain the best model.The Mac model confidence set is a method to construct the confidence set of the model proposed by Liu,Li,and Jiang in 2021.It was based on the idea of making a cut on the model sequence,so it was named Mac.This paper mainly studies the improvement of the method to construct the Mac model confidence set and its application in the field of model selection.The construction method of the Mac confidence set given by Jiang et al.is:select the candidate model corresponding to the minimum BIC value as the real model,then sort all the candidate models according to the BIC value from small to large,and simulate the probability that the model at each position is equal to the real model through bootstrap.Finally,these probabilities are summed up according to the sorted model order until the confidence level is reached.Unlike the original text.which only selects the model corresponding to the minimum BIC value as the real model,we fully consider the possibility that each candidate model is real.A theoretical formula for calculating the probability that the candidate models are real models in each position after their BIC values are sorted from small to large is given through the total probability formula.Because the calculation amount of bootstrap simulation under all candidate models is very large,we find that the probability that the first few models with the smallest BIC values are real models is relatively large through numerical simulation.Therefore,to approach the theoretical values as much as possible without excessively increasing the calculation amount,We choose the first n models with the smallest BIC value as real models to simulate and give appropriate weights to the simulation probability obtained under each candidate model to carry out weighted average to get the final simulation probability.In addition,when selecting candidate models to form a confidence setis different from selecting models from the candidate model space in the order of BIC values from small to large in the original text.We select the models from the candidate model space according to the order that the simulation probability of the candidate model is equal to that of the real model until the probability that.the real model falls into the model set reaches the confidence level.Through numerical simulation,we compare and analyze the average coverage probability,the average number of models,and the stab ility of the confidence set between the improved method and the method of Jiang et al.The results show that the improved method is superior to the original method in the average coverage probability and stability when the samples are small and medium.We try to extend the method in this paper to the field of model selection.The specific way is to use the improved method to calculate the simulation probability of each ranked model equal to the real model,and then select,the model with the largest simulation probability as the final model.To measure the effectiveness of our method as a model selection method,we compare it with the classic model selection method Lasso in the number of real model selections,The average zero variable deletion number and the average non-zero variable deletion number are simulated and compared.The results show that the real model selection times and the average zero variable deletion number of the improved method increase with the increase in sample size and are significantly higher than those of Lasso.At the same time,considering that the calculation amount of this method will become very large with the increase of variables,to reduce the calculation amount and ensure a better model selection effect when there are many variables,We give a solution.Specifically,we make use of the advantage that Lars algorithm can accurately identify important variables,When there are many variables,we consider using Lars to preliminarily screen the variables and then take the screened variables as a new set of candidate variables and use this method to select models based on the new set of variables.Finally,when there are many variables,we compare this method with Lasso,which is similar to the previous results.The mproved method is still significantly better than Lasso in the number of real model selections and the average number of zero variables deleted.To sum up,this method is also a very good method for model selection. |
