Publication Bias Testing In Meta-analysis

Background: With the development of clinical research and evidence based medicine,well designed systematic reviews and meta-analysis are considered as the best evidence forthe clinical practice. A meta-analysis combines the results of several studies that address aset of related research hypotheses. It has higher statistical power to detect an effect andmore precise estimation of effect size. The traditional meta-analysis pool the results fromrandomized clinical trials, but now more and more meta-analysis aim to synthesize theresults from epidemiological studies, for example results from cohort studies or casecontrol studies. Most outcomes of these epidemiological studies are dichotomous. Inmeta-analysis, publication bias is an inevitable problem and is one of the threats to validityof the results from meta-analysis. Thus, testing and adjusting the publication bias are thefundamental work in meta-analysis. However, this problem is still controversial andunsolved.Whether to publish a study or not largely depends on the sample size (precision) andstatistical significance of the effect size. Authors, editors and sponsors are reluctant topublish small or non-significant studies. Several measures have been carried out to reducethe influence of publication bias on the meta-analysis, for example the RCT registrationand publication policy for some reputable journals. However, as for the epidemiologicalstudies, there is lack of registration system and publication requirement, for example nogeneral registration system exists for genetic-disease association studies. The methodologyfor dealing with the publication bias has a history of more than30years. The statisticiansand clinical epidemiologists have developed various methods, for example the fail-safenumber, selection model/weighted distribution and regression or rank correlation methodsbased on the funnel plots. Unfortunately, none of the above methods is appropriate in allscenarios, especially for the binary outcomes. The Cochrane handbook has made somerecommendations for the using of methods to detect the publication bias based on thefunnel plots. However, these recommendations are based on simulation studies ofmeta-analysis with RCT. Compared to randomized controlled trials, gene-diseaseassociation studies have their own characteristics: retrospective case-control studies,balance or unbalanced sample size in case and control groups, heterogeneity caused byethnic and the possible rare event. These recommendations aren't necessarily right for theHuGE review. Up to now, no comprehensive simulation study based on these characteristics has been conducted. Hence, no reliable tests for publication bias arerecommended for HuGE reviews.Aim: In this paper, we developed new regression methods that using smoothed variance asprecision scale of individual study to test the asymmetry of funnel plot. Also, we conducteda comprehensive simulation study to compare the performance of the existing methods andnew methods. Finally, we made a concise recommendation for the publication biasdetection.Also, some important existing selection models were used to adjust the publication bias.We made some extension for the selection model based on the previous studies. Thesecond derivative of the likelihood function and Wald test were used to detect thepublication bias. In order to provide a convenient using of these complicated methods forclinical and epidemiological investigators, we coded some user-friendly R function.Methods: For the weighted distribution and selection model, we used the secondderivative of the likelihood function and Wald test to detect the publication bias. A classicalexample was used to implement these methods.For the methods based on funnel plots, we conducted a comprehensive simulation study tocompare the performance of the existing methods and new methods. In order to make thesimulation study more realistic, we reviewed all the HuGE reviews that published in theAmerican Journal of Epidemiology and extracted the parameters that could be used in thesimulation. The following information was extracted from the quantitative synthesisreviews: first author, published year, diseases, polymorphisms and genes, number ofindividual studies included in each meta-analysis, sample size in case group and controlgroup, sample size of the included individual studies, OR values, heterogeneity, methodsthat used to test the asymmetry and the results of tests. We used the extracted informationas the simulation parameters in our study. In order to find the appropriate methods undervarious conditions, we compared the type I error rate and power of the new methods andexisting methods under every simulated combination.Results: We compared several important weighted distributions, such as Logistic weightfunction, Dear and Begg weight function, Hedges weight function and Copas selectionmodel, and discussed the advantages and limitation for these methods. We illustrated thepractical application of these methods in meta-analysis of thrombolytic therapy in acutemyocardial infarction. Several R functions were coded to implement these methods. The simulation results indicated that only three regression tests, namely Peters' test,AS-Thompson test and SVT test, had the appropriate type I error rates regardless of thedegree of heterogeneity, the number of included studies, the value of odds ratio, the eventrate and the sample size ratio between two groups. The type I error rates of the othermethods were sensitive to heterogeneity. In line with Rucker's simulation results, the type Ierror rates of the three rank correlation tests decreased as the heterogeneity increased. Thetype I error rates of the other four regression tests increased as the heterogeneity increased.However, when theI2approached to zero, all the regression methods had the appropriatetype I error rates. The number of included studies had little influence on the type I errorrates under a fixed level of heterogeneity. In the figures, the green cell meant the level ofheterogeneity and the orange cell meant the sample size ratio of two arms. For the powerof tests, we compared the three tests which had the appropriate type I error rates whenheterogeneity was presented. Our newly developed test, SVT test, always had the higherpower than the Peters' test and AS-Thompson test in all scenarios. We considered the SVTtest as the preferred method. AS-Thompson test was also an alternative method whenmoderate or severe heterogeneity was presented. However, this test was rather conservativewhen no heterogeneity or mild heterogeneity was presented. Peters' test can be consideredas a complementary method when no or mild heterogeneity was presented. However,several other regression methods also can be used to detect the asymmetry when noheterogeneity was presented. When taking the event rate and the sample size ratio betweentwo groups into consideration, we found that Harbord's test appeared to be preferable toothers when the event rate was rare, the size ratio was larger than one andI2approachedto zero. Besides, the number of included individual studies was another important factor tothe power of tests. The power of Peters' test, AS-Thompson test and SVT test increased asthe number of the included studies increased. For example, the power could reach to0.80when the number of include studies was60in the absence of heterogeneity. We do notrecommend using statistical methods to detect the asymmetry when the meta-analysis hasless than10studies.Conclusions: In this paper, we reviewed the statistical methods to dealing with thepublication bias. Some extension work was carried out in the weighted distribution andselection models. We proposed a new methods that based on the funnel plots to test thepublication bias. A comprehensive simulation study based on the scenarios that extractedfrom HuGE reviews in American Journal of Epidemiology was carried out to compare the performance of the new and existing tests. We provided a concise table to show theappropriate using of the regression methods to test the asymmetry when the outcome wasbinary.