Application Of Overlap Weighting In Signal Detection Of Adverse Drug Reaction

Background:The issue of drug safety is relevant to everyone,and adverse drug reactions have been a priority research effort in the field of public health.The occurrence of adverse drug reactions is not only a significant cause of patient morbidity and mortality,but also a source of financial burden to the healthcare system.Since pre-market clinical trials of drugs are not sufficiently representative of drug safety in the overall population,post-marketing monitoring of adverse reactions to drugs is critical.Adverse drug reaction signal detection is the most important element in the monitoring of adverse drug reactions.At present,the main method of adverse drug reaction signal detection at home and abroad is the use of spontaneous reporting system data for disproportionality analysis of the studied drugs,but when using disproportionality analysis mainly focuses on the association of drugs with adverse events and has not yet considered confounding factors.The presence of confounding factors in observational study will exaggerate or narrow the true association between the drug and the adverse event.The related literature shows that among the methods to equalize confounders in adverse drug reaction signal detection,stratified analysis and logistic regression have problems of less controlled confounders,excessive stratification and covariance,while propensity score can avoid or mitigate the possible problems in these two methods.The currently used propensity score method has certain drawbacks,such as propensity score matching will lose sample size and reduce estimation accuracy,inverse probability weighting will have an excessive effect on the effect estimation in the presence of extreme weights,and propensity score stratification is not as effective as propensity score matching and weighted balanced confounders.The overlap weighting not only utilizes all the observed information but also avoids the problem of extreme weights,so the ability of overlap weighting to equalize confounding factors and the performance of applying it to the field of adverse drug reaction signal detection deserve further investigation and research.Objective:The accuracy and stability performance of propensity score stratification,propensity score matching,inverse probability weighting and overlap weighting in estimating effects under different scenarios were systematically explored through simulation studies,and the applicability conditions of the different methods were explored in order to provide some reference and suggestions for the treatment of confounding factors in observational studies.The overlap weighting was applied to the spontaneous reporting system data in the case study to further evaluate the association between the detected positive signals and the drugs.Obtaining true and credible cause-effect relationships by controlling confounding bias provides more accurate information for drug risk management.Method:This topic focuses on two parts:simulation study and case study.In the simulation study,covariates X₁-X₅（X₁,X₂ are preset confounding factors）,and treatment DRUG（DRUG=1:exposed）,DRUG=0:unexposed),and outcome ADR（ADR=1means outcomes were occurred,ADR=0:means outcomes were not occurred）were set using a monte carlo combined with the characteristics of variables from the U.S.Food and Drug Administration Adverse Event Reporting System database.A total of 72 different scenarios were generated by setting 2 treatment prevalences（10%,40%）,3 sample sizes（500,1000,5000）,3 treatment effect strengths（OR=1,OR=2.7,OR=7.4）,and 4 overlap levels（γ=0.1,γ=0.5,γ=1.5,γ=3.0）,and each scenario was simulated 500 data set in each case.The predetermined confounders X₁ and X₂ were used as covariates in the estimation of propensity score and were calculated using logistic model.The propensity score corresponding to each observation were obtained and then combined with stratification,matching,inverse probability weighting,and overlap weighting to estimate the effect of the association between treatment factors and outcomes,and the results of the effect estimates were evaluated comprehensively using point estimates,standard errors,bias,mean square error,and 95%confidence interval coverage rate to explore the accuracy and stability of the four methods in each scenario.Data from the FAERS database from the fourth quarter of 2017 to the second quarter of 2021 were used in the example analysis to detect potential adverse reactions of emicizumab.The data need to be normalized before conducting the analysis,where observations with missing important baseline data are handled using the complete case analysis.Disproportionality analysis was used for adverse reaction signal detection of emicizumab.Confounding factors that may be associated with emicizumab and adverse effects were screened from all variables in the database based on expertise and included in the propensity score model to calculate the propensity score.Using the adverse reactions appearing in the drug specification as the gold standard,the overlap weighting was applied to the detected positive signals to assess effect estimation after equalizing the confounding factors,while sensitivity analysis of effect estimates using E-value makes the results more reliable.The above study procedures were implemented using SAS 9.4.Results:Simulation study results:（1）The point estimates,bias,and MSE of the overlap weighting performed better than the propensity score stratification,propensity score matching,and inverse probability weighting in most of the simulated scenarios,with higher accuracy of effect estimates.（2）The performance of propensity score stratification,propensity score matching,and inverse probability weighting for each evaluation index became progressively worse as increasingly sparse overlap between groups.（3）Both the bias and MSE of the inverse probability weighting were at low levels when the overlap between groups was large（γ=0.1）,but tended to be largest when the overlap between groups was small（γ=3.0）.It indicates that the accuracy of effect estimation by the inverse probability weighting is strongly influenced by overlap.（4）There was a decrease in point estimates,standard errors,bias and MSE with increasing sample size for each method,indicating elevated accuracy.Propensity score matching showed the greatest variation across sample sizes.（5）The accuracy and stability of the propensity score stratification increased slightly when the treatment prevalences increased from 10%to 40%.（6）Point estimates,bias,MSE,and 95%confidence interval coverage of propensity score matching deteriorated with increasing treatment effect strength when the treatment prevalences was 40%.The decrease in the accuracy and stability of effect estimates was most pronounced at a sample size of 500.Case study results:The point estimates for the two combinations"emicizumab-arthralgia"and"emicizumab-injection site reaction"were greater than 1 and statistically significant for further evaluation using propensity score stratification,propensity score matching and overlap weighting.This was indicated as a higher incidence of arthralgia or injection site reaction events in patients using emicizumab than in those not using emicizumab,consistent with the drug instruction.The estimated effect of"emicizumab-arthralgia"using inverse probability weighting was 0.768（95%CI:0.755,0.782）.This shows that the use of emicizumab is a protective factor for the occurrence of arthralgia,which is not factually correct.The results of sensitivity analysis using E-value showed that the overlap weighting（5.589,9.769）was more robust than the association between emicizumab and arthralgia or injection site reaction observed by propensity score stratification（4.670,8.862）and propensity score matching（4.727,7.193）.The results of the case study demonstrate that overlap weighting is more robust in practical data application and can be applied to equalize confounding factors in adverse drug reaction signal detection.Conclusion:This study showed that propensity score stratification was less accurate than matching and weighting methods in the results of different scenarios.The accuracy of propensity score matching was related to the sample size and the degree of overlap between groups,and the accuracy of the inverse probability weighting is then closely related to the degree of overlap between groups.Overlap weighting has higher accuracy and stability in most scenarios and is a better choice for balancing confounders.The results of effect estimation of the positive signal for emicizumab and sensitivity analysis using E-value also showed better robustness using overlap weighting in the case study.