Doubly Robust Inverse Probability Weighting Methods For Missing Primary Endpoint Data In Clinical Trials

Background and purposeRandomized controlled trials(RCTs) are at present considered as the gold standard of study design in evaluating whether an investigational product has therapeutic effects. However, missing data due to various reasons in RCTs are very common while have their own characteristics. Missing data could counfound study results. This is bound to affect intention to treat analyses of their data and further threaten the validity of research findings. Research of missing data treatment method has been for a long time. So far the commonly used are complete case analysis, imputation methods and analysis methods based on likelihood, inverse probability weighting and so on. Application of these methods are subject to missing data mechanism, with a losser assumption when data are missing at random. In this case, current guidelines do not recommend using complete case analysis directly unless missingness does not affect the study outcome; and multiple imputation and maximum likelihood estimation method have their respective characteristics and limitations. For example, in multiple imputation(MI), the analysis model cannot contain variables, non-linearities or interactions that are not in the imputation model; therefore the requirement for compatibility between the imputation and analysis models limits the flexible modeling of analysis models. Actaully the two models prone to have conflicts in variable slections. In addition, MI is unable to produce a unique estimate given random sampling involved and many decision points considered. Use of maximum likelihood estimation method requires parametric assumptions(e.g., normality), reasonable likelihood fuctions and not too large proportion of missingness. Its estimates rely on priori information if using Bayesian posterior inference. Inverse probability weighting gave rise to more weights for complete case and hence showed intitutive analyses. However, simple inverse probability weighting(SIPW) might not utilize available information from variable- partially missing cases; any weights from Logistic regression modeling are subject to its model and ocassionaly are too large.Given these, this study expects to improve SIPW to deal with missing data at random. On the one hand, the improved outcome analysis model can simultaneously incorporate information from complete cases and observed portion of incomplete cases; on the other hand, we also expect to use new method to estimate non-missing probability estimation which can avoid too large weights for observed complete case.Moreover, we expect to conduct some sensitivity analyses for which additional statistical assumptions are not required and the results are easy to explain from a clinical perspective. MethodsFirstly, we set up the robust inverse probability weighting(DRIPW) in theory. A term of expectation 0, which might take advantage of incomplete case, was added to the original algorithm in SIPW. In addition, non-parametric random forest method was used in the estimation of propensity scores for non-missingness probability modeling and was compared further.Secondly, we used simulated data to establish DRIPW methods and implemented them in SAS and R environments. Comparisons between DRIPW and other commonly used methods were conducted for these simulated data. In this study, we assumed that primary endpoint was monotonic and missing at random, that’s to say, using Logit modeling from each subject’s baseline covariates and auxiliary variables during study to simulate data missing status of primary endpoint. In addition to normality of the primary endpoint, the study also added another three scenarios, namely random center effects included in outcome analysis model, non-normal primary endpoint, and wrong non-missingness probability model paired with correct outcome analysis model. In these four scenarios, we considered four sample size(N = 120; 240; 600; 1,000) and each sample had varying total proportion of missingness(10%; 20%; 30%). During statistical analyses, we first compared the propensity scores from Logit modeling and random forest method; then we compared the analysis results from the simple inverse probability weighting, doubly robust inverse probability weighting and multiple imputations and so on. Mean absolute error, 95% confidence interval coverage probability and mean square error was used as performance measures.Finally, we further used a diabetes study with non-inferiority design as an example, in which the propensity scores from Logit modeling and random forest method as well as the analysis results of different missingness treatment methods were compared. In order to claim robust study findings, the tipping-point method was utilized in sensitivity analysis of primary analysis results with standard deviation of missing data equal to zero, standard deviation of missing data equal to that from observed cases in the same group, or the standard deviation of the whole group equal to that from observed cases in the same group. ResultsWe used preliminary clinical findings from one study regarding type 2 diabetes to establish non-missingness probability modeling function and corresponding outcome analysis model. In various simulation scenarios, we achieved the expected total proportions of non-missingness by adjusting modeling function coefficients.Propensity scores in simulated data Regardless of the test group or the control group, different preset proportions of missingness or different study sample size, standard deviations of propensity scores from random forest was smaller than Logit modeling, and also with less extremely small propensity scores, slightly larger mean or median. In addition, among different study sample sizes, the propensity score estimates within each algorithm group are very close. With the same proportion of missingness, the propensity scores from random forest algorithm either in the test group or in the control group almost gradually increased or approached to one as sample size increases whereas the observed trend of propensity scores from Logit regression model algorithm in the four scenarios is not entirely consistent, with increase sometimes and decrease sometimes.Mean absolute error and mean square error of between-group difference in simulated data Simulated data without missingness still have some errors due to chance, but are minimal. Regardless of the missing treatment methods, the larger sample size, the smaller mean absolute error; the greater proportion of missingness the bigger mean absolute error. In four scenarios, doubly robust inverse probability weighting method was always superior to the simple inverse probability weighting. Except in the non-normal outcome scenario, the random forest propensity score weighting method often performed best while Logit modeling propensity score weighting method usually performed poor. When the mean square error was used for evaluation, similar findings as those in mean absolute error were equally observed.Coverage probability of 95% confidence interval of between-group difference in simulated data No apparent patterns were observed as those from mean absolute error when coverage probability of 95% confidence interval was used. It did not appear that doubly robust inverse probability weighting was always superior to simple inverse probability weighting; random forest was not always better than Logit modeling either. Multiple imputation method often showed good coverage.Real case study Regardless of the test group and the control group or the groups together, the average propensity score(mean and median) from random forest algorithm were always bigger, but its standard deviation was not always smaller. The MI method showed largest least squares mean of between-group difference and 95% confidence interval, 0.069(-0.148, 0.286); Logit-based simple inverse probability weighting showed smallest least-squares mean and 95% confidence interval of 0.014(-0.207, 0.235); the results observed in other treatment methods were very close. Overall, no matter what kind of treatment method was used, non-inferiority conclusions always hold true in the study. Tipping-point analysis showed that from a clinical point of view, the non-inferiority conclusion was established under three standard deviation types and hence was credible. ConclusionsIn the simulated RCTs data, when the missingness of primary endpoint was monotonic and at random, doubly robust inverse probability weighting, esp. random forest-based doubly robust inverse probability weighting performed well, usually better than simple inverse probability weighting method, and even better than the popular multiple imputation method in most cases; researchers could consider to apply these missing data techniques. In the real case study, the random forest combined with inverse probability weighting and Logit-based doubly robust inverse probability weighting method demonstrated robust analysis results. Tipping-point analysis as a sensitivity analysis method did not require additional statistical assumptions and was intuitive from clinical interpretation.In a word, this study established one method which combines random forest and DRIPW to deal with data missing at random, and it minimized too large weights and utilized incomplete cases as well. Hence, this method is worthy for use in RCTs when primary endpoint is missing at random with monotonic pattern.