| The use of software reliability models as an aid in making software release decisions is a well-established practice in software reliability engineering. If the chosen model over-estimates the mean time to the next failure MTTF or, inversely, under-estimates the current defect density, then the software could be released prematurely. A factor that could bring about such an overestimate is a poorly constructed test case suite. If during testing, one or more suites of test cases take much longer than expected to discover the next defect, the estimated defect density and MTTF can be strongly biased toward the unwarranted early release of the software. This research addresses this problem by considering as outliers the time between failures resulting from ineffective test suites. Using an approach based on order statistics, a bound is constructed such that the probability that the kth largest values (relative to their positions in the ordered series) in the dataset will exceed that bound is fixed at some level of significance, say 0.05. This research discusses the development of the order-statistics approach and validates the method by the use of simulations of failure time data which have been randomly contaminated with uncharacteristically large failure times. Additionally, it demonstrates the use of the approach as applied to a number of real datasets supplied by Data and Analysis Center for Software DACS. |
