Dynamic software reliability modeling with discrete-test metrics: How good is it?

The black-box approach based on stochastic software reliability models is a simple methodology with only software fault data in order to describe the temporal behavior of fault-detection processes, but fails to incorporate some significant metrics data observed in the testing process. In this paper we develop a proportional intensity-based software reliability model with time-dependent metrics, and propose a statistical framework to assess the software reliability with the time-dependent covariate as well as the software fault data. The resulting model is similar to the usual discrete proportional hazard model, but possesses somewhat different covariate structure from it. We compare three metrics-based software reliability models with some typical non-homogeneous Poisson process models, which are the special cases of our model, and evaluate quantitatively the goodness-of-fit from the viewpoint of information criteria. As an important result, the accuracy on reliability assessment strongly depends on the kind of software metrics data used for analysis and can be improved by incorporating the time-dependent metrics data in modeling. Significance: The software metrics observed in the software testing may depend on the product reliability. The effect of software metrics, especially, of dynamic software metrics, on the software reliability is carefully studied throughout empirical works with real software fault data.