Test suite completeness and black box testing

Model-based testing has been widely studied and successfully applied to generate and verify completeness of test suites. Roughly, test suite completeness guarantees that, in a deterministic setting, any nonequivalent implementation under test will always be identified. Several approaches discuss sufficient, and sometimes also necessary, conditions on specification models and test suites to be able to guarantee test suite completeness. In these studies, usually, test cases are required to be nonblockingthat is, they are required to run to completionon both the specification and the implementation models. However, often, it is desirable to have blocking test cases, and in some situations the presence of blocking test cases cannot be circumvented. In the present work test cases are allowed to block, both in the specification and in the implementation models, and a natural variant of completeness is then studied, here called perfectness. Perfectness guarantees that noncompliance between a specification and an implementation will always be detected, even in the presence of blocking test cases. This work characterizes perfectness in isomorphisms and establishes a relationship between the classical notions of completeness and perfectness. Also, a sharp upper bound is given on the number of states in implementations, beyond which no test suite can be completed.