First experience of conformance testing an application interface specification implementation

This paper describes our first attempt of conformance testing an implementation of the Service Availability Forum Application Interface Specification on a carrier-grade service platform for mobile communications applications. The requirements and guidelines of the IEEE Standards for measuring conformance to POSIX® have been adapted for the Application Interface Specification. The Test Method Specification structure is explained and assertions of the component registration function is shown as an example. A description of the Implementation Under Test is included, together with an explanation of the Test Method Implementation. Our experience suggests that this approach is indeed feasible and repeatable. The thorough level of testing appears to have the right balance of confidence and manageability for test cases. The intermediate test results also unexpectedly provide the developers with some useful insight into future implementations.