An Application of Metamorphic Testing for Testing Scientific Software

One of the grand challenges for adequately testing scientific software is due to the oracle problem. Metamorphic testing is the promise technique for addressing the problem through checking the satisfaction of a metamorphic relation that defines the correlation on the relation of paired test inputs and the one of their corresponding outputs. Therefore, checking the correctness of an individual test is replaced by checking the satisfaction of the metamorphic relation among a group of tests. However, the quality of the testing is highly depended on the identification of metamorphic relations. In this paper, we use an iterative approach for the development and refinement of metamorphic relations during testing process through testing an open source discrete dipole approximation program called ADDA. Through studying ADDA program, we discuss the experience and challenges of metamorphic testing for testing scientific software.