Heuristically creating test cases for program verification systems

Abstract

The correctness of program verification systems is of great importance, as they are used to formally prove that safety- and security-critical programs follow their specification. This correctness needs to be checked and demonstrated to users and certification agencies. One of the contributing factors to the correctness of the whole verification system is the correctness of the background axiomatization, which captures the semantics of the target program language. We present an optimisation framework for the maximization of the proportion of the axiomatization that is used (“covered”) during testing of the verification tool. We show how test cases for regression tests can be created based on existing ones, as the initial creation of test cases is a very time consuming process. Our study shows that the combination of different heuristics leads to a diverse set of test cases, which in turn increases the axiomatization coverage. This translates into a significant increase in trust in the program verification system.