Preparing tests

The testable unit for the test framework is a test case. Each test case is represented by a directory with an appropriate test case name. Test cases may be grouped in test suites by locating test case directories as sub directories below a test suite directory. Test suites may be grouped, again, in upper test suites etc.

Each test suite directory must contain a suite file, which contains a short description of the for the test suite. Test case directories need a suite file, which allows storing explanations for the test and a run file, which may also contain special hints for executing the test case. Content of suite and run files is needed for documentation purpose, only, i.e. files may also be empty.

In order to run a test case, one has to prepare at least the run file in the actions sub directory for calling actions to be executed. This and other actions that may be overloaded (e.g. preprocessing) are stored in the actions sub directory. When not defining preprocessing.cmd and postprocessing, those are taken from the next higher test suite directory that provided appropriate command files in its actions sub directory. The test framework always collections actions along the test suite hierarchy, where command line files provided on lower level will overwrite command line files with the same name provided on higher levels.

Test specific data has to be provided in the data sub directory for each test case. Similar to command line files in actions directories, data common for several test cases may be stored in the data sub directory of test suites on different hierarchy levels. When running test cases, test data will be collected along the test suite parent hierarchy.

Expected test results are provided in the expected sub directory. The expected sub directory must contain at least one file with expected data. Also, expected data may be provided in test suite expected sub directories.