Running a test

In order to run a test in a test framework, several typical steps are required:

• Preparing test environment - This includes providing global settings for the test run and preparing the test work area by collecting necessary test resources.
• Running the test - This may be simply a function call but also a more complex process.
• Evaluating test output: This typically includes comparing test output with expected output and creating test results for the test.

In the example testRun performs the necessary steps for a single test by calling:

• settings - provide common environment variables for running tests (settings.cmd)
• prepare - prepare test work area for execution (prepare.cmd)
• preprocessing - execute test specific preprocessing actions (preprocessing.cmd)
• run - run the test (run.cmd)
• postprocessing - execute test specific post processing, e.g. comparing output (postprocessing.cmd)
• report - create test run report information (report.cmd)

This demonstrates, how one may manage automated tests with just a few lines in a command shell.

Since the work area will be removed after testing or replaced by data for the next test run, detailed information about last test run will get lost. In order to keep test run output, the work area data may be copied to a test run/test area (e.g. by zipping the complete work area directory). In any case, test output and results should be stored separately, i.e. in a separate test run directory or in appropriate test run directories below each test suite. It might also be sufficient to copying output data differing from expected results to a test run/test directory.