Managing tests within a test run

Managing tests within a test run includes the following basic features:

• Selecting tests to be executed
• Running selected tests
• Providing test run summary reports

For a given test suite hierarchy two simple ways of selecting tests can be used. By selecting a single test suite for test run, tests for the test suite and all subsequent test suites are executed. The other way is manually selecting or deselecting tests in the test suite hierarchy. In the example, tests are indicated by a file with the name test. Removing or renaming this file will exclude a test suite from being tested. Nevertheless, there are many other ways of filtering tests for run as filtering by name.

Beside managing test runs, the test framework has to manage test run resources. This includes providing test run locations for storing results from tests within the test run. Running tests within a test run should also provide a summary of test results (true/false). Formatting the output property (e.g. as csv-file), usually allows importing test results into a connected test management system (ODABA Teat Browser or Project Manager, Polarion or other).

While running tests, also test events (e.g. start/stop time) may be written to a test run protocol. As other test run resources, this should be stored in general in separate test run location for the test suite. In order to be able to store test run summaries, each test run should provide its own test run location (e.g. directory), which contains test results and test output for executed tests.