Execute a Test Suite

Manage run configurations

You can include and exclude test cases and smart folders from a test run. To do so, simply check or uncheck them in the test suite.

The current state of checked/unchecked test cases and smart folders is called a run configuration. You can save run configurations for reuse and switch between them using the drop-down menu in the Test suite view.

Create a new run configuration

1. Click the run configurations drop-down menu
2. Click Manage run configurations…
   
3. Click Add.
4. Enter a name for the run configuration.
   
5. Click OK.

Result

You can now select the run configuration from the drop-down menu.

1. TestRun configuration, including all test suite elements
2. TestWithoutValidation configuration, excluding the validation smart folder

To disable an item:

1. Right-click the item you want to disable.
2. Click Disable.

Result

Ranorex Studio grays out the item.

Note: 
- Disabling a module group disables all modules in that module group.
- Disabling a setup/teardown region disables all items inside it.

By default, test cases and smart folders are run once during test execution. However, you may want to run them several times. You can do so with run iterations.

To set the iteration count:

1. Select the test case or smart folder.
2. Press F4 to open the Properties pad.
3. In Properties, set the Iteration count to the desired value.
   

Result

The number of iterations appears next to the test suite item. In the sample solution, the test case is iterated 5 times.

In UI testing, errors can occur because the application under test (AUT) is unresponsive. Auto-retry lets you rerun parts of the test automatically.  

When you set a retry count, the selected test case or smart folder is rerun until it succeeds or all retries are used up. 

To set the retry count:

1. Select a test case or smart folder.
2. PressF4 to open the Properties pad.
3. In Properties, set the Retry count to the desired value. 
   

Note: If there are data bindings or run iterations, retries start at the point of failure. For example, if the failure occurred at iteration 3 of 5, the retry starts at iteration 3. Only test cases or smart folders that fail every single retry are marked as failed in the report.

You can set the report level of test cases and smart folders in the test suite. Report levels help you control what information appears in the report and where it appears, which is especially useful for larger test suites.

Errors are part of testing, so it’s important to define what Ranorex should do when an error occurs. You can configure the error behavior of test cases and smart folders from their context menu.

There are four error behaviors:

• Continue with iteration

• Continue with sibling

• Continue with parent

• Stop

Note: The default error behavior is Continue with sibling.

Continue with iteration

If an error occurs, the test run continues with the next iteration of the affected smart folder (Validation) or test case (if iterations are configured).

Continue with sibling

If an error occurs, the test run continues with the next sibling test case or smart folder. In this example, this is the smart folder DatabaseCleanUp.

Continue with parent

If an error occurs, the test run continues with the next parent test case or smart folder. In this example case, this is the test case CompleteDatabaseTest.

Stop

When the Stop error behavior is selected, an error immediately stops the entire test run.

Error behavior of setup/teardown regions

Setup and teardown regions follow a special, fixed error behavior.

• An error in the setup region immediately stops the test.
• If a module fails in the teardown region, Ranorex continues with the next module in teardown.

1. Click RUN
2. Watch Ranorex Studio execute the test
3. Observe the progress information during the test

Note: After you’ve clicked RUN, do not use the keyboard or mouse. Doing so would interfere with the test actions and cause a test failure.

Result

Once the test run finishes, the report is displayed.