This article explains how to control test runs and execute tests in the Test suite view.
The explanations in this chapter are based on a sample solution. You can download it below.
Theme: Test suite run
Time: Less than 10min
Installation:
- Unzip to any folder on your computer.
-
Start Ranorex Studio and open the solution file
RxDatabase.rxsln
Manage run configurations
You can include and exclude test cases and smart folders from a test run. To do so, simply check or uncheck them in the test suite.
The current state of checked/unchecked test cases and smart folders is called a run configuration. You can save run configurations for reuse and switch between them using the drop-down menu in the Test suite view.
Create a new run configuration
- Click the run configurations drop-down menu
- Click Manage run configurations…
- Click Add.
- Enter a name for the run configuration.
- Click OK.
Result
You can now select the run configuration from the drop-down menu.
- TestRun configuration, including all test suite elements
- TestWithoutValidation configuration, excluding the validation smart folder
Disable/enable test suite items
In addition to including/excluding test cases and smart folders, you can also enable and disable:
- Recording modules
- Code modules
- Module groups
- Setup and teardown regions
Note: Disabled items are not executed during a test run.
To disable an item:
- Right-click the item you want to disable.
- Click Disable.
Result
Ranorex Studio grays out the item.
Configure run iterations
By default, test cases and smart folders are run once during test execution. However, you may want to run them several times. You can do so with run iterations.
To set the iteration count:
- Select the test case or smart folder.
- Press
F4to open the Properties pad. - In Properties, set the Iteration count to the desired value.
Result
The number of iterations appears next to the test suite item. In the sample solution, the test case is iterated 5 times.
Configure auto-retry
In UI testing, errors can occur because the application under test (AUT) is unresponsive. Auto-retry lets you rerun parts of the test automatically.
When you set a retry count, the selected test case or smart folder is rerun until it succeeds or all retries are used up.
To set the retry count:
- Select a test case or smart folder.
- Press
F4to open the Properties pad. - In Properties, set the Retry count to the desired value.
Configure report levels
You can set the report level of test cases and smart folders in the test suite. Report levels help you control what information appears in the report and where it appears, which is especially useful for larger test suites.
Configure error behavior
Errors are part of testing, so it’s important to define what Ranorex should do when an error occurs. You can configure the error behavior of test cases and smart folders from their context menu.
There are four error behaviors:
Continue with iteration
Continue with sibling
Continue with parent
Stop
Note: The default error behavior is Continue with sibling.
Continue with iteration
If an error occurs, the test run continues with the next iteration of the affected smart folder (Validation) or test case (if iterations are configured).
Continue with sibling
If an error occurs, the test run continues with the next sibling test case or smart folder. In this example, this is the smart folder DatabaseCleanUp.
Continue with parent
If an error occurs, the test run continues with the next parent test case or smart folder. In this example case, this is the test case CompleteDatabaseTest.
Stop
When the Stop error behavior is selected, an error immediately stops the entire test run.
Error behavior of setup/teardown regions
Setup and teardown regions follow a special, fixed error behavior.
- An error in the setup region immediately stops the test.
- If a module fails in the teardown region, Ranorex continues with the next module in teardown.
Run a test from the test suite view
- Click RUN
- Watch Ranorex Studio execute the test
- Observe the progress information during the test
Result
Once the test run finishes, the report is displayed.