Welcome to the new Ranorex Support Portal

Execute a test suite

In this chapter, you’ll learn about the various options to control test runs and execute tests in the test suite view.

In this chapter

Note Icon

Screencast

The screencast “test suite execution options” walks you through the information found in this chapter.

Watch the screencast now

Download the sample solution


The explanations in this chapter are based on a sample solution. You can download it below.

Download

Sample solution


Theme: Test suite run
Time: Less than 10min

Installation:

  1. Unzip to any folder on your computer.

  2. Start Ranorex Studio and open the solution file RxDatabase.rxsln

Note Icon

Hint

The sample solution is available for Ranorex versions 8.0 or higher. You must agree to the automatic solution upgrade for versions 8.2 and higher.

Manage run configurations


You can include and exclude test cases and smart folders from a test run. To do so, simply check or uncheck them in the test suite.

Checkboxes to include/exclude from test
 run

The current state of checked/unchecked test cases and smart folders is called a run configuration. You can save run configurations for reuse and switch between them using the drop-down menu in the test suite view.

Manage run configurations
  1. Click the run configurations drop-down menu.

  2. Click Manage run configurations…

Adding a new run configuration
  1. Click Add.

  2. Give the new run configuration a meaningful name.

  3. Click OK.

Result(s):

  • The run configuration can be selected from the drop-down menu.
 Different run configurations
  1. TestRun configuration including all test suite elements

  2. TestWithoutValidation configuration excluding the validation smart folder

Disable/enable test suite items


Similar to including/excluding test cases and smart folders, you can also enable and disable recording modules, code modules, module groups, and setup and teardown regions. Disabled items are not executed during a test run.

Disabling a recording module
  1. Right-click the item you want to disable.

  2. Click Disable. The item will be grayed out.

Note Icon

Note

  • Disabling a module group disables all modules in this module group.
  • Disabling a setup/teardown region disables all items in it.

Configure run iterations


By default, test cases and smart folders are run once during test execution. However, you may want to run them several times. You can do so with run iterations.

Configuring run iterations
  1. With a test case or smart folder selected, press F4.

  2. The Properties pad appears to the right of the test suite.

  3. Next to Iteration count, set the desired number of iterations.

  1. Iteration count in the Properties.

Result(s):

  • The number of iterations appears next to the test suite item.
  • In our example, the test case is iterated 5 times.

Configure auto-retry


In UI testing, errors will sometimes occur simply because the application under test was unresponsive. In these cases, one solution is to simply re-run parts of the test. You can do so with auto-retry. Test cases or smart folders with an auto-retry count will be rerun until they are either successful or all retries have been used up.

Configuring auto-retry
  1. Select the desired test case or smart folder.

  2. PressF4. The Properties pad appears to the right of the test suite.

  3. Next to Retry count , set the desired number of retries.

  1. Retry count in the Properties.

Note Icon

Note

If there are data bindings or run iterations, the retries will start at the point of failure. For example, if the failure occurred at iteration 3 of 5, that’s where the retry will start

Only test cases or smart folders that failed every single retry will be marked as failed in the report.

Configure report levels


You can also set the report level of test cases and smart folders in the test suite. Report levels allow you to control what information appears in the report, and where it appears. This is especially useful for complex tests with many test cases and smart folders, to keep the report structured.

Download

Further reading


Report levels are beyond the scope of this chapter. They are explained in Ranorex Studio fundamentals > Reporting > ⇢ Report levels.

Configure error behavior


Errors are part of testing.  This is why it’s important to tell Ranorex what to do when an error occurs. You can do so by configuring the error behavior of test cases and smart folders in the test suite.

The error behavior is set up in the context menu of a test case or a smart folder. There are four different types, explained below.

Specification of error behavior
Note Icon

Hint

The default error behavior is Continue with sibling.

Continue with iteration

Error behavior: Continue with iteration
  1. Continue with iteration

The test run will continue with the next iteration of the smart folder Validation.

Continue with sibling

Error behavior: Continue with sibling
  1. Continue with sibling

The test run will continue with the next sibling test case or smart folder. In our case, this is the smart folder DatabaseCleanUp.

Continue with parent

Error behavior: Continue with parent
  1. Continue with parent

The test run will continue with the next parent test case or smart folder. In our case, this is the test case CompleteDatabaseTest.

Stop

When the Stop error behavior is selected, an error immediately stops the entire test run.

Error behavior of setup/teardown regions

Setup and teardown regions follow a special, fixed error behavior.

  • An error in the setup region immediately stops the test.
  • If a module fails in the teardown region, the next module is run.

Run a test from the test suite view


Running a test
  1. Click RUN.

  2. Watch Ranorex Studio execute the test.

  3. Observe the progress information during the test.

Note Icon

Attention

After you’ve clicked RUN, do not use the keyboard or mouse . Doing so would interfere with the test actions and cause a test failure.

Result(s):

  • Once the test run finishes, the report appears.
Download

Further reading


Reports are described in

Ranorex Studio fundamentals > Reporting > ⇢ Introduction.

Download

Reference


You can also pause and resume test runs. This is explained in

Ranorex Studio fundamentals > Ranorex Recorder > ⇢ Run and debug recordings.

Ranorex Test Suite Runner


The Test Suite Runner is a stand-alone program which executes test suites without Ranorex Studio. It opens automatically when you double-click a test suite file in Windows.

You can use the Ranorex Test Suite Runner to execute entire test suites, run certain test cases and smart folders, or just run a specific module.

In addition, you can create new run configurations the same way as in Ranorex Studio.

You can’t make changes to the test suite itself, however.

  1. In Windows, double-click a test suite file. The file opens in the Test Suite Runner.

  2. Click RUN.

  1. Stand-alone Test Suite Runner

  2. Currently loaded test suite

USERS LOVE US

SUBSCRIBE TO OUR NEWSLETTER

Get the latest test automation information right in your mailbox.

FOLLOW US

© 2024 Ranorex GmbH. All Rights Reserved