Welcome to the Ranorex Support Portal

Running tests and reporting to TestRail

After you’ve completed an import or export, you can run your test suite and report the results to TestRail in real time. You can run your test from Ranorex Studio, or you can use Continuous Integration tools. Either way, the results on TestRail will be the same. The only difference is in how the run is started.

In this chapter

E3030-0000010

Running from Ranorex Studio


  1. Click the TestRail run button. The Launch test run screen will appear.

  2. Select which project connected to TestRail you want to run.

  3. Select whether you want to report the results ⇢ to a new or ⇢ an existing test run on TestRail.

  4. If to a new run, enter a name, if to an existing run, select to which one.

  5. Click on Run and report results to TestRail to launch the test run.

Running from a CI tool


  1. Click the TestRail run button. The Launch test run screen will appear.

  2. Select which project connected to TestRail you want to run.

  3. Select whether you want to report the results ⇢ to a new or ⇢ an existing test run on TestRail.

  4. If to a new run, enter a name, if to an existing run, select to which one.

  5. Click the green copy button to the right of the generated string to copy it to the clipboard. Do not use manual copy + paste, or the password will not be copied correctly.

  6. Paste the string into the appropriate field in your CI tool. You can add ⇢ more arguments if desired.

Whichever method you choose for running, the test will always execute according to the currently selected run configuration. Whether a test case is connected to TestRail or not is irrelevant for execution.

Reporting to TestRail


Once Ranorex Studio has started running the test, results will be reported to the TestRail test run in real time. The Ranorex report will always show the full results for the selected run configuration.

Reporting to a new test run

Ranorex Studio will report results only for test cases that are linked to TestRail AND part of the current run configuration. Other test cases will not be reported to TestRail.

Reporting to an existing test run

Ranorex Studio will report results only for test cases that have a counterpart in the existing TestRail run AND are part of the current run configuration. Other test cases will not be reported to TestRail. The Ranorex report will contain a message detailing which test cases didn’t have a counterpart in TestRail.

Iterations

In principle, the results for test cases with iterations are displayed just like in the Ranorex report. In other words, if all iterations pass, the case will display as passed, if one fails, the case will display as failed in TestRail. However, the pass/fail information for the individual iterations is only displayed in the Results & Comments section of a case in TestRail.

Blocked test cases

In a Ranorex report, blocked always means not executed. This is not the case in TestRail. Therefore, blocked test cases are reported as “untested” in TestRail.

USERS LOVE US

SUBSCRIBE TO OUR NEWSLETTER

Get the latest test automation information right in your mailbox.

FOLLOW US

© 2024 Ranorex GmbH. All Rights Reserved