Running tests and reporting to TestRail
After you’ve completed an import or export, you can run your test suite and report the results to TestRail in real time. You can run your test from Ranorex Studio, or you can use Continuous Integration tools. Either way, the results on TestRail will be the same. The only difference is in how the run is started.
In this chapter
Running from Ranorex Studio
Click the TestRail run button. The Launch test run screen will appear.
Select which project connected to TestRail you want to run.
Select whether you want to report the results ⇢ to a new or ⇢ an existing test run on TestRail.
If to a new run, enter a name, if to an existing run, select to which one.
Click on Run and report results to TestRail to launch the test run.
Running from a CI tool
Click the TestRail run button. The Launch test run screen will appear.
Select which project connected to TestRail you want to run.
Select whether you want to report the results ⇢ to a new or ⇢ an existing test run on TestRail.
If to a new run, enter a name, if to an existing run, select to which one.
Click the green copy button to the right of the generated string to copy it to the clipboard. Do not use manual copy + paste, or the password will not be copied correctly.
Paste the string into the appropriate field in your CI tool. You can add ⇢ more arguments if desired.
Whichever method you choose for running, the test will always execute according to the currently selected run configuration. Whether a test case is connected to TestRail or not is irrelevant for execution.
Tip
As the test is running, you can open the run in TestRail to check its status. This is especially useful for long test runs.
Reporting to TestRail
Once Ranorex Studio has started running the test, results will be reported to the TestRail test run in real time. The Ranorex report will always show the full results for the selected run configuration.
Reporting to a new test run
Ranorex Studio will report results only for test cases that are linked to TestRail AND part of the current run configuration. Other test cases will not be reported to TestRail.
Reporting to an existing test run
Ranorex Studio will report results only for test cases that have a counterpart in the existing TestRail run AND are part of the current run configuration. Other test cases will not be reported to TestRail. The Ranorex report will contain a message detailing which test cases didn’t have a counterpart in TestRail.
Iterations
In principle, the results for test cases with iterations are displayed just like in the Ranorex report. In other words, if all iterations pass, the case will display as passed, if one fails, the case will display as failed in TestRail. However, the pass/fail information for the individual iterations is only displayed in the Results & Comments section of a case in TestRail.
Blocked test cases
In a Ranorex report, blocked always means not executed. This is not the case in TestRail. Therefore, blocked test cases are reported as “untested” in TestRail.