In this chapter, you’ll learn where reports are stored, what they contain, how to read them, and where you can change report settings.
In this chapter
The file name of a report is generated automatically. It changes depending on whether the test is executed from a recording module or from a test suite.
.rxlog, short for Ranorex Log
You can customize how report file names are generated. This is explained in Ranorex Studio system details > Settings & configuration > ⇢ Report settings.
In the projects view, reports are listed in the
Reports folder of your project.
In Windows, you can access reports from two different folders. The original reports along with the layout files and the raw data files of the report (explained later as part of customizations) are stored in the Reports folder in the output directory.
The Reports folder in the project’s folder only contains a shortcut to the original reports, and does not contain any of the stylesheet or raw data files.
See here for an illustration:
Reports folder in the project’s folder.
Reports folder in the output directory of the project.
Layout files for the report.
Shortcuts to the original report files.
Original report files with raw data files.
To ease the report file maintenance, right-click Reports in the Project View panel.
The following options are listed for report deletion which is based on the difference between the creation date and the current date:
The reports are removed from the current configured directory.
Ranorex does not track configured report directories that are changed. If the user changes the configured report directory, the deletion does not affect any report files in previous directories.
The files with the following extensions that are not deleted:
A confirmation dialog displays to proceed with the removal.
The report header displays a range of useful data and summarizes the test results.
Test suite name/recording module name and description
The name of the test suite or recording module from which the test was run. Under the name, find the description that you added in the Report properties.
System and test data
If there’s at least one warning in the report, an orange notification appears, as shown below.
A pie chart summarizing the test results.
A green pie chart means all test cases were successful.
A red pie chart means all test cases failed.
By default, the details for test cases, smart folders, etc. are collapsed. Simply click the arrow next to an item’s name to show the details.
The report is structured in the same way as the test suite or recording module on which the test run was based. Each structure element in the test suite appears the report, as well as each test action item. Report messages appear for all actions in the order of execution provided that they met the report level threshold.
The structure is the same as in the test suite of this test run.
The Recording1 item contains all the report messages for the actions in it.
By default, this column displays the relative execution time measured from the beginning of the test run. This can be changed in the ⇢ report settings.
The second column shows the report level of the action performed.
This column shows the action type performed.
The report message contains more detailed information as to what happened during the execution of the action.
In case of a failure, the report will also include two screenshots, one at the moment of failure and one right before the failure occurred.
Screenshot immediately before the failure.
Screenshot during failure.
In data-driven tests, a test case or smart folder bound to data goes through a number of iterations depending on the data source. In the report, each of these iterations is shown separately.
Data rows in test report
Variable and value
Data iterations and masking data are explained in Ranorex Studio advanced > ⇢ Data-driven testing.
If a test case or a smart folder is run iteratively, each run iteration appears in the report with its details.
Run iterations are explained in Ranorex Studio fundamentals > Test suite > ⇢ Running tests.
There are three filters available in the report:
Use the checkboxes in the upper part of the report to filter test containers by their success status.
Success: displays report messages with report level ‘Success’ if checked, and if these messages are included in the report based on report levels. Hides these messages if not checked.
Failed: displays report messages with report level ‘Failure’. Hides these messages if not checked.
Blocked: displays blocked test cases of a test run when checked. Hides blocked test cases if not checked.
You can also filter action messages in modules by their report level.
You can also filter the action messages in the modules by their action type using the Category filter.
You can directly jump from the report message to the corresponding action or test suite item in Ranorex Studio.
Mouse over a report entry, i.e. a test container or a report message for an action.
Click the Jump to item button that appears in the right upper corner of the report entry.
Jump to item button.
Corresponding action with linked repository item.
When you open a report outside of Ranorex Studio, the Open in Spy button becomes available. Click the button to open the corresponding report item in Ranorex Spy.
Open in Spy report functionality.
Video reporting lets you record a test run as a collection of videos. This also works on Ranorex Agents.
Video reporting is disabled by default. You can enable and configure it in the ⇢ Report settings.
In the report, click Play video next to a test case to play the respective video.
The videos are saved in the output directory in the Reports folder. For each test run, a separate folder is created.
Reports folder in the output directory
Three video folders
Ranorex generates the report as the test run progresses. You can view it at any time during the test run. This is especially useful for very long test runs.
To access the report settings:
In the test suite view, right-click the test suite.
Click the Report tab.
Report settings are explained in Ranorex Studio system details > Settings & configuration > ⇢ Report settings.