Ranorex Standard Reporting

Report location in Ranorex Studio

In Projects view, reports appear under the Reports folder in your project.

In Windows, you can find reports in two places:

• Project folder → Reports
  This location typically contains shortcuts to the original report files.

• Output directory → Reports
  This location contains the original reports, plus additional files used for reporting and customization (for example layout files and raw data files).

  Note: If you’re customizing reports (e.g., adding images like a logo), you’ll usually work with the files in the output directory Reports folder, because that’s where the report assets are processed and stored.

Delete old reports

To keep your workspace clean, you can remove reports directly from Ranorex Studio:

1. In Projects view, right-click Reports.

2. Select one of the available cleanup options:

   • Delete all reports
   • Delete reports older than 7 days
   • Delete reports older than 14 days
   • Delete reports older than 28 days
     

Cleanup affects only the currently configured report directory. If you changed the report directory in the past, Ranorex doesn’t retroactively track older directories, those old report files won’t be deleted.

Note: Some file types are not removed (for example: .css, .png, .xsl).

Legacy behavior (Studio 9 / older solutions): For solutions created in earlier versions and opened in Ranorex Studio 9, reports (and related files) may be stored directly in the project output directory instead of inside a Reports folder.

The report header summarizes the test run and provides useful context, including:

1. Test suite name/recording module name and description: The name of the test suite or recording module from which the test was run. Under the name, find the description that you added in the Report properties.
2. System and test data
3. Error/warning counters: If there’s at least one warning in the report, an orange notification appears, as shown below.
   

4. Test results: A pie chart summarizing the test results.

   Note: The chart counts test cases only. Smart folders are not counted in the chart, but you can still see their results in the report details.
   

5. Successful test: A green pie chart means all test cases were successful.
6. Failed test: A red pie chart means all test cases failed.
7. Test summary: Pie chart with successful, failed, and blocked test cases. In the example, the run configuration contained 8 test cases. The first two test cases passed successfully. The third failed, after which the test was aborted, blocking the remaining 5 test cases.

By default, the details for test cases, smart folders, etc. are collapsed. Click the arrow next to an item’s name to show the details.

The report mirrors the structure of the test you ran:

• The report hierarchy matches the test suite (or recording) used for the run.

• Each container (test cases, smart folders, modules) appears in the report.

• Each action can produce messages, depending on the configured report level threshold.

1. The structure is the same as in the test suite of this test run.
2. The Recording1 item contains all the report messages for the actions in it.

1. Time: By default, this column displays the relative execution time measured from the beginning of the test run. This can be changed in the report settings.
2. Level: The second column shows the report level of the action performed.
3. Category: This column shows the action type performed.
4. Message: The report message contains more detailed information as to what happened during the execution of the action.
   
   Report message stating that the key sequence ‘Harry’ was performed on a specific repository item.

In case of a failure, the report will also include two screenshots, one at the moment of failure and one right before the failure occurred.

1. Screenshot immediately before the failure.
2. Screenshot during failure.

In data-driven tests, a test case or smart folder bound to data goes through several iterations depending on the data source. In the report, each of these iterations is shown separately.

1. Data iterations: Data iterations are indicated by the Rows: # labels next to test containers. Rows # indicates how many times the data container will be iterated – one time for each row.
2. Data rows in test report: Each data row/iteration and its details are shown in the report.
3. Variable and value: Variables and values used in this iteration.

Note: Secret test data can be masked in the data source dialog. This will also apply to the report. If, for example, age and gender are masked, the data appears as shown below:

Reference: Data iterations and masking data are explained in Data-driven testing.

If a test case or a smart folder is run iteratively, each run iteration appears in the report with its details.

• Run iterations are indicated by the Iterations: # and Run: # labels next to the test containers.
• Iterations: # tells you how many times the test container is run.

Reference: Run iterations are explained in Running tests.

There are three filters available in the report: 

• One for filtering test containers
• One for filtering messages for actions
• One for filtering the action category

Filter test containers

Use the checkboxes in the upper part of the report to filter test containers by their success status.

1. Success: displays report messages with report level ‘Success’ if checked, and if these messages are included in the report based on report levels. Hides these messages if not checked.
2. Failed: displays report messages with report level ‘Failure’. Hides these messages if not checked.
3. Blocked: displays blocked test cases of a test run when checked. Hides blocked test cases if not checked.

Filter action messages

You can also filter action messages in modules by their report level.

Filter category messages

You can also filter the action messages in the modules by their action type using the Category filter.

You can directly jump from the report message to the corresponding action or test suite item in Ranorex Studio.

1. Mouse over a report entry, i.e. a test container or a report message for an action.
2. Click the Jump to item button that appears in the right upper corner of the report entry.

1. Jump to item button.
2. Corresponding action with linked repository item.

When you open a report outside of Ranorex Studio, the Open in Spy button becomes available. Click the button to open the corresponding report item in Ranorex Spy.

1. Open in Spy report functionality.

Video reporting lets you record a test run as a collection of videos. This also works on Ranorex Agents.

Enable video reporting

Video reporting is disabled by default. You can enable and configure it in the Report settings.

Play video reports from the report

In the report, click Play video next to a test case to play the respective video.

Video directory

The videos are saved in the output directory in the Reports folder. For each test run, a separate folder is created.

1. Reports folder in the output directory
2. Three video folders

Ranorex generates the report as the test progresses, so you can view it even during long runs:

• The report starts collecting data as soon as the run begins.
• Ranorex writes the report to disk based on Auto save interval (default often shown as 30 seconds; configurable).
• Once the report is written, you can open the in-progress report by double-clicking the report file.

Note: Naturally, if you try to open the report on the machine the test is running on, you will very likely cause a test failure because you will be interacting with the UI.

To access the report settings:

1. In the test suite view, right-click the test suite.
2. Click Properties.
3. Click the Report tab.

Further reading: Report settings are explained in Report settings.