Report levels

Report levels categorize events that happen during a test run and govern what information is included in the report. When your tests contain hundreds of test cases with thousands of modules, report levels are the key to striking the balance between detail and brevity. In this chapter, you’ll find out how report levels work and how to use them.

In this chapter

    tipp icon


    The screencast “Report levels” walks you through the information found in this chapter.

    Watch the screencast now

    Download the sample solution

    The explanations in this chapter are based on a sample solution that you can download below.

    Sample solution

    Theme: Build a test
    Time: 15 minutes

    Download sample file

    Install the sample solution:

    Unzip to any folder on your computer.
    Start Ranorex Studio and open the solution file Introduction.rxsln

    tipp icon


    The sample solution is available for Ranorex versions 8.0 or higher. You must agree to the automatic solution upgrade for versions 8.2 and higher.

    The default report levels

    Ranorex Studio comes with 6 default report levels, each with a corresponding color and integer value.

    Predefined Ranorex report levels

    Default Ranorex report levels

    • The higher the integer value of a report level, the more important it is, i.e. a Debug notification is less important than an Info notification.
    • Recording modules use the default report levels.
    • In code modules, you can ⇢ define your own report levels.

    The purpose of report levels

    In a test report, one executed action means at least one report message. In large test suites with thousands of actions, the test report can quickly become cluttered. With report levels, you can control which messages are included in the report. Report levels categorize report messages by importance and act as a filter.

    The image below illustrates this.

    Actions raising report attention by report levels

    Actions raising report attention by report levels

    Actions #1 through #4 and #6 through #7 have the report level Info (value = 20).

    A validation always either succeeds or fails, and it’s usually crucial to include this in a test run. Therefore, a successful test validation (action #5) has the report level Success (value = 110). A validation that fails has the report level Failure (value = 120).

    Report level threshold

    The report level threshold is the integer value that defines the minimum report level a message must have to be included in the report. If the report level is below this value, the message will not appear in the report.

    The report level threshold is set in the test suite for structure items, i.e. the test suite item, test cases, and smart folders.

    The images below illustrate this.

    Report threshold level of 20 (Info)

    Set recognition level = Info (value=20)

    Threshold set to Info (value = 20).

    In the report, all messages with a minimum report level of Info are included.

    Report level threshold of 110 (Success)

    Set recognition level = Success (value=110)

    Threshold set to Success (value = 110).

    In the report, all messages with a minimum report level of Success are included. In this case, that’s only the message notifying you of a successful validation.

    Set report level threshold

    The report level threshold is set in the test suite for structure items, i.e. the test suite item, test cases, and smart folders. By default, structure items inherit the setting from their direct parent.

    Change to the test suite view.

    Right-click the test suite, a test case, or a smart folder.

    Click Report level and select the desired value.

    Setting report level for test suite structuring elements


    When this option is selected, the setting is inherited from the direct-parent structuring item.


    Let’s apply the above explanations to an example. We’ll insert a log message and set its report level.

    Initial situation

    The recording module in the sample solution contains 7 recorded actions, one of them a validation action.

    The recognition level of the test case is set to Success. This means that currently, only the successful or failed validation would be reported during a test run.

    We now want to add a log message which tells us that the Submit button has been clicked, and of course we want it to show up in the report, whereas all other actions except for the validation shouldn’t show up.

    Here’s how:

    Add a log message after action #4.

    Example log message

    Enter a message and set its report level to Success.


    The report levels for the actions in this recording and the report level threshold of the test case now look like this:

    Test report of application example

    Log message with report level Success in report.

    Successful test validation with report level Success in report.