Advanced image-based testing

As we’ve seen on the previous page, image-based testing is sometimes the solution to an issue. However, as with any other testing approach, you need to apply it correctly or it will fail.

Proper image-based testing can get challenging when the recorded and the actual image during test execution don’t match. This can be caused by simple things like a UI element being highlighted or icons changing during test execution. False positives, can also be an issue, when instead of the intended UI element, a similar-looking one is identified.

On this page, we’ll show you how to use advanced image-based settings to solve these two issues. At the end of the page, you’ll

In this chapter

    Issue: Image not found

    Sometimes minor changes, such as background or font color changes, can cause test failures. The challenge here is to ensure that UI elements are identified correctly despite these minor changes. Read on to find out how to do so.

    Create the failing test

    We’ll first create the test so it fails for the above reason. Then, we’ll fix it using the solution further below.

    To create the failing test:

    Create a new recording module, start recording and turn on image-based recording.
    In the calendar view of the Ranorex Studio Demo Application, click a date and then stop recording.

    Check the resulting action in your actions table.

    Image not found - test example preparation

    Run the test

    Now let’s run the test and see when it’s successful and when it fails.

    Image not found - test run

    Successful test run

    • When you run the test and the recorded calendar date is not selected in the demo app, the run succeeds.

    Failed test run

    • When you run the test and the recorded calendar date is selected in the demo app, the run fails.
    • This is because the date’s appearance changes slightly – clicking it gives it a gray background.
    • This change is enough for Ranorex Studio to not recognize the element using the default image-based properties.
    Initially tracked & identified UI-element and changed UI-element

    Error message in report

    Solution

    To solve this issue, we need to make changes to the image-based properties of the recorded action.

    Select the action and press F4 to open its properties.
    Click Image-based.

    Opening and changing image-based property settings

    Change the following properties:

    Seeting image-based properties

    Preprocessing

    • The preprocessing properties let you apply various filters to your comparison image.
    • These filters can make image identification robust against changes in color, sharpness, brightness, and more.
    • For our example, select the filters Edges, EdgesSobel, and Grayscale.

    Similarity

    • Preprocessing filters can remove a good deal of image differences, but not all of them.
    • The similarity property lets you set an overall value of how similar the comparison image and the actual one need to be. The default is 1.0, i.e. 100 % or identical images.
    • For our example, reduce the similarity to 0.98.

    Further reading

    For more information on preprocessing filters and similarity, refer to

    Ranorex Studio advanced > Image-based testing > ⇢ Image-based properties.

    Rerun the test

    Now rerun the test with the changed properties.

    Improved test run with successful test report

    Modified calendar view with date and gray background
    Thanks to the changed properties, the date is identified correctly and the test run is successful.

    Issue: Wrong UI element found

    When running an image-based test, it may sometimes happen that Ranorex Studio doesn’t find the recorded UI element, but a different, incorrect one that looks like the one recorded – a false positive. There are two ways to solve this issue. We’ll show you both. We also recommend combining them for best results.

    Create the failing test

    We’ll first create the test so it fails for the above reason. Then, we’ll fix it using the solution further below.

    To create the failing test:

    Start a recording, enable image-based recording, and in the calendar view of the demo app, click a single-digit calendar date.
    Note the size of the frame around the action spot, where the mouse click will be performed. This is the image-find region where Ranorex Studio will look for the 7. Its size is part of the issue. We’ll fix this later as part of solution 1.
    Stop the recording and check the resulting action in the actions table.

    Wrong image found - test example preparation

    Run the test

    Now let’s run the test to see how it fails.

    Note icon

    Note

    For our example, we’ve intentionally also made sure the recorded date (7) is selected before the test run, so that it has a gray shade and looks different from what we recorded. This isn’t necessary, but it ensures the test will find an incorrect 7 every time.

    If you don’t do this, the test may or may not find the correct 7, which is still unacceptable for a robust test. Give it a try yourself.

    Wrong image found - test run

    June 7th is the date we initially recorded. Compared to the recording it now has a gray background and therefore cannot be identified by Ranorex Studio (see note above).
    In its search for a matching UI element, Ranorex Studio finds a different 7: the one in today’s date below the calendar

    Solution #1

    Remember the action spot we noted when creating the failing test? The image-find frame around it is too narrow here, i.e. doesn’t contain enough information to make it unique. This is why it catches other 7s as well. We need to extend it using the built-in image editor.

    Image-based object selection refinement

    Frame of the default action spot: Surrounds the element very closely. This is why it also catches similar elements.
    Extended action spot frame: A wider frame includes more information (even if it’s just whitespace here) and is therefore less likely to catch incorrect elements.

    Change the frame in the image editor

    To extend the frame, open the built-in ⇢ image editor:

    Select the action and click the breadcrumb button next to the referenced image.

    In the image editor toolbar, click the button Select image find region and drag an extended image-find region around the 7, including more whitespace than before (zoom in using the magnifying glass buttons if necessary).

    Click OK.

    The test should now only find the correct 7 when you run it again. As mentioned above, this is one way to solve the issue. Read on to find out about the second way, which works even better if you combine it with the first.

    Solution #2

    The second way to fix this issue makes use of the image-ignore region feature. Here, this feature lets you define regions in the reference screenshot where Ranorex Studio won’t look for the image feature defined in the image-find region.

    This way, we can simply ignore the part of the image that causes a false positive: the bottom part showing today’s date. To do so:

    In the ⇢ image editor, click the button Select image ignore region.

    Drag a frame around the section showing today’s date.

    Now when you run the test, Ranorex Studio will ignore this section and not look for the action spot/the image-find region there, preventing a false positive.

    Tips for image-based testing

    Image-based testing can often require lots of experimentation and a good deal of experience. Follow these tips to make your life easier:

    Uniqueness

    The images you want to identify should be as unique as possible, with no, or few, alternative matches. Also use image-ignore regions to exclude similar features and prevent false positives.

    Size

    Make your comparison image as small as possible and as large as necessary. This may require some testing and experimenting.

    Similarity

    A similarity of 1.0 will only find an identical image. This is often counterproductive. A similarity of 0.99 is usually a better idea. Sometimes, you may need to go lower, but you should rarely go beyond 0.95. At this similarity value, images can already differ significantly.

    Preprocessing filters

    Use preprocessing filters! They can make your test more robust against changes in color, detail, and brightness. For example, if you have a UI element with a unique shape that may change its color due to input, you can use grayscale and the Edge filters. This way, Ranorex Studio will still find it despite color changes.

    Further reading

    Similarity and preprocessing properties are explained in

    Ranorex Studio advanced > Image-based testing > ⇢ Image-based properties.