Welcome to the new Ranorex Support Portal

Ranorex
Try DesignWise with Ranorex
Help Center › User Guide › Ranorex DesignWise › DesignWise Features
User Forum

How do I see what specific combinations of test inputs are covered by a set of tests?

Learn how to use the Coverage Matrix to better understand specific interaction coverage after every scenario.

Click on the “Analysis” button to use this feature

Understanding the Matrix

No, you’re not Neo, but this is like having super powers.

Each square of the matrix chart represents a pair.

How to read the chart:

Read the values listed across the top

Read the values listed down the left side

Find an intersection of two values (one from top and one from side):

  • If it is green: the pair is already covered.
  • If it is red: the pair is not covered yet given the scenario you selected via slider.
  • If it is black: the pair is marked as invalid.
Slide your way to precise coverage!
The slider at the top allows you to visualize which pairs are covered with each test.

When you arrive at the Matrix Chart screen, you’ll automatically see the chart cycle from Test Case 0 through Test Case N (the final test in your set). By default, the slider will “settle” at the position that achieves around 80% coverage.

Communicate your testing coverage precisely

With the Matrix Chart, communicating your testing coverage has been made even easier! Imagine a situation where you have created some tests and your manager comes by and says, “Development is running behind, so we’ll only have time to execute 2/3s of your tests before Go-Live. What will be missed?” You can easily respond with a slide like the one below:
Can I see 3-way and higher interactions?

At this time, the Matrix Chart only supports demonstrating the pairs that are covered with each test.

Special Note for Color-Blind Users

Pressing “C” on your keyboard changes the color scheme of the Matrix Chart. The following 3 options are available:

Option 1: Default

Option 2: Blue-Orange
The Blue squares represent pairs that have been covered while the Orange squares represent pairs that have not been covered yet.

Option 3: Purple-Orange

The Purple squares represent pairs that have been covered while the Orange squares represent pairs that have not been covered yet.

Dealing with Large Numbers of Test Cases

As seen in the above picture, large numbers of test cases can appear to be overwhelming in the Matrix Chart. DesignWise can help make these overwhelming instances more manageable with Zoom and Click-Drag features. If I want to key in on the Java console: Show console and Show sandbox warning banner combination: Select, I can hover my mouse roughly over where I think that combination will appear in the Matrix Chart and use my mouse wheel to zoom in. Like this:
As you can see, the square that accounts for Java console: Show console and Show sandbox warning banner combination: Select is able to be examined easily after zooming. By hovering my mouse over that square, I can see that this particular pair of values is covered within the current 11 test cases and this pair is first feature in Test Case 2. It is also possible to click and drag the Coverage Matrix to check out different squares at your desired zoom level.
By clicking and dragging and then hovering over the square for the combination of Shortcut Creation: Prompt user and Show sandbox warning banner combination: Select, I can see that this pair of Parameter Values would not appear until Test case 26. With this information in mind, I can then weigh my testing options. Do I need to test for this 2-way interaction? Is it worth generating 26 test cases? Am I better off writing a requirement that involves these two parameter values etc.? The Coverage Matrix can be invaluable in the decision-making process.