How do I create extra-thorough sets of scenarios?

This lesson explains how to generate scenarios that are more thorough than standard sets of pairwise tests using the algorithm setting on the Scenarios tab.

Whenever you click on the “Scenarios” menu option, DesignWise creates a 2-way (“pairwise”) set of test cases. That means that every possible pair of values will appear together in at least one of the scenarios the algorithm generates.

Details On Pairwise and Higher Coverage

To get a detailed explanation of the difference between less-thorough 2-way (AKA “pairwise” or “all pairs”) tests and more-thorough 3-way tests, click the link shown in the graphic.

Now you’re familiar with the differences between simple pairwise tests and more-thorough 3-way (and higher) sets of tests, but what if you want to generate an even more thorough set of tests? Proceed by clicking on the “Scenarios” button, and select a higher interaction strength from the drop-down list as shown below.

Selecting 3-way coverage

Within seconds, DesignWise will generate a new set of tests that will ensure coverage of your new, higher coverage strength.

In this example, you would need more than 4 times the tests needed for 2-way coverage in order to ensure that every 3-way combination of test conditions were all covered together in at least one test case. How many more tests will be needed depends on the total number of parameters and parameter values.

The number of tests will continue to increase as you increase your coverage thoroughness requirements.

Selecting 5-way coverage

5-way coverage – an extraordinarily thorough coverage level by any measure – would require 963 tests.

DesignWise generates testing coverage all the way up to spectacularly thorough 6-way coverage.

Realistically speaking, you would not ever want to execute complete 6-way sets of tests like these unless you were (a) automating the test execution, or (b) sending astronauts into space or some other similar life-and-death testing objective.

Speaking of sending people into space, it is interesting to note that NASA regularly uses this test case prioritization approach to scientifically prioritize the software tests they choose to execute in order to test their systems as absolutely thoroughly as possible with a manageable subset of test cases.

You will also notice a “mixed-strength” coverage option. That allows you to implement risk-based testing at the parameter level and select higher coverage for the parameters that are most critical.