How do I create extra-thorough sets of scenarios?
This lesson explains how to generate scenarios that are more thorough than standard sets of pairwise tests using the algorithm setting on the Scenarios tab.
DesignWise Creates a Pairwise Set of Scenarios by Default
To get a detailed explanation of the difference between less-thorough 2-way (AKA “pairwise” or “all pairs”) tests and more-thorough 3-way tests, click the link shown in the graphic.
Selecting n-Way Test Coverage
In this example, you would need more than 4 times the tests needed for 2-way coverage in order to ensure that every 3-way combination of test conditions were all covered together in at least one test case. How many more tests will be needed depends on the total number of parameters and parameter values.
The number of tests will continue to increase as you increase your coverage thoroughness requirements.
5-way coverage – an extraordinarily thorough coverage level by any measure – would require 963 tests.
Realistically speaking, you would not ever want to execute complete 6-way sets of tests like these unless you were (a) automating the test execution, or (b) sending astronauts into space or some other similar life-and-death testing objective.
Speaking of sending people into space, it is interesting to note that NASA regularly uses this test case prioritization approach to scientifically prioritize the software tests they choose to execute in order to test their systems as absolutely thoroughly as possible with a manageable subset of test cases.
You will also notice a “mixed-strength” coverage option. That allows you to implement risk-based testing at the parameter level and select higher coverage for the parameters that are most critical.