What’s the impact of using different combination strengths in DesignWise?
When should you use 2-way? When should you use a higher strength? Learn the strategies behind using different coverage settings in DesignWise
Isn’t coffee great?
This means that every test condition was used at least once. This, however, does not guarantee that any parameter values were tested together. In other words, a parameter value merely appearing in this test set is satisfactory as far as DesignWise is concerned. That’s not particularly thorough.
In this situation, our barista friend might not be prepared for any combination of particular requests when customers place their orders. Search for a combination of the values ‘Drink Type = Hot’ and ‘Drink Size = 20-ounce’ together in our set of 7 scenarios. If solely this set of tests were referenced, the barista would be wholly unprepared to make a hot 20-ounce coffee. Let’s make sure those two values are together!
When we create a set of 2-way interactions, 45 scenarios are required to achieve our coverage goal of testing every possible PAIR of test conditions
A hot, 20-ounce drink is guaranteed to appear in your set of tests. ‘Drink Type = Hot’ and ‘Drink Size = Short’ has been tested for. ‘Hot’ and any of the sizes listed have been tested for. Likewise, ‘Hot’ and any number of espresso shots has been tested. But this concept doesn’t just apply to ‘Hot’. It applies to ‘Frozen,’ ‘Decaf,’ ‘Grande,’ and any other parameter value in your set of inputs.
These examples go on, but there’s a catch: a hot, 20-ounce drink with espresso may not have been included. In 2-way interactions, only pairs have been tested. Enter the 3-way interactions.
215 scenarios achieve the goal of testing every possible “triplet” of test conditions
By increasing the coverage level to “3-way interactions,”, you have guaranteed that a hot, 20-ounce drink with any number of espresso shots has been included. The same goes for every other possible 3-way interaction in our set of tests.
Let’s take this a step further; what if you absolutely needed a hot, 20-ounce drink with a specific number of espresso shots and decaf (or other caffeine type)? Although espresso shots with decaf in the same drink is unlikely in reality, people can have unusual tastes.
You may have also noticed the increased wait time to generate more-thorough scenarios. Not only is the number of tests getting larger, but any margin of error in your System under Test decreases. As you may have guessed, this tradeoff between test quantity and quality continues all the way up to 4-, 5-, and even 6-way interactions. This is because tests will include every quadruplet, quintuplet, and sextuplet of test conditions, respectively.
2-way (pairwise) coverage strength is often a good default strength for many types of functional tests
4-way and higher coverage strengths are rarely used except in situations where (a) all the tests are automated (therefore cheap and quick to execute) and/or (b) the potential risk for error could be extremely disastrous or expensive. For instance, safely sending people into space requires that no errors be made.
If you’re considering the pros and cons of, say, executing a smaller number of 2-way tests vs. a larger number of 3-way tests, it is extremely important for you to understand additional test design strategies that you can use to decrease (or increase) the number of tests DesignWise generates.