Sample Test Cases – Airplane Ticket Reservation

Learn how an optimized set of scenarios for a ticket reservation application is efficiently generated in DesignWise.

We want to test this simple flight reservation process relatively thoroughly.

We know that testing each item in our system once is not sufficient; we know that interactions between the different things in our system could well cause problems. As thoughtful test designers, we want to be smart about testing for potential problems caused by interactions without going off the deep end and trying to test every possible combination.

DesignWise makes it quick and simple for us to select an appropriate set of tests whatever time pressure might exist on the project or whatever testing thoroughness requirements we might have. DesignWise-generated tests automatically maximize variation, maximize testing thoroughness, and minimize wasteful repetition.

How should we describe the quantity of adult tickets that can be purchased? How many values should we use here? This test design consideration comes up very frequently. How we answer the question has significant implications here (and in many DesignWise test models).

The drop down menu above shows 6 different quantities. The drop down list under Children includes “zero” as an option as well.

• An obvious solution would be to include separate Values for each of the numbers.
• If we did that though, we would create a model with at least 6 X 7 = 42 tests.
• An alternative option that we’ve decided to use here is the idea of Equivalence Classes.
• Here’s the basic idea behind equivalence classes: whether there are 2 tickets or 6 tickets probably won’t impact the system. Or so everyone tells us. So we decide to describe our system using a smaller number of ticket quantity Values (in our case: 0, 1, and “more than 1”). The equivalence class test design approach has the advantage of keeping the number of tests much lower without sacrificing testing coverage.

It is often useful to start by identifying a verb and a noun for the scope of our tests

Consider additional variation ideas to ask about our verb and noun using “newspaper questions” – who, what, when, why, where, how, how many?

Designing powerful software tests requires people to think carefully about potential inputs into the system being tested. As described in this blog post, we strongly encourage test designers to ask the newspaper questions of who?, what? when? where? why? how? and how many?

What

• What class of ticket is being reserved? (Coach, Business, First)

Where

• Where is the flight leaving from? (India, the Philippines, the United States)
• Where is the flight going to? (the United States, the Philippines, India)

How Many

• How many adult tickets? 1 to 6 –> (1, more than one)
• How many children tickets? 0 to 6 –> (0, 1, more than one)

Once we have thought through what test conditions are important enough to include in your model (and probably excluded things that will not impact how the system being tested operates), DesignWise makes it easy to quickly create powerful tests that will allow us to maximize our test execution efficiency.

Once we enter our parameters into DesignWise, we simply click on the “Scenarios” link in the left navigation pane.

DesignWise helps us identify a set of high priority scenarios within seconds

The coverage achieved in the 10 tests above is known as pairwise testing coverage (or 2-way testing coverage). DesignWise-generated pairwise tests have been proven in many contexts and types of testing to deliver large thoroughness and efficiency benefits.

DesignWise gives test designers control over how thorough they want their testing coverage to be. While this is a very simple model with only 162 total possible scenarios, in many cases, DesignWise would allow testers to quickly generate dozens, hundreds, or thousands of tests using DesignWise’s “coverage dial.”

Selecting “3-way interactions” generates a longer set of tests which cover every single possible “triplet” of Values

DesignWise generates and displays this set of 3-way tests to you within a second or two. This 3-way coverage strength would be dramatically more thorough than typical sets of manually selected scenarios typically used by large global firms when they test their systems.

If a tester spent a couple hours trying to select tests by hand that achieved 100% coverage of every single possible “triplet” of Values (e.g., (i) From the United States, (ii) flying First Class, and (iii) with More than 1 child), the following results would probably occur:

• It would take far longer for a tester to try to select a set of tests that would be nearly as thorough.
• The tester trying to select tests by hand would create far more than 34 tests (which is the optimized solution, shown above) to achieve this extremely thorough 3-way coverage goal.
• Almost certainly, if the tester tried to achieve this coverage goal in 50 or fewer tests, there would be many, many gaps in coverage (e.g., 3-way combinations of Values that the tester accidentally forgot to include).
• Unlike the DesignWise-generated tests, many of the tester’s hand-selected scenarios would probably be highly repetitive from one test to the next; that wasteful repetition would result in significant wasted effort in the test execution phase.

The Auto-scripting feature saves testers a lot of time.

You document a single test script in detail from the beginning to the end. As you do so, you indicate where your variables (e.g., “the United States” or “India” or “the Philippines”) are in each sentence. That’s it. You’re ready to export all of your tests.

From there, DesignWise automatically modifies your sample/template test script and inserts the appropriate Values into every test in your model (whether it has 10 scenarios or 1,000).

We can even add simple Expected Results to your detailed test scripts

If you describe Expected Results like the one above, DesignWise will automatically add Expected Results into every applicable step in every one of your tests.

DesignWise’s Expected Results feature makes it easier to maintain your test sets over time because Expected Results automatically update as test sets get changed over time.

After the first 23 tests were executed in the “3-way coverage” version of this model, more than 84% of all possible “triples” would have been tested for.

With DesignWise’s matrix reports, we can even tell which specific pairs have not been tested after any test

The matrix coverage chart shows that after the first 8 tests (of the 2-way coverage version of this model), there are exactly 11 pairs of values that have not been tested together yet. Pairs like “Flying to = India” and “Class = Business” have not yet been tested in the same test.

If you open up this sample model and look at the matrix chart view when number of tests is set to 10, you will see that the entire chart is green (because all pairs of values have been tested together at least once in the model’s 10 tests).

DesignWise’s Scorecard provides automated analysis to help you review each of your models.

Mistake Identification: We have helped testers build test models for years. In doing so, we have seen the most common test design mistakes that new users sometimes make. We have put in automated warnings to alert testers when something looks like it might be wrong with their set of tests.

Feature Usage Ideas: The scorecard identifies features that have not yet been used in your model.

You can export from DesignWise in several different formats. Mind maps can be a great option if a tester wants to get quick, actionable guidance from stakeholders about which variation ideas should (or should not) be included.

As shown below, detailed test scripts (complete with tester instructions and Expected Results) can be exported also.

The detailed test scripts shown above were created using DesignWise’s Auto-Scripts feature.

Other possible export formats could include test data tables in either CSV or Excel format or even Gherkin-style formatting.