Structures and How to Choose One – Ranorex

Selecting the right structure for your Ranorex Studio project depends on the complexity of your application and the size of your testing team. This article outlines the three main structural approaches to help you decide which one fits your needs.

Example test requirements

To compare the different test structures, use the following test requirements:

Two smoke tests ensure that the AUT is installed correctly and can be started and closed by Ranorex Studio.
Two sanity tests ensure the correct version of the AUT is used and that all UI elements are implemented and work according to specifications.
Three functional UI tests ensure that the AUT behaves correctly for its administration, database, and file-importing functions.

Basic test structures

There are three basic ways to organize your automation projects. For each of the structures, find a list of advantages and disadvantages as well as an example of how to translate the example test requirements into the test structure.

No single structure is the best. It all depends on your requirements.

Solution-based structure

In this approach, there is one solution for each of the three different test types. The actual tests are then structured in projects, with one test suite per project.

Ranorex Studio solution for the smoke tests
Ranorex Studio solution for the sanity tests
Ranorex Studio solution for the functional tests

Advantages

Disadvantages

Very clear structure at the top level. Useful for extensive and complex test requirements.
Particularly useful when testing an AUT that has different UIs and requires multiple repositories, for example, testing frontend and backend UIs.
Easier to keep test suite structures simple and efficient.
Can make test execution quicker.

Higher maintenance: Every solution has its own settings, repository, and module library that you need to take care of.
It’s not possible to execute all three solutions automatically in sequence from Ranorex Studio. You can only do so with external scripts, for example, in a continuous integration environment.
Each solution has its reports. This can make it harder to collect and present results.
There is no easy way to exchange data between the three functional tests.

Project-based structure

In this approach, there is one solution that contains one project for each of the three different test types. The actual tests are then structured in multiple test suites, with one test suite per test.

The solution AuT_Testing contains all three projects
Project FunctionalTest for the implementation of the three functional tests
Project SanityTest for the implementation of the two sanity tests
Project SmokeTest for the implementation of the two smoke tests

Advantages

Disadvantages

Convenient: One solution contains all the tests.
Easier maintenance: No differing settings, and usually only one module library and repository.
Even though the structure is less clear at the solution level, it’s very clear at the project level. Therefore, it keeps the advantage of simple and efficient test suite structures.
You can easily run the projects and the contained test suites in sequence from Ranorex Studio by using the test sequence functionality.

Each project/test suite has its own reports. This can make it harder to collect and present results.
There is no easy way to exchange data between the three functional tests.

Test suite-based structure

In this approach, there is one solution that contains one project. This project then contains one test suite for each of the three different test types. The actual tests are then structured in test cases inside those test suites.

The solution AuT_Testing contains one project
The project AuT_Testing contains one test suite per test type.
The individual test suites contain the actual tests as test cases.

Advantages	Disadvantages
Convenient: One solution contains all the tests. Easier maintenance: No differing settings, and usually only one module library and repository. You can easily run the test suites in sequence from Ranorex Studio by using the test sequence functionality. Exchanging data between the three functional tests is now easy, but still difficult between the different test types.	Quickly leads to bloated and overly complex test suite structures, making maintenance, troubleshooting, and collaboration much harder and affecting test performance negatively.

Choose your structure

This section describes the factors you should consider when choosing your structure.

Avoid extremes

For most requirements, the project-based structure is a good choice.

It offers a flexible approach with good organization, simple test logic, and high convenience.

Both the purely solution-based and the test suite-based models are extremes. We recommend using them only if your test requirements fit them perfectly.

Use the solution-based structure for very large, complex tests. The very high maintenance required by this approach is the most important disadvantage. You have different settings, repositories, and module libraries for each solution.

Use the test suite-based structure for small, simple tests. It’s easy to set up a test suite-based structure and quickly get a test running. Maintenance is also low — but only if your tests stay small and simple! With more complexity, you quickly get bloated, hard-to-manage test suites.

Consider your UI elements

The UI elements involved in your tests are one of the more important criteria when choosing a structure. If your tests deal with multiple different UIs, like a frontend and a backend, it makes sense to use multiple repositories.

With multiple repositories, the solution-based structure becomes more attractive. However, you can also use multiple repositories in the other structure types. So if your test isn’t very complex despite two different UIs, a project-based structure with two repositories is probably still better.

Data exchange as a requirement is overrated

Exchanging data between tests is hard when they’re not in the same test suite. However, it’s often not truly necessary. You can avoid data exchange by designing reference data and test cases differently.