UI elements identification

Let’s first take a look at how we as humans identify UI elements, the process is usually quite intuitive.

Take the start page of the Ranorex Studio demo application, for example:

1. Title bar

2. Main menu

3. Tabs for different panels

4. Text label showing a welcome message

5. Text field for entering a name

6. Text link for resetting the welcome message

7. Exit button for closing the application

User’s point of view

To better understand the challenge of reliable UI element identification, let’s take a look at one particular UI element, the text input field.

How do you know that the framed UI element is a text input field?

Possible answers may be:

• I read it in the user guide of the software
• I know from experience
• I tried entering text and it worked
• Someone told me/showed me
• Because of the prompt ‘Enter your name’ and the ‘Submit’ button

All of these are good indicators, and if the field accepts text entry, you can be quite certain that it’s a text input field. However, unless you have access to the software’s code, you can’t tell with absolute certainty what the UI element does in detail. Instead, you have to rely on context-based clues and your general idea of what a text input field does, for example, an “inner representation” of the UI element.

Ranorex Studio’s approach

As an application, Ranorex Studio doesn’t have access to indicators like user guide information, being shown by others, or human intuition. Its approach is based on hooking into an AUT, detecting which technology is used for the GUI, and then identifying UI elements and categorizing them according to roles, capabilities, etc.

However, in some ways, Ranorex Studio works in a very similar way to human UI element identification: It also generates “inner representations” of UI elements, called repository items. These have a unique identifier assigned to them, the RanoreXPath, that locates the respective UI element in the GUI structure of the software. This is a two-step process:

1. UI elements in the GUI of the AUT

2. Representation of these UI elements

Important to know:

• Ranorex Studio represents UI elements as repository items in the test solution.
• Ranorex Studio identifies the type of the UI element on a best-guess method and maps it to a set of specific roles and capabilities.
• Ranorex Studio assigns a unique identifier to every repository item. This identifier is called an item’s RanoreXPath and reflects the position of the referenced UI element in the GUI structure.

The UI elements in an AUT are organized in a certain structure. When Ranorex Studio scans a GUI to identify UI elements in it, it translates this structure into a hierarchical element tree that you can see in Ranorex Spy.

1. Form RxMainFrame
   The top-level UI element of the AUT, i.e. the demo application program window as a whole.
   This UI element contains almost all other UI elements of the demo application (there are special cases like context menus and list items, which normally appear as separate branches in the element tree).
   Examples of UI elements contained in this tree are the Exit button, the main menu, or the copyright text label.

2. TabPageList RxTabControl
   The tab container with the tabs Introduction, Test Database, Image-based automation, UI-element test area, and Upload.
   Ranorex Studio gets the name of this UI element (RxTabControl) directly from the AUT.

3. TabPage RxTabIntroduction
   The Introduction tab in the demo application.
   This UI element contains all other elements present in this tab.
   Ranorex Studio gets the name of this UI element (RxTabIntroduction) directly from the AUT.

4. Text field txtUsername
   The text input field for changing the name is displayed in the welcome message.
   This UI element, along with the others in this panel, is contained in the tab element RxTabIntroduction.

The above example is simplified for explanation purposes. The full hierarchical tree would look like this in Ranorex Spy: