Challenges and solutions in automated functional UI testing

 

In this article, Challenges and solutions in automated functional UI test, we will discuss the below topics:-

• Why functional UI test automation is flaky?
• Testable UI
  • Enter Katalon Recorder
• XPATH/ CSS Locator Builders
  • Other helpful features from Katalon builders

 

 

Why functional UI test automation is flaky

We know that functional UI test automation helps save QAs a lot of time so that they can focus more on User Acceptance Testing and Exploratory Testing. However, some drawbacks prevent many organizations and teams from successfully adopting it. The cost of writing and maintaining scripts, as well as the level of flakiness, are generally higher than other types of testing. That is also the reason why some people generally criticize UI automation testing.

Nowadays, web applications evolve quickly with Agile-style short development cycles. Moreover, in favor of agile development and deployment, the architecture of web applications on both the backend and frontend side is getting increasingly complex with various components separated logically and physically that are only linked together via web services. Even with a substantial amount of automated unit tests and integration tests, doing functional UI test automation is still mandatory to make sure the software will behave as desired.

Therefore, instead of abandoning functional UI test automation, we had been actively looking for a way to mitigate its defects in our team, and as a result, maximize its benefits and ROI. When we take a closer analysis, we realize that three leading causes reduce the effectiveness of UI automation testing:

1. Firstly, the time spent on writing good and stable XPath/CSS locators. Additionally, updating them whenever the underlying HTML markup changes.
2. The network itself is sometimes flaky, which leads to testing failures.
3. The code does not handle some rare cases such as race conditions.

The third cause is actually by bugs inside the software, and the engineering team should fix it (don’t call it flaky tests!). The second one can mitigate by putting the application under test (AUT) and the tests in the same data center with a robust network (e.g., Amazon Web Services). Therefore, we focused on resolving the first cause.

 

 

Testable UI

To solve this problem, we need to understand why unit testing is highly robust and reliable. If you have googled around the internet on unit testing, you can see thousands of articles about writing testable code. Indeed, if the code is not structured carefully, the effectiveness of unit testing will be significantly affected, and it will take longer to write unit tests with excellent coverage.

Similarly, having a useful XPath locator for an element is always easy even with QAs if the element were composed of unique and stable attributes. They may be made up by the developers themselves, or by the JavaScript libraries used inside the software. The list of these attributes, along with their changes, should be communicated timely between developers and QAs. We call this kind of UI “testable UI,” and the responsibility to write testable UI lies in the hands of the developers.

However, we do not let our developers write automated UI tests. One reason is the budget — it depends on teams and companies here. We need to focus our development resources on building new features. Another reason is that QAs often do a better job in writing tests, because they focus on the users’ perspective (that’s what the word “functional” is for), while developers’ priority is often to prove their code works (no offense!). Therefore, we came up with a straightforward workflow:

1. Developers write the code.
2. They then write straightforward automated tests to prove that they have covered all expected scenarios.
3. These tests are attached to corresponding Jira issues.
4. QAs write complete automated tests based on “hints” in developers’ submitted tests.

We call simple UI tests that developers write “functional UI unit tests.” They are concise and dead simple scripts that we can throw away later. They should be composed without much time thinking about structure, design, coding convention, etc.

This workflow brings about many benefits for both developers and QAs:

1. QAs can verify very quickly if the developers have understood and covered expected scenarios.
2. It encourages developers to make the UI testable, with all elements easy to locate. QAs can also verify the testability by taking a look at the functional unit tests.
3. The direct consequence of #1 and #2 is that the communication overhead will reduce. QAs rarely need help from developers to resolve issues in the test scripts, or to clarify their understanding about scenarios.
4. QAs spend less time on writing XPath/CSS locators and instead can pay more attention to the stories and the test project’s structure, which we should treat as a complete and serious software project on its own.

For the next step, we need an excellent tool to ease this workflow.

 

 

Enter Katalon Recorder

Katalon Recorder on Chrome

 

Katalon Recorder on Firefox

 

To help make the workflow as mentioned above, even more useful, we adopt Katalon Recorder, a small extension for Chrome and Firefox. This extension is a companion tool with Katalon Studio – one of the best automation testing tools in 2020. Albeit its simple appearance, Katalon Recorder has a lot of hidden gems inside for us. The most important feature that we are making use of at this moment is its ability to generate XPath and CSS locators, with built-in locator builders as well as custom locator builders. There are other useful features, and I will go through them in later sections.

If you are interested in Katalon Recorder, please install it in your browsers (Chrome version, Firefox version).

 

 

XPath/CSS locator builders

Katalon Recorder lets users generate XPath/CSS locators by point-and-click at the elements on the web page. Katalon Recorder includes a handful of locator builders — they recognize tags, attributes, IDs, classes and calculate as many different locators as they can.

What makes Katalon Recorder truly stand out from other similar tools is the ability to import custom locator builders. We write these builders write in plain-old JavaScript, which is our developers’ native language!. We have developed several locator builders — we share them between different teams. The beauty of this feature is that all rules and guidelines for the locators now encode nicely inside those scripts — “document as code” as we call it. Here is a sample of a custom locator builder as seen in their documentation:

With the help of these builders, QAs can generate recommended locators simply by point-and-click:

 

XPath/CSS locator builders in Katalon Recorder

You can read more about this feature here.

 

 

Other helpful features from Katalon Recorder

Custom locator builders are the most praised feature for us, but there are also other features that our developers and QAs have been using regularly, and they have saved us a lot of time.

1. Record and playback all actions, along with locators generated in #1.
2. Export recorded tests to other programming languages, with the ability to write custom exporters.

Record and playback

We can create tests in Katalon Recorder by just recording your actions on the web page. We can organize them into test suites. Each step in a test consists of three parts: command, target, and value.

• Command: This column describes the type of action to take. For example, if we need to open an app, then we will use “Open”, and we will use “Type” to enter a value.
• Target: In this column, we define element locators. The locators generate, as described in the previous section.
• Value: This defines the value, for example, if we have a text box in which we need to enter a text. The value of the text would come under the “value” column.

This design may sound unadorned to professional engineers, but that’s what we expect — something dead simple to remove distractions from developers’ work. Our QAs design and write the complete tests, after all.

 

Record & Playback using Katalon Recorder

Export tests to other programming languages

Tests created by the above methods can export into different programming languages. While the built-in set only includes popular programming languages, Katalon Recorder provides a platform for us to build custom exporters. They translate all tests to our in-house Selenium-based frameworks that are different from team to team. You can learn more about the platform here.

 

Export tests to other programming languages using Katalon Recorder

List of Katalon Recorder’s supported languages:

Conclusion

The XPath/CSS locators are just only one problem that needs solving in automated functional UI testing. There are other challenges that we have to overcome to integrate this type of test into the continuous delivery pipeline as an accountable component inside. We will discuss these challenges and our approaches in the next articles.