Creating 3000 Auto Tests Framework from Scratch

Provided services

Test Automation Services, Manual Testing Services, Software Integration Services, Software Maintenance and Support Services

Client

A leading provider of network services to digital infrastructure and telecommunications operators.

Product

The “Application for Managing Products and Services” (hereinafter referred to as the Application) is part of a group of applications that support the business processes of telecom companies. These processes include enabling/disabling services, informing customers, and charging subscribers for mobile operators.

The Application consists of 2 services:

The Status Manager service receives information from adjacent services regarding changes related to subscribers. It then makes decisions about enabling or disabling certain product or service packages.

For example, a subscriber has activated a tariff plan that provides the basic services package (calls, SMS) even in case of a high debt. When the subscriber’s balance exceeds the allowable debt limit, the services included in the current tariff must be disabled until the debt is paid. In this scenario, the Status Manager is notified of the limit exceedance and sends a request to the adjacent subsystem to enable the basic service package for this subscriber.
The Loader service performs a “cold start” of the Status Manager. It is necessary to load real data into the cache so that the Status Manager can start working with other third-party services. That is what Loader does. It downloads subscribers, their current status, and activated services with tariff plans from databases of other services in the system, and uploads them to the Application cache.

Challenge

Given the complexity of the Application, deploying it to production without a stable automation framework would be extremely difficult. So, the client decided to proceed with the automation framework development.

To successfully develop and maintain such a framework, it was important to carefully select specialists with extensive hands-on experience in such projects. Seeking such specialists, the client approached JazzTeam.

We needed to perform the full cycle of test framework development, from requirements analysis to test creation. The requirements for testing were as follows:

All the services must be run in Docker within one network.
To run the services and Groovy scripts, it was necessary to use YAML files, which were supplied to the customer during the release.
Third-party services of related teams must be closed by a Mock server.

For automated testing, it was necessary to develop a comprehensive test framework that would include all the required functions for creating automated tests. This test framework was required to be compatible with all the services integrated with the “Service for Managing Products and Services”, including Couchbase, Kafka, Oracle DB, ZooKeeper, Mock Server, and REST.

The task was complicated by the lack of a clear understanding of how the final product should function because the development was still in progress when the test automation work began.

Solution

In collaboration with the teams responsible for implementing other services, it was necessary to conduct thorough testing on the stands. The configuration of these stands closely resembled the customer’s environment.

Part of our solution was comprehensive testing, which included the following:

Developing comprehensive test scenarios. The description of the scenarios was made in TestRail. In total, around a dozen scenarios have been described.
Consulting other teams about the work and configuration of the Status Manager.
Executing test scenarios and assigning bugs to developers from all the teams when these bugs were found during testing.

Development Approach

We used JUnit5 and SpringBootTest for the test project. TectContainers was used to run services in Docker. During the build process, before running the tests, we parsed the YAML files. This resulted in the creation of all the required Groovy scripts in Zookeeper. We also added a configuration file with the settings for the third party services to the Application container.

Here is a review of the test framework architecture:

Configuration layer: all the Spring configurations for running the testing infrastructure.
Constants layer: classes with constant values used in the tests.
Object_builders layer: builder classes for test data. For example, it includes a builder for a JSON object sent to the Application during a request.
Services layer: classes for working with services such as different DAOs and clients.
Utils layer: Util classes.
YAML layer: classes for YAML files.

Technological Overview

The diagram below shows a description of how the Application works:

Description of the Status Manager operation

One distinctive feature of the Status Manager architecture is its flexible customization. Customers can independently configure the rules for enabling/disabling products and services, without needing to modify the core code of the service itself.

This is achieved by storing the rules description in Groovy scripts in Zookeeper. When a new script with rules is added, the Status Manager automatically uploads it into its memory.
Information about subscribers, enabled products, and their statuses is stored in the Couchbase NoSQL database.

When the Status Manager receives a subscriber change event through the asynchronous channel (RabbitMQ), it sends a REST request to enable a product for the subscriber. This is done based on the existing rules and the presence of the subscriber and their services in Couchbase. Afterward, the external service responsible for enabling products for the subscriber sends a request to the Status Manager. Based on this request, the enabled/disabled product is saved in Couchbase.

Description of the Loader operation

The Loader also offers a feature that enables flexible customization without modifying the core code. This customization is achieved by executing rules for downloading subscribers and services from external sources. These rules are described in Groovy scripts and stored in ZooKeeper. When a new script with rules is added, the Loader automatically saves it to its memory.
To start downloading subscribers and services, the user sends a REST request to the Loader. Depending on the existing rules, the Loader performs one of two possible operations:

Downloads specific subscribers from the external Oracle DB and saves them to its own Couchbase DB.
Performs a REST request to download the subscriber’s products from the external Products Downloading Service. Once the request is successfully executed, the Product Downloading Service begins sending the product data to Kafka. The Loader then saves this data to the Couchbase DB.

Result

The test framework was developed, configured and deployed to overcome the app’s complexities. To get full test coverage we created test scenarios for existing functionality even though the development was still in progress when we started the test automation work. We also did R&D to be able to automate services housed within Docker images.

Due to the need for knowledge transfer, we created a comprehensive knowledge base with instructional videos to streamline the process of creating new tests and expanding the framework capabilities.

We went beyond our scope of work, configured other services and communicated with distributed teams across different time zones. To top it all off, we did it all within the specified deadline.