Automation Framework Development from Scratch to 3000 Auto Tests Running
Duration: 5 years 4 months
Industries: Telecom Industry; Finances
Services: Test Automation Services; Manual Testing Services; Software Integration Services; Software Maintenance and Support Services
Software Categories and Types: Integration Solutions; Payment & Billing Systems
IT Architecture Paradigms and Approaches: Test Driven Development (TDD); Enterprise Integration Patterns; Software Frameworks Development; Architectural Patterns
Technical Expertise: Test Automation; Expandable Software Solutions (Plugin Systems)
Technologies: CI/CD Automation Servers; RabbitMQ; Apache Maven; Docker; TestContainers; SQL; Data Bases; JUnit / TestNG; Apache ZooKeeper; Project Management, Collaboration and Bug Tracking; Jira; Git; Various Utility Languages; Apache Groovy; Gradle; Java; Bash; BitBucket (Stash); TestRail; TeamCity; Test Management Tools; Spring Boot; JVM Languages; IDE; Build and Dependency Platforms and Tools; Backend; Couchbase; MobaXTerm; JSON; YAML; Containers and DevOps Tools; Apache Kafka; Postman; Spring; Oracle SQL Developer; Source Code Management (SCM); Virtual Machines; Test Automation; REST Assured; CI/CD and DevOps; Message Queues and Service Buses; Languages, Protocols, APIs, Network Tools; Network Tools; WireMock; vRA (vRealize Automation); Software Engineering and Management Tools; Confluence; Java Libraries; NoSQL DBs; Development Infrastructure and Tools; Lombok; DB Languages
Team size (4):
1 Project Manager
2 QA Automation
1 Backend
Project summary
The “Application for Managing Products and Services” (hereinafter referred to as the Application) works as part of a group of applications which ensure business processes of telecom companies related to enabling/disabling of services, informing customers, and charging subscribers for mobile operators.
The Application consists of 2 services:
The first service – Status Manager – receives information from adjacent services about changes related to subscribers and makes a decision about enabling/disabling certain products/services packages. For example, a subscriber has activated a tariff plan that provides the basic services package (calls, SMS) even in case of a high debt. In such a case, when the Status Manager receives the information that the subscriber’s balance has crossed the allowable debt limit, at which the services included in the current tariff must be disabled until the debt is paid, it will send a request to the adjacent subsystem to enable the basic service package for this subscriber.
The second service – Loader – performs a so-called “cold start” of the Status Manager. In order the Status Manager can start working with other third-party services, it is needed to load real data into the cache for it. This is what the Loader is used for. The Loader downloads subscribers, their current status and activated services with tariff plans from databases of other services in the system, and uploads them to the Application cache.
A description of how the Application works is shown in the diagram below:
Figure 1: A description of how the Application works
Description of the Status Manager operation
A distinctive feature of the Status Manager architecture is the possibility of flexible customization – a customer can independently configure the rules of products and services enabling/disabling without making changes to the core code of the service itself.
This is achieved by storing the rules description in Groovy scripts in Zookeeper. When a new script with rules is added, the Status Manager automatically uploads it into its memory.
Information about subscribers, enabled products and their statuses is stored in the Couchbase NoSQL database.
When the Status Manager receives a subscriber change event via the asynchronous channel (RabbitMQ), it sends a REST request to enable a product for the subscriber based on the existing rules and the presence of the subscriber and his services in Couchbase. After that, the external service responsible for enabling products for the subscriber sends a request to the Status Manager, based on which the enabled/disabled product is saved in Couchbase.
Description of the Loader operation
The Loader also has a feature that allows flexible customizing without making changes to the core code. Flexible customization is made by executing the rules for downloading subscribers and services from external sources, which in their turn are described in Groovy scripts and stored in Zookeper. When a new script with rules is added, the Loader automatically saves it to its memory.
To initiate the downloading of subscribers and services, the user makes a REST request to the Loader. Based on the existing rules the Loader performs one of the two possible operations:
Downloads certain subscribers from the external Oracle DB and saves them to its own Couchbase DB.
Performs a REST request for downloading of the subscriber’s products from the external Products Downloading Service. After successful execution of the request, the Product Downloading Service starts sending products data to Kafka, from which the Loader saves them to the Couchbase DB.
JazzTeam’s responsibilities
Ensuring project test automation from scratch.
The necessary requirements for testing were:
All the services must be run in Docker within one network.
To run the services and Groovy scripts, it was necessary to use YAML files, which are supplied to the customer during the release.
Third-party services of related teams must be closed by a Mock server.
Test framework development
For automated testing it was necessary to develop a test framework, which had all the necessary functions for creating automated tests. The test framework had to be able to work with all the services which the Service for Managing Products and Services is integrated with (Couchbase, Kafka, Oracle DB, Zookeeper, Mock Server, REST).
The test project itself was implemented based on JUnit5 and SpringBootTest. The TectContainers library was used to run services in Docker. During the build, before starting the tests the parsing of YAML files was implemented, as the result of which all necessary Groovy scripts were created in Zookeeper, and the configuration file with the settings of its connection to third-party services was added to the Application container.
The test framework architecture was implemented in the following way:
Figure 2: The test framework architecture
The configuration layer contains all Spring configurations responsible for running the testing infrastructure.
The constants layer contains classes with constant values used in the tests.
The object_builders layer contains builders classes for test data. E.g., a builder for creating a json, which is sent to the Application during a request.
The services layer contains classes for working with the services: different daos and clients.
The utils layer contains util classes.
The yaml layer contains classes for working with yaml files.
Comprehensive testing
Together with the teams implementing other services, it was necessary to perform comprehensive testing on the stands, the configuration of which was close to the customer’s environment.
The comprehensive testing included the following tasks:
Developing comprehensive test scenarios. The description of the scenarios was made in TestRail. Around a dozen scenarios have been described in total.
Consulting other teams about the work and configuration of the Status Manager.
Executing test scenarios and assigning bugs to developers from all the teams when these bugs were found during testing.
Test automation libraries: Junit5, TestContainers, SpringBootTest, Rest Assured.
Project features
At the beginning of the Test Automation work, the development was not yet complete and there was no full understanding of how the product should function.
It was required to perform the full cycle of the test framework development — beginning with requirements analysis and ending with the tests creation.
Our team performed the comprehensive testing, during which it was required to configure not only our service, but also all related ones. To do that, we had to establish communications with related teams, including those in other time zones.
All manual tests for the current functionality should have been automated (100% coverage).
Project results
The test framework was developed, configured and run.
All test scenarios for the current functionality were covered with tests.
The knowledge base in the form of videos on working with the test framework were created for further writing new tests and extending the test framework functionality.
Company’s achievements during the project
R&D for test automation of services located in Docker images was performed.
The task was successfully completed within a very tight time frame.
The seamless interaction with the team distributed across different time zones was established.
The “Application for Managing Products and Services” (hereinafter referred to as the Application) works as part of a group of applications which ensure business processes of telecom companies related to enabling/disabling of services, informing customers, and charging subscribers for mobile operators.
