A software product company, USA (California).
Complex multi-module system in the field of telecommunications.
The customer is engaged in the development and delivery of an IT solution for large telecom companies. Due to the specifics of the domain area, the product has a non-trivial delivery scheme, for example, in the most complex topology, it is deployed on at least 11 different servers.
The customer needs to organize automatic deployment of the system on the servers of customer companies. At the same time, various features should be taken into account: each customer company has different configurations of equipment for storing a large amount of data. In addition, it is necessary to ensure high-quality operation of the application using automated testing.
- A sophisticated product that includes an emulator of various devices at the TCP IP stack level.
- The application has 6 deployment options. For example, one option involves deploying different application components on 11 machines that are united into a cluster.
- The process of compiling and building a product can take several hours.
- Using a special distribution of the operating system with a focus on minimalism, fault tolerance, and security.
- The main function of the application is to work with the network traffic. A special tool was used for automatic tests that generated traffic on the network.
- In the process of implementing the task, our team gained experience in implementing CI/CD in one of the most technologically challenging solutions we have encountered in our practice. It was necessary to develop our own utilities for implementing CI/CD, as well as a separate web application for high-level management of deployment configurations.
Approaches and solutions
- Improvement of the CI process on the project. In order to continuously deploy and configure the product, our team carried out the following activities:
- Creating a Configuration Builder application to visualize and configure the deployment process. Our team created an application with a special menu, which conveniently presents all possible options and settings for deployment. The settings are selected from the drop-down lists or set manually. Based on the rules selected in the menu, the application generates a configuration yaml file. It specifies the version, server, and conditions for deployment. This yaml file is read by another application component based on Ansible scripts. So, the yaml file is synchronized with the pipeline. Then, the rules initially specified by the user will be automatically taken into account in the customer’s CI/CD process, which will significantly simplify the delivery of new versions of the product.
- Creating automated tests. JazzTeam specialists prepared combined Selenium + CLI autotests, which are run for each new build every night.
- The continuous integration process was configured and implemented. The customer’s team was trained, and documents on the use and administration of the CI process on the project were created.
- 30+ test environments (>100 virtual machines) were configured for testing.
- Universal solutions within CI Jenkins were implemented. They minimize manual activities for CI administration and application deployment.
- All 6 deployment options became automated with the help of the deployment process configuration solution created by our team.
- The application build process was improved. The possibility to collect several versions at the same time was implemented. Also, the patch build for the earlier versions of the application became fully automated.