Implementation of CI/CD on the Existing Project Without Outside Help. Part 2: Implementation

In this article, we will consider the actual realization of CI/CD implementation process on the project with a long history. You can read about the initial stage of the implementation process here.

When project documentation for CI/CD processes was ready, we just had to implement the planned changes. As you remember from the first part, within the framework of this project we planned to implement CI/CD by the development team only, without a DevOps engineer involved. The project had established processes, which were difficult to automate, so it was the main challenge of implementation.

The previous article describes the stages of initiating and planning a project, and this article will describe the stages of executing and completing it.

It should also be noted that the process of introducing changes was step-by-step – we deeply decomposed the implementation tasks and, step by step, incrementally, implemented CI/CD.

It is important to note that the introduction of new functionality, especially if the initiative does not come from engineers, is often accompanied by a lack of motivation and learned helplessness. Therefore, the organization of work with the team was an important issue. Each engineer had his own dedicated tasks and expectations related to the results of their execution.

The focus was often lost in the process and there were deviations from the plan, therefore, in this context leadership on the project is very important, which, on the one hand, maintains motivation, and on the other hand, does not allow  deviating greatly from the intended goals.

The CI/CD implementation process was divided into the following steps:

After all these activities, we were able to apply full GUI autotests to isolated functionality as part of the work on GitFlow. This allowed us to ensure development with a significantly smaller number of bugs, and it is now much easier to localize the remaining ones.

Among other things, this opened the door for us to containerization and more reliable ways of operating.

The primary condition for the introduction of automation was the introduction of an automated build system of the application. The application manual build system had long been existing on the project, so it was a challenging issue. There was a special utility that prepared the data for the build. The developer used this utility to create special template files. After that, he used IDE to create a working build of the application with these files.

As part of automation, we implemented the Maven application automated build system, which wrapped the use of this utility and allowed us to make application building fully automated.

As part of this stage, Unit testing of the finished application was also added.

The core component of a CI/CD system is, of course, the orchestrator. Within the framework of our project, we used Jenkins, as its functionality fully met our needs. Jenkins was installed on the main test server and, step by step, we began to implement CI/CD in it. The final result can be seen in the screenshot below.

As part of this step, we automated the deployment of the frontend application on IIS server. Since the frontend of the application is rarely updated on this project, we did not consider the automated build of this application within the framework of the CI/CD system. The frontend of the application was deployed to the stopped server, after which the server was started.

At this step, we automated the deployment of the backend application to the Tomcat server. The following functionality was added to Jenkins:

We used Groovy scripts in Jenkins. This allowed us to reuse the code and settings, which made it possible to greatly simplify the works for the support of the entire system in the future.

We apply these practices at all times on all our projects and we can proudly say that we have accumulated a lot of experience in proper setting up and using Jenkins.

Customization for specific customers is a separate component of the system that allows changing the functionality of the application for individual users. The complexity of automating this system lies in the fact that customization can be performed within a very wide range and, in fact, is a way to integrate the application into the customer’s business system.

Accordingly, this system is highly flexible, so it was not a trivial task to implement automation. Possible changes affected a lot of very different systems, required control over the files of specific users and a deep study of the update process.

In our case, a separate engineer was responsible for this area of work, who performed a full cycle of research work for this system – from writing requirements and the sequence diagram to implementing a ready-made solution. As part of this activity, we initially made a proof of concept, then finalized the application architecture, and implemented deployment automation.

The project uses an SQL database as a DBMS. In this case, a separate instance of the database is created for each customer, which is based on the same diagram. The database is updated using Liquibase technology.

Jenkins in the main pipeline checks whether it is necessary to update the databases and then updates each instance.
This led to very non-standard solutions because the databases had to be updated in at least 3 different ways:

This resulted in the fact that the process of approbation and testing of all the necessary functionality lasted more than six months. In the process, a few different launch methods were applied – through Jenkins and through a bash script. In the end we came to a conclusion that Jenkins works directly with the SDK Liquibase.

A separate, but very important element of a CI/CD system is automated testing. We implemented it in the following way. With each build of the application, all Unit tests are run.

At the same time, GUI autotests from the master branch are run every night on the test bench, and they completely check the current instance for possible regression.

When all tests are completed, a report is generated, which is analyzed by a QA Automation engineer in the morning.

When all the work on automating deployment and testing was completed, we started trial operation on a test server. The test period lasted two months, during which we tested the operation of all elements of the system.

At the end of the test period, we began to use this automation on the Live server. This allowed us to automate the work of developers, thus reducing the likelihood of human error when deploying to a Live server.

As you can see, the CI/CD implementation process was relatively easy, primarily due to our careful preparation. Planning the work and writing high-quality documentation allowed us to solve the problems of implementing CI/CD by the team itself.

As a final result, the frequency of delivery increased by 4-5 times, the reliability of the application became significantly higher, and the number of requests from users decreased.

The development and deployment process became much more transparent, now stakeholders can observe the delivery process themselves and even deploy the required test version of the application to their test bench.