Project summary
The customer has an application for modeling the behavior of a physical object under specified conditions. Work on its creation started in 2013, and quite quickly a self-sufficient product was created, which has its own niche in the market. However, the product itself is extremely complex, since we are talking about modeling the behavior of the physical environment through the numerical solution of differential equations.
A project manager and a QA engineer participated in this project on the part of JazzTeam. The main customer’s pain points were problems with a large amount of accumulated technical debt and ineffective development processes. The following organizational and technical changes were the goal of the project manager:
- Scrum implementation;
- minimizing team resistance to innovations;
- team building;
- communication optimization;
- building a business analysis process;
- building a testing process;
- building a process of delivery to the end customer.
Testing processes were introduced, a test plan was created and a culture of quality product testing was instilled thanks to the work of the QA engineer.
By the time JazzTeam joining this project, the state of affairs on the project was as follows:
- Highly coherent code in two languages: C++ and C#. The calculation part is written in C++, the UI layer and business logic of the application — in C#. The architecture of the application did not meet the current needs of the customer for the further development of the product.
- The product is a desktop solution, therefore, protection that uses USB hardware keys is created before delivery. Each distribution is unique and is supplied for a specific workplace of a specific customer.
- The process of creating a distribution was partially automated in the project – hardware protection of software was automatically applied. All other processes – creating a license, defining parameters, forming a finished distribution – were carried out manually. The process of automated tests run was not automated. Less than 10% of the code was covered with Unit tests.
- Testing was carried out irregularly and there was no test management.
- The customer could not be sure of every delivery, so releases were carried out once a year and with great delay. Due to a large number of bugs, delivery could be delayed by more than 4 months.
- There were communication problems within the team caused by a large number of unresolved technical and organizational problems. The state of learned helplessness.
Our efforts were aimed at solving problems in the following areas:
- Scrum implementation.
- Process of setting technical requirements.
- CI/CD implementation.
- Manual testing.
- Automated testing of main components.
- Risk management.
- Release management.
So, the project consists of two parts – consulting in the form of management transformation and direct management of engineering application development.
Scrum implementation
Scrum implementation started with the introduction of stand-ups for team synchronization. It should be noted here that the project team is initially cross-functional, since researchers who perform the functions of business analysts for developers, developers and administrative staff work in a fairly close connection. At the time we joined the project, the product testing was performed by researchers.
The first stand-ups in a team of 9 people lasted about 40 minutes. They took too much time, since it was affected by large technical debt, tension in the team, insufficient elaboration of technical specifications and unclear prioritization. 9 months later a stand-up in the team of 11 people lasted 20-25 minutes already.
A physical board with cards was introduced almost simultaneously with stand-ups. This was because the process of working with tasks in Jira was very fragmented. In fact, Jira was primarily an information storage system, and it was completely impossible to track the specific status of the task, who was performing it and what exactly was being performed.
The physical board made it possible to determine the status of those tasks on which work is being carried out here and now. The prioritization of tasks also became clear. Since the stand-up is held directly at the board, this made it possible to ensure its relevance and effectiveness of use. In particular, thanks to it we were able to do the following:
- Easily track exactly what tasks each specific developer performs.
- Prioritize them.
- Ensure the complete implementation of the task, i.e. not just develop, but also quickly go through the review and testing stages.
- Estimate tasks in general and what exactly will be done during a day.
After that the iterative development approach was introduced. A two-week sprint duration was chosen as the most optimal for the project. A serious difficulty in implementing the iterative approach was that in the company’s culture tasks in Jira were not sufficiently decomposed and worked out in terms of business analysis.
It was not enough just to put the task in the sprint, as it is almost guaranteed that it will not be finished during the sprint. Therefore, at the initial stage tasks with a mandatory comment – what exactly will be done on this task within the sprint – were added to the sprint. Six months later, immediately after the next release, we moved on to the next stage, when the Jira project was reorganized so that in one board we go through the BA stage for the task, then we create sub-tasks and already within the Scrum board in Jira we carry out these sub-tasks in the sprint to full readiness.
Holding of retrospectives should be noted separately. This made it possible to significantly reduce tension in the team and transfer communication into constructive activity. Retrospectives were carried out in two formats – general for the entire team without a given topic and private for a part of the team. For example, a retrospective on refactoring with developers.
As a result, all planned organizational and technical changes were introduced during the year. It should be noted separately that since the start of the changes all key specialists of the customer’s team have been working on the project.
CI/CD implementation
As part of CI system the main goal was to create daily automated quality control through Unit and UI testing. Data driven tests were actively used in Unit testing, which is especially important for a modeling system with a large number of calculation modules.
In UI testing the main task was a full-fledged regular testing of critical functions in the distribution with already applied hardware and software protection. This approach allowed us to change the current product code with almost no difficulties, since the risk of missing regression bugs were significantly reduced. These processes were described in more detail in another article.
The major achievement was the introduction of automated testing of ready-made copies of software. According to preliminary estimates, this achievement alone saved more than 200 working hours during the first year since the introduction of this practice.
Manual testing
The quality assurance process on the project was not properly organized. The application was tested by business analysts, since, in the customer’s opinion, QA engineers could not have sufficient knowledge of the subject area for full-featured testing.
As part of this project, we created a Quality Assurance department with a dedicated manual QA engineer. The test management process was established, when, at the start of work on a task, the process of knowledge (on the selected functionality) transferring from a business analyst to a manual QA engineer began simultaneously with development, which made it possible to deeply analyze possible risks and problems with it. At the same time, they began deeper test software on the project. This process was especially helpful in finding regression bugs that were often omitted in error by business analysts.
Risk management
The risk assessment wasn’t carried out on the project for a long time, which led to the fact that from time to time the finished product had critical bugs and vulnerabilities. Therefore, we implemented the risk assessment methodology on the project. At the stage of requirements elaboration, the whole team assesses possible risks for some tasks and works out measures in order to process them.
Release management
The release process was not regulated in any way and was carried out randomly. This approach resulted in regular crashes during the release process. As part of this activity, we established release processes with the elaboration of the following stages:
- Codefreeze implementation – stopping the development of new functionality two weeks before release.
- Pre-release regression testing.
- Preparation of supporting documentation.
- Creation and tracking of the customers’ delivery plan.
Technologies
Stack: .NET, C#, C++, Managed C++.
Infrastructure: Windows, Jenkins, TestLink.
Test Automation libraries: Appium, White.
Screenshots
Project features
- The project combines consulting on management transformation and work on current development tasks.
- The established culture of development and business analysis that did not match the current needs of the company. The company faced new challenges that required a significant increase in software development productivity.
- The current state of testing processes did not provide the expected level of product quality. The customer understood that changes were needed in this area.
- It is a high-tech product, so it is very important to establish interaction between scientists and engineers on the project.
- Complex product delivery system. Formation of an individual copy of the program for each separate hardware protection key. Labor intensity in testing, since due to the peculiarities of hardware protection, it was required to test each instance of the application before delivery.
Project results
- The speed of product development and delivery increased by more than twice.
- The automated testing process was introduced from scratch.
- CI/CD was implemented from scratch.
- Test management was implemented.
- We significantly reduced the complexity of bug fixing operations and delivery of new application instances. During the first year from the start of management transformation more than 300 working hours of work were saved through organizational and technical measures.
- We significantly reduced the impact of the human factor with the help of delivery automation and daily automated testing.
Company’s achievements during the project
- Successful organizational transformation by involving the project manager in the customer’s team. Achievement of all planned goals and psychological atmosphere improvement in the customer’s team.
- Overcoming the state of learned helplessness inherent in the customer’s developers.
- After reaching the primary goals, the project did not end. Cooperation continues at the time of writing this article in the format of a long-term partnership.
- The significant increase in the development speed. The increase in the frequency of deliveries by 2-3 times.
- Implementation of test management processes. The Quality Assurance process took to another level.
- Implementation of the testing process and Continuous Integration on the desktop application with hardware protection.
- Achieving the involvement of the customer’s team in writing UI and Unit tests. The team accepted UI and Unit tests as a prerequisite for high-quality development.
- Increase from 0 to 55.000+ data lines in data-driven testing during the first year.
Feedback from the customer CEO
When you see a significant leap in the development team’s results, you realize that the practices implemented in your company really work. Speed, transparency of processes, quality improvement are no longer just words. And because of these changes, as the CEO of my company, I can now focus on the strategic goals rather than dealing with the same routine issues every day. I highly recommend JazzTeam as a company that can help you to achieve really big results with small, targeted changes.