Implementing a testing process for a large ERP system from scratch

• It was necessary to build testing processes from scratch and transfer knowledge to the customer’s team so that in the future they could cope with this activity on their own.
• Tight project timeline: it was necessary to establish testing processes within 3.5 months.
• The team of consultants constantly analyzed the implemented changes and, if necessary, plans were changed.
• The product owners, technical leader, and project team provided active support to the consultants, which helped to achieve results on time.
• The testing specialist of JazzTeam consultants had knowledge of the subject area: he understood the processes and needs of large manufacturing companies.

Approaches and solutions

Stage 1: project audit

To conduct a detailed analysis, JazzTeam professionals requested all the necessary information about testing on the project. Among other things, JazzTeam specialists gained access to the documentation (user stories, tasks setting), the bug tracking system used by the customer, and the product codebase. The audit process also included communication between JazzTeam specialists and the product owners, the company’s technical leader, and its business analyst.

During the analysis, our team got a full picture of how the testing process was organized on the project, what difficulties the customer’s team faced, and what was the main pain of the customer. We also learned how and within what timeframes defects on the project were found, registered and processed. The most important factor in this process was the high level of trust of the founders and the team to JazzTeam specialists. The customer was confident in our expertise, we received assistance, support and saw the team’s desire to change the situation on the project.

After conducting a thorough analysis on the project, the following gaps in the testing process were identified:

• Tests for the backend part of the system were developed, but were not used.
• There were no Unit tests on the frontend side.
• There was no testing phase in development.
• Test automation was not implemented.
• The continuous integration (CI) process was not implemented.

Stage 2: Testing strategy development

After identifying the main problems on the project, our team began to develop a testing strategy. One of the features of an ERP system is a large number of elements having similar functionality. Therefore, it was necessary to create autotests to test such pages. It was decided to check business cases that are difficult to reproduce and implement using manual testing.

The next step was to write test cases and test plans for manual testing. The work was clearly phased, each stage was planned and agreed with the customer and was limited by the deadline.

Stage 3: manual testing

The functionality of the UI part of the ERP system provided for a large number of small operations (for example, create, edit an object card, delete, duplicate). Due to the UI high complexity, it was decided to cover the entire functionality of the application with test cases, including minor features of the pages. Our team created a basic and complete set of tests for smoke and regression testing in the amount of 60 and 158 tests respectively.

Also at this stage, JazzTeam manager conducted a number of consultations for the customer’s team related to the process of including testing in development. Transferring the culture of testing was an important task for us. It was necessary to convey to the team that quality assurance is a mandatory part of every sprint and the development process in general. Engineers and testing specialists should interact to fix bugs every day, not just before delivery to the customer when there is not enough time for testing. Attention to the testing process and its timely implementation allow to constantly maintain the high quality of the product and promptly correct defects in the software.

Stage 4: Automated testing development and implementation

At the next stage, our team created a plan for the implementation of autotests. It was agreed and approved by the customer. Further, JazzTeam consultants developed and implemented a flow of tasks for new functionality automated testing, as well as retesting of fixed defects (regression testing). As part of automated testing, the following activities were implemented:

• Jenkins and TestLink servers were installed and launched, plugins were configured for the integration of results from Jenkins into TestLink.
• CI/CD pipelines were built (the scheme of the interaction order between jobs on Jenkins was worked out and applied).
• Autotest templates were created for subsequent reuse.
• Automated testing through TestLink was organized.
• Autotests were analyzed and subsequently updated.

Autotesting became a mandatory part of quality assurance at the customer’s company. JazzTeam specialists were able to prove the effectiveness of this approach to the project team. The consultants showed in practice how automation allows achieving the desired result and saving time and efforts. Understanding the value of autotests prompted the team to write and use them on a regular basis.

Stage 5: improving the release management process

Among other things, our team discovered that the project had difficulties with the release management process. We consulted on how to set up releases, conduct regression testing, and effectively manage development to improve the quality of the system.

JazzTeam specialists contributed to the improvement of the release management process on the project:

• Introduced an iterative approach to development planning, making it possible to draw up a release plan.
• Regression testing became an obligatory part of quality assurance: runs of existing autotests and analysis of their results followed by fixing the defects found were carried out on a regular basis.
• Prepared instructions for releasing.
• Created a checklist to help the person responsible for release delivery.

Stage 6: knowledge transfer to the customer’s team

In the final phase of the consulting work, our team documented testing processes at different levels: from specific instructions for team members to describing job algorithms for building a product on Jenkins.

Our specialists created a number of documents to help the customer’s team:

• checklists and test cases for manual testing;
• delivery (release) instructions;
• requirements for the design of test cases;
• a checklist to help the person responsible for release delivery;
• step-by-step system testing algorithm.

Subsequently, a QA specialist joined the customer’s team, to whom JazzTeam experts transferred their knowledge and best practices related to testing on the project. In particular, we conducted training on the established processes of manual and automated testing, conducting regression testing, competent processing and analysis of test results. We taught the team to correctly register defects in Redmain in order to fix and accept them according to the Definition of Done.

Company’s achievements on the project

In just 3.5 months of work on the project, we managed to achieve the following significant results:

• The processes of manual and automated testing in the company were successfully built from scratch. Now the quality check is carried out in a timely manner and by a dedicated QA engineer. The product quality improvement contributed to the advancement of the customer’s reputation.
• A detailed plan for system testing automation was developed and implemented, all bottlenecks are covered with autotests, which will allow the customer to save on testing without loss in product quality.
• A continuous integration (CI) process was implemented and launched, which helps detect software bugs regularly in a timely manner . Early detection of bugs can reduce the cost of fixing them.
• Test management system (TestLink) was launched and integrated with Jenkins. It allowed the customer to track and store all test results in one place, and eliminate the need to transfer test results from Jenkins to TestLink for each build. In addition, the use of TestLink allows generating convenient reports that can be attached to the delivery as a document confirming the high quality of the software.
• Instructions and recommendations or product testing and defect registration were developed and implemented.
• Effective regular deliveries of the product were established.