Implementing a testing process for a large ERP system from scratch

IT Consulting, Test Automation

A consulting and IT company that focuses exclusively on solving problems in planning and managing production and supply.

An IT product company develops a fast-growing ERP system for accounting and cost planning at large manufacturing enterprises in the field of mechanical engineering. The functionality of the IT solution constantly changes, but the company does not have enough resources for testing. The quality assurance process is not established; the project does not have the necessary specialists. Bugs are fixed only in the product lifecycle late stage as part of acceptance testing. It is too expensive for the customer and can negatively affect their reputation. The team is unable to independently ensure the high quality of the system by the date of delivery to the end customer.

The customer’s company had no systematic and clear testing process, when they contacted JazzTeam specialists for advice. The application quality control was carried out just before the integration specialists sent the product and only with the help of manual testing, without extremely important and new functionality.

There was no dedicated testing team on the project: a business analyst was involved in product quality control. He had not enough competence and time to ensure high system quality and carry out testing on time. At the same time, the solution was put into production long ago and was actively supplied to end users.

Also, the product owners had extensive plans for the development of the system and increasing the functionality. Therefore, the issue of quality assurance in the company was extremely acute.

JazzTeam consultants (manager and QA specialist) joined the customer’s project to establish, formalize and debug testing processes, introduce a testing culture, and transfer best automation practices. Our team proposed the following service plan:

PROJECT GOALS

To develop the product testing strategy, create the necessary test plans and test cases

To build processes and transfer best testing practices to the customer's team

SOLUTION STAGES

• 1
  Project audit Analysis of the situation on the project, identification of the testing problems that the customer's team has
• 2
  Development of the testing strategy Determination of the functionality that needs to be covered by autotests Identification of the most complex and hard-to-reproduce business cases for manual testing
• 3
  Manual testing Creation of basic and full test suites for smoke and regression testing Transfer of best testing practices to the customer's team
• 4
  Development and implementation of Test Automation Plan creation for Test Automation implementation: Installing and running Jenkins servers and TestLink, configuring plugins for results integration Setting up CI/CD pipelines Creating automated test templates for further reuse Organization of Test Automation through TestLink
• 5
  Improving the process of release management Implementing an iterative approach to planning Implementing mandatory regression testing
• 6
  Transfer of knowledge to the customer's team Documentation of testing processes at different levels in the form of instructions, checklists, and step-by-step work algorithms

RESULTS

• 
  Manual and automated testing processes built from scratch
• 
  The detailed plan for Test Automation of the system was developed and implemented
• 
  Continuous Integration (CI) process and test management system (TestLink) were implemented
• 
  Regular and high-quality deliveries of the product to end users are ensured

Implementing a testing process for a large ERP system from scratch

To conduct a detailed analysis, JazzTeam professionals requested all the necessary information about testing on the project. Among other things, JazzTeam specialists gained access to the documentation (user stories, tasks setting), the bug tracking system used by the customer, and the product codebase. The audit process also included communication between JazzTeam specialists and the product owners, the company’s technical leader, and its business analyst.

During the analysis, our team got a full picture of how the testing process was organized on the project, what difficulties the customer’s team faced, and what was the main pain of the customer. We also learned how and within what timeframes defects on the project were found, registered and processed. The most important factor in this process was the high level of trust of the founders and the team to JazzTeam specialists. The customer was confident in our expertise, we received assistance, support and saw the team’s desire to change the situation on the project.

After conducting a thorough analysis on the project, the following gaps in the testing process were identified:

After identifying the main problems on the project, our team began to develop a testing strategy. One of the features of an ERP system is a large number of elements having similar functionality. Therefore, it was necessary to create autotests to test such pages. It was decided to check business cases that are difficult to reproduce and implement using manual testing.

The next step was to write test cases and test plans for manual testing. The work was clearly phased, each stage was planned and agreed with the customer and was limited by the deadline.

The functionality of the UI part of the ERP system provided for a large number of small operations (for example, create, edit an object card, delete, duplicate). Due to the UI high complexity, it was decided to cover the entire functionality of the application with test cases, including minor features of the pages. Our team created a basic and complete set of tests for smoke and regression testing in the amount of 60 and 158 tests respectively.

Also at this stage, JazzTeam manager conducted a number of consultations for the customer’s team related to the process of including testing in development. Transferring the culture of testing was an important task for us. It was necessary to convey to the team that quality assurance is a mandatory part of every sprint and the development process in general. Engineers and testing specialists should interact to fix bugs every day, not just before delivery to the customer when there is not enough time for testing. Attention to the testing process and its timely implementation allow to constantly maintain the high quality of the product and promptly correct defects in the software.

At the next stage, our team created a plan for the implementation of autotests. It was agreed and approved by the customer. Further, JazzTeam consultants developed and implemented a flow of tasks for new functionality automated testing, as well as retesting of fixed defects (regression testing). As part of automated testing, the following activities were implemented:

Autotesting became a mandatory part of quality assurance at the customer’s company. JazzTeam specialists were able to prove the effectiveness of this approach to the project team. The consultants showed in practice how automation allows achieving the desired result and saving time and efforts. Understanding the value of autotests prompted the team to write and use them on a regular basis.

Among other things, our team discovered that the project had difficulties with the release management process. We consulted on how to set up releases, conduct regression testing, and effectively manage development to improve the quality of the system.

JazzTeam specialists contributed to the improvement of the release management process on the project:

In the final phase of the consulting work, our team documented testing processes at different levels: from specific instructions for team members to describing job algorithms for building a product on Jenkins.

Our specialists created a number of documents to help the customer’s team:

Subsequently, a QA specialist joined the customer’s team, to whom JazzTeam experts transferred their knowledge and best practices related to testing on the project. In particular, we conducted training on the established processes of manual and automated testing, conducting regression testing, competent processing and analysis of test results. We taught the team to correctly register defects in Redmain in order to fix and accept them according to the Definition of Done.

In just 3.5 months of work on the project, we managed to achieve the following significant results: