Manual testing of a complex product in the area of the Internet of Things (IoT)

Customer

IoT data service provider, which hasits own network of base stations located in all regions of Russia.

Product

A service for interaction with acloud-based IoT platform designed for data collection and energymetering devices management. With its help, the user can receive datafrom existing devices, configure their operation, and connect newdevices.

Business challenges

After a negative experience whilecooperating with an outsourcing company, the customer wasskeptical about manual testing services. It did not support theidea of implementing this activity on the project, although theirneeds became very acute and critical at a certain point.
Thefunctionality of the product was constantly expanded and thesystem was unstable: there were always bugs in production, dueto which the customer regularly received negative feedback from endusers.

JazzTeam had already collaborated withthis company within the framework of another type of service, and hadpresented itself in the best possible light. Therefore, we proposed organizing the quality assurance process on the project in order todispel the customer’s doubts about this type of service.

Technological features of the project

Complex IoT domain area that required the team to develop specific knowledge.
To ensure quality, it was necessary to clearly understand what was happening on the system backend, how the data was converted, and what behavior to expect from the system. Therefore, a test engineer was required who could get a deeper understanding of the system from a technical perspective, and also learn the principles of the necessary protocols and standards.
Working in a distributed team environment.

Manual testing of a complex product in the area of the Internet of Things (IoT)

Approaches and solutions

Involving a JazzTeam test engineer in the project for a trial period (3 months, 4 hours a day). To dispel the customer’s doubts, our company arranged a trial period for the QA specialist on the project. It was agreed that the cooperation would continue only if the results of the test engineer’s work satisfied the customer.
Test engineer’s immersion in the context of the project. This included communication with the team, participation in all activities, as well as the study of available project documentation.
Preparation of testing checklists to test the work of the product’s basic functionality.
Creating test cases. On the basis of the prepared checklists, decomposition was carried out and test cases were created for more detailed checks, as well as for regression, integration, and smoke testing.
Implementation of a test management system. TestLink testing process control system was introduced on the project.
Establishing the procedure of new functionality testing on the project. After the developers place new functionality on the test environment, the test engineer performs regression testing. Smoke testing is carried out immediately after launching for production.
Preparation of a detailed test plan for manual testing. It was the basis for the creation of an automation plan. The plan greatly simplified the immersion of the new test automation specialist in the project.

Results and achievements

All the functionality of the developed service was covered by test cases. As a result, the number of bugs in the product was reduced by several times.
The effective testing system allowed for the elimination of bugs during production.
Long-term cooperation with the customer was established. The trial period for the provision of manual testing services was successful. The cooperation continued, and the time of the test engineer’s work on the project increased to 8 hours a day.