Development of a service for the interaction with utility metering platforms

Project summary

The project involves the development of a service intended for interaction with utility metering platforms. This service processes readings from metering devices and provides remote access to them. The developed service allows the user of the platform (for example, the operator of the power supply service) to receive data from metering devices, find out the amount of energy consumed, monitor the dynamics of costs, remotely control equipment, and change the settings of metering devices.

Technical description of the project

The service is a RESTful system consisting of a set of microservices. Microservices are a layer between end users and metering devices. They allow receiving data, as well as controlling metering devices via HTTPS, DLMS and IEC-104 protocols.

All the data received from metering devices is stored in PostgreSQL and TimescaleDB databases. Redis is also used to temporarily store certain types of data.

A UI component was developed in parallel with the RESTful system. It allows users to interact with metering devices. It is worthwhile noting that not all users work with the UI component. Some of them use their own software, which directly interacts with REST API.

Also, for the convenience of development, when functionality is not yet available in the UI component, Swagger framework is used on the project. It allows using the functionality of the service without reference to the UI and helps speed up the development of the client application. Thanks to Swagger, developers, test engineers and technical support can see all API services, available methods, as well as additional useful data (for example, the response status, content type, list of parameters).

Metering devices can be used in various locations and conditions. It happens that for some reason a metering device may be unavailable (for example, the metering device is not connected to the network server at the time of the request) and data from this metering device is not transmitted to the system. A component that provides additional data collection was developed for such cases. The system records cases of missing data and from time to time sends requests to the metering device. At the moment when the communication is restored, the metering device receives the request from the system and gives it the data.

JazzTeam responsibilities

Implementation of the microservice architecture in accordance with the customer’s technical requirements.
Development of a UI subsystem and a module for additional data collection.
Implementation of services that interact with metering devices via DLMS and IEC-104 protocols.
Preparation and delivery of releases (release management).
Carrying out manual regression, integration and smoke testing of the system.

Technologies used

Programming languages: Java 11, JS, Kaitai, Kotlin

Infrastructure: Maven, Docker, Fabric8, TeamCity, Git, IntelliJ idea, Jira, Swagger, Grafana

Test libraries: Junit 4, Mockito

Other libraries: Spring, Hibernate, Kafka, Akka, Lombok

DB: PostgreSQL, TimescaleDB, Redis

Protocols: HTTPS, DLMS, IEC-104

Project features

Active participation of the team in experimental infrastructure technical support at any time of the day.
Active involvement of the product owners in the work of the team, which significantly increased the development speed.
Data processing and control of different types of metering devices with different software versions, which affected the architecture and complexity of the code.
The development was carried out by a geographically distributed team.

Project results

The MVP (Minimum Viable Product, a product with basic functionality) was put into trial operation within a short time. This was achieved through close coordination with customer representatives and a focus on high-priority tasks.
In parallel with the trial operation, JazzTeam significantly expanded the functionality of the product in accordance with the requirements of the customer, as well as the needs of end users.
We implemented monitoring of the state of services through the introduction of Health Checks, which significantly reduced downtime.
Automatic project build is set after each commit; it is accompanied by all Unit tests, so it is possible to maintain a stable code quality at a high level.
Coverage of the developed code base by Unit tests at the level of more than 90%.

Company’s achievements on the project

JazzTeam offered the customer and applied significant improvements during project implementation. For example, we ensured the stability of the system at high loads. In particular, the process of saving data when there is much incoming information was optimized. We also used data caching approaches, which reduced the load on services.
The team jointly with a JazzTeam manager proposed and organized the use of Jira on the project. Moreover, all project participants, including product owners, also implemented full-fledged.
Scrum with all its inherent attributes: task evaluation, sprint planning, daily scrum, regular demonstration of implemented functionality, retrospectives.
The process of manual regression, integration and smoke testing of the system was introduced.
JazzTeam specialists mastered new field-specific technologies.