Development of a solution for data migration and synchronization

Duration: 1 year
Services: Product Development, Backend Development, Custom Software Development, Software Integration Services
Technologies: JUnit / TestNG, Data Bases, Apache Maven, Swagger, Java, Spring Boot, Java Enterprise Edition (EE), MySQL, Java Message Service (JMS), Build and Dependency Platforms and Tools, Backend, Spring, Mockito, Development Infrastructure and Tools, Standards and Protocols, Test Automation, OAuth2, CI/CD and DevOps, Languages, Protocols, APIs, Network Tools, Spring Kafka, H2 DB
Business Domains: IT Industry
Software Categories and Types: Integration Solutions
IT Architecture Paradigms and Approaches: Enterprise Integration Patterns, Software Frameworks Development
Technical Expertise: Java Server Side Development, Databases
DevOps Expertise: CI/CD Basics

Team size (2):
1 Project Manager
1 Backend

Project summary

JazzTeam engineers participated in the implementation of a corporate data change capture solution. This solution allows to integrate changes in the customer’s legacy systems (mainframes) with external systems.

The solution was implemented using the Change Data Capture pattern, which provides the ability to synchronize data changes with external data stores or systems.

The main advantages of the approach we used:

Precise and efficient synchronization of data changes in real time.
Protection of the mainframe from high loads, which may occur due to a large number of queries to the database.
Operational synchronization of data of various types without the need to make changes to the core of the system being developed.
Ability to ignore irrelevant changes (such as mass updates, delete/insert of the same record).

Technologies

Java, Spring Boot, H2, Kafka, Maven, JMS, OAuth2, MySQL, Swagger, JUnit, Mockito.

Data synchronization scheme with an option to configure Change Data Capture processes via API layer and Web UI

Project features

This project continues the series of the customer’s solutions that were created with the participation of JazzTeam engineers. By the way, each new product has more and more architectural complexity and non-triviality. Thus, in cooperation with the customer, our team has gone from creating simple products to developing complex corporate solutions.
The solution included a deep level of asynchrony, so JazzTeam engineers were required to have excellent knowledge of Kafka and JMS technologies.
The need to implement configuration and control of the solution via a web interface and an API layer with the ability to deploy the solution in any ecosystem where different types of mainframes are used.
Provision of customized configuration of the solution for the unique requirements of the client.
Preparation of the solution for the licensing process.

Project results

An API and auxiliary components (such as logging) were created to implement the Change Data Capture pattern in the solution being developed. Additionally, an external module was created to log transactions and configure the error handling process.
We also created a reusable component that can be used to interface with other targeted systems and mainframes.

Company’s achievements on the project

In a short time, JazzTeam engineers immersed themselves in the specifics of the customer’s vision, comprehended the architectural challenges and were able to fully implement the customer’s wishes.
JazzTeam professionals seamlessly integrated into a multinational development team and efficiently solved the tasks that were part of a large and complex software architecture.
During the work, our team followed the best practices for applying the Change Data Capture architectural pattern.