Empowering Telecom Company with Financial Analysis / Decision-Making System

Product Development, Backend Development, Test Automation Services, Project Management, Software Integration Services, Software Maintenance and Support Services

The client is a US-based telecom enterprise with a team of over 200 employees and a rich history spanning over three decades. They specialize in developing advanced enterprise solutions for various industries, with their products aimed at facilitating business transformation. This is achieved by aligning processes, creating digital ecosystems for employees, and improving IT departments’ efficiency.

The product we worked on is a Multifunctional Financial System (MFS), a part of the client’s broader finance and billing system designed for telecom companies, mobile operators, and other large customers in the telecommunications industry.

The product analyzes financial data and customer information to help make decisions on account management, service notifications, and subscriber lifecycle control. It enables quick access to information and allows interaction with external systems to perform tasks such as connecting or disconnecting services or sending notifications to customers.

In the implemented configuration (which can be expanded), MFS could handle the following tasks:

The client faced a challenge in stabilizing their complex, innovative product built with multiple technologies. Lacking the necessary expertise in-house, the client was looking for a reliable tech partner.

They needed someone proficient in stabilizing complex systems through automation testing and the reworking of integration autotests, who would ensure the quality and stability of the product and be able to conduct research and development.

The client also wanted to work in an agile manner using short iterations, so their potential partner was expected to be able to lead the process and convey the best agile methodologies practices.

With the necessary experience and knowledge, JazzTeam met all of the client’s requirements and was chosen to address the client’s challenges.

JazzTeam diligently worked in three directions — development, testing, and cybersecurity.

During the work on the project, our QA team created more than 3000 autotests and executed a range of activities to ensure the reliability and functionality of the product. The team grew from a single test engineer at the beginning to a group of seven specialists over time.

We began by creating automated tests for new functionalities:

We created test cases, suites, and test runs in TestRail. We also updated and refactored autotests on previously created functionality to maintain efficiency and accuracy.

Testing activities encompassed a diverse range of tasks, including running autotests and analyzing the results after the run. Any defects identified were promptly recorded in JIRA. Manual tests were conducted where autotests were not feasible.

We also conducted integration testing of the product alongside other products on the clones of the customer’s industrial stands. In addition, we meticulously crafted testing product documentation, as well as functional and technical specifications for completeness, accuracy, uniqueness, and consistency. Finally, we controlled the product delivery process to the client.

We developed technical specifications in alignment with functional specifications. Our team closely collaborated with the client representatives (business analyst, representative of the product owner) and directly with the client. Throughout the development phase, we maintained active and regular communication with the technical support team, analyzing issues, and brainstorming solutions to address them effectively.

A significant aspect of our work targeted the implementation of internal business logic in the form of open and accessible for Drools rules changes. Additionally, we worked with the Oracle remote database, downloading data from Oracle to Couchbase local NoSQL store. We also integrated many related products using the RabbitMQ message broker, allowing for seamless communication and event processing across interconnected systems.

Our efforts also involved leveraging Akka to construct chains of actors (or flows), each dedicated to specific tasks within the system, and utilizing Zookeeper to store and manage the environment’s configuration. In addition to our technical endeavors, we actively participated in grooming sessions, meetings, and architectural discussions related to the product.

Given the product’s role in storing our clients’ customers’ financial data, we placed special emphasis on developing a solution resistant to hacking attempts.

This experience was successfully applied within JazzTeam. We introduced methodologies, checklists, and guidelines for developing secure solutions. Additionally, we compiled a set of basic information security literature for our developers.

We consistently consider common problems and risks related to information security. At the corporate level, our company has implemented principles and standards for secure development, addressing SQL injection, cross-site scripting, and other system-level vulnerabilities.

Our team’s engagement on this multi-component project spanned over two years. During this time, we adhered to a flexible development approach guided by the Scrum methodology. We focused on creating a Potentially Shippable Product Increment and our team functioned in the format of a Feature team. This involved a clear separation of roles between a scrum master and a team lead, and planning was facilitated using planning poker.

We carried out grooming during the analysis of new functionality and held daily status meetings to keep everyone updated. Demonstrations of our work were showcased on Demo days and during Sprint reviews. We adopted an iterative approach with work conducted in two-week sprints, with regular retrospectives held at the completion of each sprint, or in case of emergencies and problems. Preparation and shipment of patches/releases were done in accordance with regulations.

We placed a high emphasis on quality, enforcing mandatory code cross-review and compliance with Code style. The release engineer set high standards for the team regarding the quality of the delivered product, especially when preparing and delivering releases to the customer. All new functionality was required to be covered by test cases. We worked collaboratively across geographically distributed teams and divisions of the customer.

The client’s product, Multifunctional Financial System, involves three applications:

Here’s how it works:

MFS receives RabbitMQ messages from multiple senders in a large billing system.

Information from external senders is analyzed using business rules written in the Drools framework.

Then, based on the business logic, the MFS begins managing the subscribers’ features/services. It also sends requests for the creation or removal of various notifications, such as SMS messages, emails, faxes, and phone calls.

After making decisions, MFS sends RabbitMQ response messages to the corresponding external products.

In the next stage, the product saves the changed business objects (subscribers) to the local NoSQL Couchbase database for quick access.

Additionally, MFS offers a special tool for restoring or updating the NoSQL database. This tool can be particularly useful when starting in a new environment or restoring the database after an incident.

Alexander Zolotarev

Head of Laboratory, Billing and Finance, Nexign JSC (earlier Peter-Service Inc), Software Development Company for the Telecommunications Industry

The customer testimonial about project: The system of accounting, analysis and decision-making on the financial condition of customers and subscribers

"We have been working with JazzTeam company for many years. During this time, the JazzTeam engineers have become for us not only exceptional developers and QA engineers, but also true partners and friends.

Their ability to identify hidden problems and eliminate them quickly and efficiently. In my opinion, this is a natural component of any project in which JazzTeam participates.

Stable and confident work of the team, reasoned actions, clearly posed questions - all that makes JazzTeam the professionals in their field.

I particularly wish to commend the management team, which has set a high level of responsibility from the very beginning: they always not only listen to our requirements, but also hear them, and implemented everything exactly in the form that we need and that is the most important thing.

Cooperation with JazzTeam is a pleasure, strong friendship and gained boundless trust, that was fully gained by the guys’ work. I sincerely believe that our effective cooperation will be continued. I want to thank every member of the team and wish them good luck and further professional success!"

At this stage, the development of a new functionality is in progress, and support for the current functionality is being implemented. Efforts are also underway to customize the solution for other customers including mobile operators. The product operates 24/7, managing databases that store data on tens of millions of unique subscribers.

Over 2 years, our team expanded from 1 engineer to 7 specialists and fulfilled its potential as a Feature team. We conducted a full cycle of iterative development, including daily standups, groomings, sprint reviews, retrospectives, and weekly client demos. We handled analysis, development, testing, and delivery of releases or patches, and implemented numerous new business cases.

Throughout the project, we created 3312 autotests and updated a large number of tests. All tests were stabilized during the optimization of regression testing.

Our engineers demonstrated responsibility and self-organization and learned new technologies like Drools, RobotFramework, and ELOG. They also acted as initiators of improvements in the project processes. Our team contributed to a new test architecture development and product distribution and joined a scrum masters community.

We performed product quality support at a high level by increasing code coverage of the code (Unit tests) and business cases (Autotests). We also conducted integration tests to verify the complex work with a set of related products.

Additionally, we developed a program to introduce new personnel into the project, ensuring a smooth onboarding process for new team members.

Java SE + JDBC, Spring, XML, JSON, etc., Robot Framework, PostgreSQL, Oracle, TestNG, Allure Framework, CI (Jenkins, TeamCity, VRA), Maven, Git.