Test Creation Speed Boosted with XML-Based Automation

Provided services

Product Development, Frontend Development, Manual Testing Services, Software Integration Services

Client

The European market leader for Telecom Management Technology, in the Netherlands.

Product

The XML2Selenium product is an extensible, plugin-based platform for creating and managing automated tests utilizing Java technology. XML2Selenium features integration with JUnit and operates on top of Selenium (which can be modified). It enables the creation of automated tests in a format that is simple and understandable even to QA engineers without programming skills. XML2Selenium also allows for the management of all stages of working with automated tests, from creation to test management. It addresses all key issues of test automation and offers an elegant and straightforward solution—using XML as the test scripting language. XML2Selenium is integrated with Maven, Jenkins, JUnit, Selenium, and BrowserStack.

Challenge

Our client tasked us with addressing several critical challenges to improve their automated testing processes:

Accessibility of Automated Testing for Non-Programmers. The client, with a diverse team of quality assurance professionals, many of whom lacked extensive programming knowledge, required a solution that would enable these professionals to effectively create and manage automated tests. This challenge involved designing a tool that was not only powerful but also intuitive and accessible to users without a technical background. The goal was to democratize the process of automated testing, making it feasible for non-programmers to participate in and contribute to the testing efforts.

Stability with Scalability. They also needed a scalable plugin architecture that could support the continuous addition of new features without compromising system stability. Creating a scalable plugin architecture, inspired by the Eclipse plugin system, posed a challenge for the team. Developing a plugin platform that could accommodate the necessary scalability for integrating new features while simultaneously ensuring a high level of stability presented a significant complexity: each new plugin could potentially disrupt the entire system, making it fragile and susceptible to errors. The continual addition of features increased the risk of system failures, necessitating the development of a comprehensive and reliable testing approach.

Ensuring Easy Integration of Automated Testing into Customer Production Processes. The client required the new framework to be seamlessly integrated with their existing development and testing systems of the future customers. The aim was for customers to seamlessly implement the automation testing process into their production workflows, thereby improving the speed and quality of their software development.

Ensuring Transparency and Traceability of Testing Results. A key task of our project was to create an effective reporting system that could automatically generate detailed reports upon completion of tests. These reports needed to be not only technically informative but also understandable to business users, to ensure a full understanding of the test results at all levels of management. The challenge was compounded by the need to eliminate errors in data and presentation of results.

Solution

We leveraged our existing ideas and innovations, aligning them with the client’s requirements to develop an effective solution. Our team already had several concepts and prototypes that perfectly matched the client’s needs, which significantly expedited the development process.

Enabling Automated Testing for Non-Programmers. To overcome this challenge, we developed a proprietary DSL (Domain-Specific Language) based on XML. This language was specifically tailored for testing needs, allowing QA engineers to easily describe test scenarios using XML structures, which are familiar due to their widespread use. At the time, this was an innovative solution that effectively integrated individuals without programming skills into the testing processes. One of the main challenges we faced was ensuring the DSL was intuitive yet powerful. We conducted numerous iterations and user testing sessions to refine the language, making it both accessible for non-programmers and robust enough for complex testing scenarios.

Our DSL supports all the necessary programming constructs, such as inheritance, function overriding, variable management, and conditional operators, making it not only a powerful tool for developers but also an accessible solution for those new to test automation. For more advanced usage, we have implemented support for Groovy and JavaScript expressions, as well as support for object-oriented programming in XML, deepening the technical capabilities and flexibility of our platform.

Stability with Scalability. To tackle this challenge, we adopted Data-Driven Testing (DDT), which allowed us to effectively monitor interactions between plugins. For each test, we constructed a complete tree of method calls, events, etc. This enabled us to control the logic and see what was happening with a specific test at a specific location. This approach improved our understanding of component interactions and allowed us to promptly identify potential issues, thereby stabilizing the platform’s operation.

Additionally, we prioritized testing, starting with the most critical and frequently used plugins, to quickly begin receiving feedback and promptly make necessary improvements. This strategy not only minimized risks but also accelerated the early development iterations, allowing us to rapidly enter the market and start collecting feedback for further enhancements. We faced significant obstacles in managing the scalability of our testing infrastructure. By gradually scaling up our test environments and implementing rigorous monitoring, we ensured that our solution could handle increasing loads without compromising stability.

Ensuring Easy Integration of Automated Testing into Customer Production Processes. Our plugin architecture facilitated smooth integration with key industry tools. Using Maven eased project and dependency management, while Jenkins supported continuous integration and delivery. JUnit provided a reliable basis for unit testing, and Selenium enabled web testing automation. Alternatively, other web application testing automation tools could be used in place of Selenium. Integration with BrowserStack allowed testing across various browsers and operating systems. This multifunctional integration ensured that XML2Selenium could be easily adapted to customer production processes, simplifying the implementation of test automation and enhancing its efficiency.
Ensuring Transparency and Traceability of Test Results. To address this challenge, we created a system that automatically generates HTML reports at the end of all tests, covering various aspects of testing — from business metrics to technical details of system behavior. We implemented a self-testing feature for the reporting system, which checks each generated report for errors and inaccuracies. Additionally, to ensure transparency of testing results, our product includes support for video recording of tests, snapshots of HTML page source code, and screenshots.We encountered several hurdles in ensuring the accuracy and reliability of these reports. By incorporating a feedback loop and continuously refining our reporting system, we were able to enhance the clarity and utility of the test results for our clients.

Testimonials

Bram Cool

CEO, Ezwim, a global leader in Telecom Management, the Netherlands

The customer testimonial about project: Xml2selenium – the JazzTeam company's product

"We now are in a much better position and detect bugs before our customers find them. The test automation frees our testers to focus on other quality aspects and new functionality. As a result, we offer our customers an even higher level of quality and stability of service."

Hein Remmen

Director Product Management, Ezwim, the European market leader for Telecom Management Technology, the Netherlands

The customer testimonial about project: Xml2selenium – the JazzTeam Company’s product

"Since we release new versions of our platform on a monthly basis, we spent a lot of time on manual regression tests.

JazzTeam completely automated our regression test set and put up the required testing infrastructure for us. Their XML2Selenium framework enables our testers to implement new test scripts without any Java knowledge. The XML2Selenium reports help us to quickly assess bugs and problem areas in our platform. The integration with BrowserStack allows us to test on all combinations of browser and OS that we want to support, without any extra effort.

JazzTeam provided excellent support and swiftly responded to all our questions and problems."

Result

Successful Launch of Our New Product with Official Website. The new product, XML2Selenium, has been successfully launched, complete with its official website at http://xml2selenium.com. Sales kicked off with strong performance indicators. Our key client has installed and seamlessly integrated XML2Selenium into their automated testing processes. The efficiency and reliability of our solutions have been highly rated by their team, as evidenced by their positive feedback about their collaboration with our team.

We have developed specialized training materials, including an interactive web-based tutorial designed to assist novice users in mastering the product.

During the development of XML2Selenium, our team wrote several thousand tests to validate the framework. Implementing such a volume of tests in real project conditions has enabled our team to enhance the quality of the software being developed and to promptly identify and resolve potential issues.