Android App release optimization: The power of Automated Testing

Client

Our partner is a leading American company that has been at the forefront of developing innovative mobile solutions for the gaming industry for over a decade. Throughout these years, the company has established itself as a key player in the global market by introducing cutting-edge technologies into the entertainment sector. Our collaboration is focused on creating innovative products that set the standards for quality and safety in the industry.

Product

The product is a mobile app builder with a web interface, specifically designed for gaming industry owners. This tool allows them to easily create mobile applications for their clients, offering flexibility and ease of use. The mobile application consists of various modules, enabling users to engage in online gaming, view current casino promotions and offers, book hotel rooms, explore menus at nearby restaurants and cafes, contact the casino administration, and more. Users can download the mobile application for free and enjoy 24/7 interaction with the casino.

Additionally, the application leverages Geofencing technology to attract the attention of potential casino users within a specified radius. This is achieved by creating a virtual zone of any shape and size on the map. The mobile application tracks when a client enters this zone and then interacts with them, for example, by sending notifications about promotional offers.

Challenge

The primary goal of our team was to enable the client to enhance the quality control of their Android application delivery, considering the current production process with weekly releases. The team faced numerous engineering challenges:

Conduct research and select the optimal set of tools for developing and debugging automated tests, taking into account the specifics of the project.
Set up a CI/CD process that would allow automated tests to be run in a cloud environment and generate reports for the client.
Have the flexibility to configure the testing infrastructure, which would allow testing the application on various devices and in different configurations.
Cover all application modules with automated tests, which in itself was a substantial task.
Have the ability to manually and automatically run automated tests for each module individually, as well as a specific selection of automated tests.

JazzTeam Challenge

Before initiating the development process for automated tests, our team conducted a detailed analysis of the application modules’ source code, which revealed a significant problem. In the context of traditional Android app development, user interface layouts are organized in XML files, thereby providing the ability to extract identifiers for UI elements. This approach is critical for the development of automated test scenarios, as it requires access to these interface objects. However, the analysis of the application’s source code showed the absence of necessary packages with XML files for the modules, which questioned the ability to access UI elements and, consequently, the feasibility of creating automated test scenarios.

Solution

Our team successfully provided the client with the ability to enhance the quality control of their Android app delivery, adapting to their current production process with weekly releases. To achieve this, several challenges were addressed.

Phase 1. Selection of a Toolset for Automated Testing

For the development of automated tests, our team settled on Espresso – an open-source testing framework developed by Google. Espresso offers a comprehensive API suite for creating user interface tests for applications. This API enables the writing of concise and reliable tests. Espresso is particularly well-suited for white-box testing, where the test code leverages implementation details from the application being tested. Given our access to the customer’s application source code, this was a significant factor in favor of choosing Espresso. Another point in Espresso’s favor is its support for both Java and Kotlin languages, and the customer’s Android application was entirely written in Java. Additionally, Jazzteam specializes in Java and has extensive experience in developing automated tests in this language. Starting with version 2.0, Espresso has been available as part of the Android Support Repository (within the Android SDK), simplifying its integration into projects as it can be used without the need for additional downloads or installations. Further benefits of Espresso include:

Synchronization with the UI thread. Since tests run in a separate thread, without synchronization with the UI, they can be unstable. For instance, a test might attempt to press a button before it’s rendered, as rendering the interface takes some time.
A straightforward and easily extensible API.
Informative error messages about test failures. Espresso will display the application’s view hierarchy (like a navigation bottom menu) and explain why a particular “view action” or “check” failed.

For creating automated tests, Android Studi, which is the official IDE from Google, provides convenient tools for the development, testing, and debugging of Android applications. This enhanced development efficiency and ensured deeper integration with the Android ecosystem.

For debugging automated tests, we chose Genymotion, which offers a wide range of virtual devices with different Android versions, screen resolutions, and features. This allowed us to test the application in various scenarios and configurations, improving the quality and reliability of the product. Although the built-in emulators in Android Studio can also be good options, we opted for Genymotion for several reasons:

Performance and efficiency. Genymotion’s virtual devices typically run faster than the standard Android Studio emulators, thanks to their optimized performance.
Ease of setup. Genymotion is easy to configure and integrates seamlessly with Android Studio, simplifying its use for debugging tests.
Additional features, such as the ability to change GPS coordinates, simulate calls, and send SMS. This was necessary for testing various scenarios in our project.

Phase 2. Flexible Configuration of the Testing Infrastructure

An adaptive testing infrastructure was configured using Firebase Test Lab:

This cloud infrastructure enables testing the application across a variety of devices and configurations by utilizing real devices (rather than simulating them) located in Google’s data centers.
This approach provides us with test data that offers deeper insights into how the app will perform in the hands of actual users.
It features a convenient reporting tool that captures screenshots and videos of test runs, including the occurrence of errors with stack traces.
It also allows for testing specific test cases by recording a scenario through Android Studio, uploading the scenario file to the cloud, and initiating the test.

Opting for Firebase Test Lab provided more accurate and realistic testing outcomes. Firebase Test Lab also offered the capability for both manual and automated initiation of tests for each module separately, as well as for launching a specific selection of automated tests.

Phase 3: Configuring the CI/CD Process

Approach: We configured the CI/CD process to automatically initiate automated tests in a cloud environment and generate reports for the client.
Business Impact: This ensured:

Continuous and timely updates to automated tests.
Helped detect and fix errors in the application at an early stage.
Reduced the time and resources required for manual configuration and deployment of the application.
Easily integrated new features and changes, maintaining flexibility in development.

Our team set up a fully automated deployment process and the execution of automated tests on a daily basis. After debugging on Genymotion, the automated test code is transferred to a GitHub repository. Then, the CI/CD automation program Jenkins pulls the code from GitHub, compiles the .APK application file using the Gradle build automation tool.

Program for automating the CI/CD process Jenkins

The file (with the .APK extension) is uploaded to the Test Lab control panel of the Firebase console. There, tests are initiated on a configured list of devices in “instrumentation test” mode (a test written using the Espresso or UI Automator platforms). A “robot” will test the application’s apk file and send a report. We also configured parallel execution of automated tests in Firebase TestLab, separately for each module of the mobile application.

Phase 4: Automated Test Coverage for All Application Modules

Initially, we developed a comprehensive test plan that included covering the most critical functionalities of all application modules with automated tests. Then, we created over 1000 automated tests that were executed on a daily basis. We set up a convenient reporting system with reports on the outcomes of the automated test runs for product management. This allowed for timely tracking and fixing of issues in the application code.

Development Approaches

Analysis of the application modules’ source code and issues with XML files

Before starting the creation of automated tests, our team carefully analyzed the source code of the application modules. During this process, we encountered an unexpected problem: the necessary packages containing XML files for all the application module layouts were missing. In traditional Android development, application layouts are stored in XML files, where you can obtain the object identifier needed for test access. For example, with the identifier android:id=”@+id/navigation_home”, we can access the “Home” button in the code for testing purposes:

@Test public void clickButtonHome() {
onView(withId(R.id.navigation_home)).perform(click())
}

Access to the “Home” button in the code for the test to work

However, the client’s mobile application did not contain any fixed content; all content was dynamically sent from the server, so there were no XML layouts in the code as in a traditional application. This raised the question: “How can we obtain information about the structure of the application’s user interface and understand its components?” We solved the problem in the following way. To view the layout at runtime, we used the Chrome DevTools with any “beta version” of the mobile application. By enabling USB debugging, you need to navigate to chrome://inspect and click “Inspect.”

To view the layout at runtime, the Chrome dev tools tool was used with any “beta version” of the mobile application

Since the application does not use identifiers, we cannot use the withId() search method in Espresso tests. Instead, we used withTagValue() to find an element by its name in the web console. In this example, to press the “ENABLE” button, the following Espresso code should be executed: onView(withTagValue(equalTo(“enable_location”))).perform(click());

The Espresso test used the withId() id search methodn

Testimonial

Head of QA Department

Leading Provider of Innovative Mobile Solutions for the Casino Gaming Industry, Santa Clara, USA

"The team delivered exceptional results, setting up the entire mobile automated testing process for the project within a very tight timeframe. We achieved excellent coverage of test cases for the mobile Android application and maintained high-quality control over the app.

As this was the next phase in setting up automated testing for our project, I want to reiterate the high professionalism of the JazzTeam members. They can quickly analyze problems and find solutions, possessing vast experience and knowledge in automated testing, enabling them to complete tasks efficiently and on time."

Result

As a result of our project, we achieved significant improvements in the client’s software performance. Test automation reduced the time to release updates by 25%, enabling more frequent and stable releases. This led to a 30% reduction in manual testing costs, significantly enhancing overall profitability. Furthermore, improved quality control greatly reduced the number of error reports from users. Our innovations contributed to increased customer satisfaction, which was reflected in an increase in our client’s sales. The client expressed a high degree of satisfaction with the project, highlighting the effectiveness of the solutions and their positive impact on their business.