Black Box Testing by developers

Writing test scenarios for applications with a non-standard architecture requires the developer to have superior skills in general and knowledge of niceties of implementing the system to be tested in particular. This article will describe the process of developing a unit that allows reducing the level of requirements for developers, which in turn reduces the cost of Unit testing and provides additional opportunities when stabilizing the application, introducing new features, and optimizing the development process.

When setting the task, the main requirement was to conceal the implementation of the system core and provide a simple API for the creation of tests. This approach is consistent with the definition of the Black Box Testing concept.

The application to be tested was based on the Model Driven Development concept, but had a characteristic feature: business entities were fully responsible for behavior on all layers. The transport layer, representation, database, and business logic were all located and processed within specific business entities.

The algorithm for writing tests differed depending on the type of tests. Below there are algorithms for writing Unit and integration tests. UI tests are not described because they are not the subject of this article.

It follows from the presented algorithm that the higher the level of abstraction within the framework of tests is, the more we need to delve into the implementation of the application core only to “turn off” redundant dependencies.

Writing and executing integration tests requires even more effort than creating Unit tests. To create and carry out integration tests, the environment should be first created:

The examples described below relate to the specific application for which Black Box tests were created, and they are given to illustrate possible problems.

At the first stage, we face a problem: it is not possible to replace a locally installed MySQL server with an in-memory database (for example, H2), since the application uses a certain syntax and features of a specific version of MySQL, which are not fully supported by any of the in-memory databases (for example, stored procedures).

Also, to write integration tests, you need to serialize business objects in the http query params set and parse html when receiving a response from the server to de-serialize the received object and bring it to the required type.

In short, integration testing requires significant effort associated with manual testing, which slows down the process of integration testing.

We decided to develop a separate test unit. In the course of its implementation, there were several problems that were successfully solved:

To fix the first problem (the need to use a local MySQL server), it was decided to use Test Containers. During initialization, the base class from which test classes are inherited raises the test container with a specific version of MySQL and prepares it for further operation. Docker should be installed for that.

This solution allows the developer to avoid the need to install and configure MySQL locally and is an alternative to in-memory databases.

When executing requests to the server, the business object should be serialized to a set of http url-encoded parameters. A utility class that serializes the source object to the set of parameters described above was created for this purpose.

According to the project architecture, the server returns a ready HTML representation of the object generated based on the object attributes and set of templates. To analyze the response, it was required to present it in the form of an object.

The initial implementation contained parsing of HTML code received as a response. The input tags were parsed and the name attribute was checked for compliance with the format used by the application.

Subsequently, it was decided to abandon this approach and a better solution was found. Using the Mockito library, we managed to intercept the object being serialized in the HTML server, and thus we obtained not just a completely identical object, but the business object itself, which allowed us to call the methods of this object without losing additional meta-information that is lost when the object is serialized in the HTML server.

Having solved the previous problems, we implemented the possibility of writing Black Box tests for the system with the test data that was input in advance. In order to completely free the developer from manual data preparation, we implemented automated application initialization and filling with a basic set of test data. After raising the test container, the base class initializes the application and creates administrator and test user accounts. It is also possible to configure test account settings.

Below there is an example of a test in which a comprehensive business logic is tested — generation of a purchase order for items that are not in stock.

public class TransactionTest extends BaseBlackBoxTest {
…
@Before
public void setup() {
…
blackBoxService = new BlackBoxService(getCurrentAuthorizedUserSession());
…
}
…
@Test
public void returnsPurchaseOrdersForNotAllocatedLineItemsOnSalesOrder() throws Exception {
Transaction baseOrder = transactionBuilder. … .build();
Transaction expectedPurchaseOrder = transactionBuilder. … .build();
Transaction order = blackBoxService.invokeMethod(baseOrder, "allocate");
Transaction purchaseOrder = blackBoxService.invokeMethod(order, "createPurchaseOrder");
…
assertTrue("The Purchase Order must have all not allocated line-items from the Order.", compareTransaction(purchaseOrder, expectedPurchaseOrder));
…
}
}

A number of implemented builders allow to quickly create test objects by omitting the initialization of basic components and specifying only the basic set of data directly used for testing.

The general testing service emits the basic logic of calling global methods (note: a feature of the project architecture is the absence of a set of end points in the general sense). The system has a single end point that receives the object and parameters of the ‘global method’ to which the request flow is transferred for management.

o, the test developer can completely abstract from the peculiarities of implementation, use only the set of required data, and call global external methods of the object.