The automation of the product testing is a high priority issue, which brings certain challenges:
– When should the testing be applied?
– How to save time on testing?
– Which automation tools to use?
– What approach is better for test generation?
As regards the testing time, it often depends on the scale and the complexity of a project. Ideal is to apply testing at all the development stages. Keeping this in mind, one should already start writing tests when working on the initial version of the product. The tests must take into account the specifics of a system, and, if necessary, be able applicable for lots of different instances of the system. For example, for testing of all simultaneously developed versions of the project. Test input parameters should be treated with accuracy to be able to run them on servers with various configurations and do not change the code. It is preferable to register the largest possible quantity of input parameters in separate files.
The experience of time saving shows that the quality of the product directly depends on the investments in testing. It is necessary to properly maintain the documentation and mark the link with a corresponding business case for each test. This will save a lot of time on testing support in the future.
There exist a wide range of automation specialized programs to choose from. For web applications, one can take advantage of Selenium. It includes a variety of products for different testing methods, including Selenium RC Server, Selenium IDE, etc.
Selecting an automation approach, one should take into account all the diversity of methods: special programs, which generate tests on for common manipulations with browser; writing tests using procedural and objective-oriented programming; or creating them with the help of DSL (Domain Specific Language). Let’s have a closer look at the variety of approaches and the fundamental differences between them.
The plugin FireFox Selenium IDE is an example of such a program. To create a test, you should enable test logging in the plug-in, reproduce a case in the browser and your test is ready! Such work can be entrusted even to a QA engineer. They will be able to create many tests in a couple of days, and check a large part of the application. Moreover, tests can be delivered as a code in almost any programming language. Such tests can be launched in both the plug-in and other Selenium products. Note that the generated tests will be performed in a copy-paste style, unless some additional checks are inserted manually in the code. If it is necessary to change several steps in all the tests, you will have to edit all the files, maybe altering them completely. Having to exercise a large number of tests, it may seem very time-consuming. Large-scale projects may demand several hundreds of tests. In this case test maintenance becomes complicated, and the tests themselves lack flexibility. As a result, this approach is effective for fast test generation, but at the same time is void of other possibilities.
This method can be applied for small-scale projects where test generation speed is a crucial parameter.
To obtain the desired flexibility, tests should be coded in a programming language, rather than generated. To perform the tests one can use the following Selenium libraries: Selenium RC or WebDriver. They allow you to access a browser directly from the code. This task may require specific programming expertise. It shouldn’t be delegated to an inexperienced developer, as they risk to make common mistakes, such as copy-paste, hard-code and others. The doubtful quality of tests will make them lack all the specific properties for which the decision to write them manually was taken. It is advisable to adhere to the following rules when creating a test:
– bring out input data from the code, put them in files, or pass as parameters
– introduce logging in the tests
– employ a comprehensive way of error handling, so that it would concisely indicate what kind of error occurred
– apply the parameterization to reduce copy-paste and other ineffective approaches.
Due to the manual coding of all the test stages, it will be more time-consuming to apply them.
Another disadvantage of this approach is that if tests are written as a sequence of actions, they lose their desired software capacity. Such code is difficult to modify since the “net” of commands may intertwine, and certain situations may even require copy-pasting. It is hard to replace or modify the steps without affecting other scripts, when dealing with a large number of tests. All this impedes creating stable tests that one could easily scale and adjust. More time will be required for each new change.
This method is applicable for large projects, but if testing is necessary for over one year, the tests’ maintenance may cause significant problems.
The PageObject template is a perfect solution for attaining a complete flexibility of tests. The essential difference from the previous approach lies in the emphasis on architecture. This is a challenge for an experienced developer, who should be able to define and separate test logic from the actions to be applied on a web-page. Such approach assumes the description of tests in the POP style only. At first a POP model of the whole tested system is created; it describes each element (buttons, input fields, etc.) presents all possible actions (keystrokes, text input, etc.). This model alone is not a test yet, it is more of a prototype of the system under test and an interface for future tests, which will allow to access the system. For web tests, the prototype is a view of all pages, the cookie cutters from page, web elements, etc. In this prototype, you can get any of the objects and run all the types of actions it can actually perform. For example, for a log-in page the actions are:
– enter data in the field “log in”
– enter data in the field “password”
– press the “enter” button
The last action, in addition to the designated operation (pressing the button), would return an object of the next page. And for that object there will be available other operations, that are the only relevant to it.
– a high expertise standard of a QA engineer
– a slow speed of test generation
– the complexity of tests (sometimes it is comparable with the architecture of the application under test)
This method is usually applied to large and durable projects where the speed of test creation is of minor importance, but there is a need for continuous testing of a constantly changing project.
he prime demand for testing is a maximum flexibility in a minimum time. Auto-tests are often performed by testers rather than developers. Generally they lack programming skills to create an effective POP testing model. And there is simply no time for their education. In this case, persons with no programming background can use a tool which allows to create tests in a comprehensible way. This tool is the XML2Selenium framework. It is based on WebDriver. There is no need to write a complex code, it is enough to submit a test in a DSL language based on XML. Simply create an XML file and describe the sequence of tags. Each tag stands for an action in the test. It allows to generate illustrative and easily adjustable tests in short time. To run a test, you need only the framework, a browser, and the test described in an XML format. To maintain test flexibility, it is possible to implement inheritance for the tags, which will minimize the repetition of identical steps in different tests.
Platform distinctive features:
– possibility to easily add a new plug-in for the realization of necessary actions
– simple API for plug-in creation
– integration with Junit and Jenkins (but full independence from them and also from Maven, with the ability to create proper runners)
– remote debugging on a server
– absence of dependence from selenium, possibility to use another tool
– data driven support (ability to describe test results in a comprehensible manner, understandable even to someone who didn’t create the test)
– the possibility to self-test the behavior, i.e. to write tests for the framework in the same language
– smart logging, the ability to create snapshots and screenshots
– the minimum of programming – jaxb
– eclipse plug-in – a simple editor for creating new tests, doesn’t requires XML skills
– test cases validation
– data randomization
– thread saved, the ability to run any number of the core versions, data record to different directories (load testing)
– user friendly exception processing
– reports for a business-user in a bdd style of any format
– if/for tags support
Using the platform, you can rapidly create quality tests, and get plenty of opportunities for their convenient application. If necessary, create your own application-specific framework extensions.