Launch of Selenium tests on CircleCI using Docker

    Introduction

    The article is intended for people familiar with Selenium technology, since the article is referred to launch of Selenium tests on CI platform CircleCI. But first, let's deal with the basic concepts of this topic.

    CI

    Continuous Integration is the practice of software development, which is to perform frequent automated project builds to quickly identify and solve integration problems.

    CI processes organization

    On a dedicated server service (Job) is organized, which includes the following tasks:

    • getting the source code from the repository;
    • project build;
    • tests execution;
    • completed project deployment;
    • reports sending.

    Builds can be carried out:

    • on request;
    • on schedule;
    • on commit.

    Benefits

    • automatic and regular running of unit tests;
    • availability of a stable version for demonstration and testing;
    • quick detection of the effects of poor-quality code.

    Disadvantages of classic CI systems

    • need for a dedicated server;
    • support expenses.

    Why CircleCI?

    One of the main differences of CircleCI from others CI is that assembly of a build takes place in the cloud, that means there is no need in a local build machine. The service allows to configure build testing flexibly, since a user has access to sudo. As well there is a local version that can be deployed in a private cloud.

    CircleCI supports the following technologies:

    • Python, Node.js, Ruby, Java, Go etc.;
    • Ubuntu (12.04, 14.04), Mac OS X (paid accounts);
    • Github, Bitbucket;
    • AWS, Azure, Heroku, Docker, dedicated server;
    • Jira, HipChat, Slack.

    CircleCI advantages

    • easy and quick start;
    • free version for commercial use;
    • small and easy to read configuration files in YAML format;
    • no need for a dedicated CircleCI server.

    CircleCI disadvantages

    • CircleCI in free version supports only Ubuntu versions 12.04 and 14.04. MacOS using is a paid service;
    • Although CircleCI can work with any programming languages, only Go (Golang), Haskell, Java, PHP, Python, Ruby / Rails, Scala are supported from the box;
    • if you want to customize the system for yourself, in some cases problems may arise, and then third-party software will be needed to achieve the goal.

    Project

    As an example, let's use the test project on node.js, which comes with the selenium-cucumber-js library package. This project is a bundle of Selenium + Cucumber, in fact, it doesn’t matter what technologies are used, the main thing is that the project works on Selenium.

    To run selenium tests on a remote CI there are several options:

    1. To connect a virtual monitor in which the browser will be displayed and you can watch the passing of the tests by connecting to it;
    2. To use Selenium Grid.

    We’ll use the second option as Selenium Grid components are easy to configure on CircleCI.

    Project configuration

    To configure, it’s necessary to slightly modify our project.

    CircleCI configuration

    It’s necessary to add a new catalogue .circleci in the root of the project and create config.yml file in it.

    File link: https://github.com/JazzTeam/selenium-cucumber-js/blob/master/.circleci/config.yml

    Let’s analyze the file.

    config.yml is a CircleCI configuration file, the environment, various commands and launch parameters for our application are described in it.

    Docker

    In docker section we specify docker images we want to use, in our case these are:

    selenium / standalone-chrome: 3.11.0 - image, which ensures the operation of Selenium Grid, read more here:

    1. circleci/node:7.10 is an image, which is an environment for work of node.js applications;
    2. selenium/standalone-chrome:3.11.0 is an image, which ensures work of Selenium Grid, read more here: https://github.com/SeleniumHQ/docker-selenium.

    Steps

    This piece of code was generated automatically, in fact, there is a final setting of the environment, the installation of components.

    Further there are three Selenium Grid configuration commands:

    1. create docker network - creates an isolated grid net for our docker containers;
    2. install selenium-hub - launch of Selenium-hub, within the grid net, with forwarding of port 4444 to connect our application to it during tests launch;
    3. install selenium-node - launch of selenium-node, within the grid net, and connection of it to the previously launched selenium-hub.

    To view the results of the tests after the completion of job, you need to open the step - launching of Selenium-node container within the grid net with Chrome browser installed and configured.

    And finally two commands:

    1. install-npm - downloads all the dependencies necessary for our application;
    2. run tests - tests launching.

    Project set up

    In order for our tests to run not locally, but work through the Selenium-hub, you need to modify the file, or perhaps it is a class, that is responsible for configuring of web driver.

    In our case this will be the file chromeDriver.js:

    All we did was indication that the web driver should use the external Selenium Hub to run the tests, which in our case is available on localhost via port 4444 (the installation and configuration of the hub was covered in the previous section).

    GitHub

    Our test project is ready, now we can upload it on github, as it was said earlier, CircleCI has synchronization with github accounts.

    Let's log in to github, create a new repository and upload our project into it.

    Here is a link to the test repository:

    https://github.com/JazzTeam/selenium-cucumber-js

    CircleCI

    Let’s go to https://circleci.com/ and log in under the same github account.

    Now go to the ADD PROJECTS section. Here we see our project, which we previously uploaded to github. Select Linux operating system and click the Set Up Project button of the right project.

    CircleCI should determine the configuration for our project by itself. Depending on the selected configuration, the system itself will generate us a configuration file for job. Points one to four in the screenshot below describe the steps to install this configuration file (we did this in the CircleCI Configuration item).

    In this form, we see the basic configuration of the project and the steps to install the job config file:

    Scrolling below, we see an example of a config file of job, which was generated with the system according to our configuration. It was it which was taken as the basis of the configuration, which we had considered earlier.

    Then let’s run our tests and press the Start Building button.

    Job has launched:

    In real time, we can observe the process of building a project and running of tests, by clicking on any of the steps, we will see its logs.

    To view the results of the tests execution, at the end of the execution of job, you need to open step run tests.

    An example of run tests open step with logs of passed tests:

    Now let's see what will failed tests look like. To do this we will change the expected result of one of the tests on a knowingly false one:

    This test is implemented using js framework cucumber.js.

    As soon as we commit our changes to github, a new build according to our job will be automatically initiated and it will fail as it is expected:

    Let's look at the failed test logs:

    In the log, we clearly see the reasons of the test failing.

    Conclusion

    CircleCI is well suited for small projects in which it’s necessary to run continuous integration as soon as possible. Due to the integration with GitHub, it is suitable for open source projects.

    Jenkins and systems similar to it I recommend for use on large projects that require a serious setting of the environment. In Jenkins you can configure anything with plugins, but it will take a lot of time.

    Which CI to choose depends on the needs of your project.

    Good luck!

    Inspired by the following articles:

    https://djangostars.com/blog/continuous-integration-circleci-vs-travisci-vs-jenkins/
    https://habr.com/post/264075/