Adam Hepner

Running your first test with Robot Framework using docker

Note: this article had originally been published on the [WonderProxy Blog](Running your first test with Robot Framework using docker) in March 2019.

Intro

This is the first in a series of 3 articles. When you complete them all, you will have a functional and extensible, albeit basic, test automation solution for web applications. This solution will be prepared for inclusion in a CI/CD Pipeline, as well as for local execution.

The complete solution will use Robot Framework as a generic test automation tool, and use Selenium Webdriver to perform very basic browser automation. Two methods of parametrization of test runs will be introduced:

After you complete this article, you will have successfully created your first (empty) test case, and executed it using a test runner docker image.

Please note, that the article is meant to be quite hands-on, and even though I will do my best to explain the concepts being introduced, I cannot effectively describe all possible aspects of the solution in detail, as I would be doubling the excellent documentation of the projects that are in use. If you’d rather just look at the code, feel free to clone this companion repository. Code relevant to this specific article is available under specific tag.

Who is this for

I assume that you:

There are no specific assumptions about your:

Motivation

During my career, I’ve seen multiple teams struggle with test automation solutions on all levels, including web testing automation. A quick Google Search returns a Quora thread with list of usual challenges related to this area of our daily work. In short, as a test automator, you will most definitely face those questions:

If you were to use Robot Framework as your automation solution, I’d say you were on a good path to solving almost all of those problems. How? Every one of those could be possibly its own article! But first…

Why Robot Framework

Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). It has easy-to-use tabular test data syntax and it utilizes the keyword-driven testing approach.

(Robot Framework Official Webpage)

I admit that Robot Framework is my personal preference when it comes to test automation, but I hope to at least intrigue you enough to give it a go. Here are some of the reasons why I like and recommend using Robot Framework:

Why Selenium

Even though there have been new web test automation frameworks popping up (see: TestCafe or Cypress), Selenium remains one of the best choices to consider, if you have a website or webapp to automatically verify. What is Selenium? In words of its authors:

Selenium is a portable framework for testing web applications Selenium provides a playback (formerly also recording) tool for authoring functional tests without the need to learn a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including C#, Groovy, Java, Perl, PHP, Python, Ruby and Scala. The tests can then run against most modern web browsers. Selenium deploys on Windows, Linux, and macOS platforms. It is open-source software, released under the Apache 2.0 license: web developers can download and use it without charge.

(Selenium Wikipedia Page - emphasis mine)

The important aspect for us, is that Selenium allows us to use most mainstream browsers in uniform way in order to automate web testing. Robot Framework comes equipped with SeleniumLibrary, that can be used to interact with a remote SeleniumHub. For our purposes, a hub can be brought up with help of Docker (again), but in the future you can either roll your own, or use one of many services like BrowserStack or SauceLabs.

Prerequisites

In order to complete this article’s goals, you will need the following:

I assume that you are somewhat familiar with code editing and working on the command line.

Writing and understanding basic test file

OK, so in order to run a first test case, we need a first test case. Robot Framework allows us to write test cases in simple text files using natural language. You may be familiar with this concept if you have ever worked with Cucumber. This is important, because from the perspective of test automation specialist, it helps focus on business oriented aspects of the tests, instead of technical aspects. In other words, it’s more important to say that a user account must exist, than to describe, step by step, how to make one.

Let’s say that we want to test this testers-friendly page in both Chrome and Firefox. In order to do that, we will be taking baby steps. First, we need to get Robot Framework going, and running an empty test suite. This will be a file called initial_test_suite.robot in directory tests/ of our project:

tests/initial_test_suite.robot

Robot Framework uses the filesystem as a natural way of organizing test suites into hierarchical structures. Each directory that contains test text files may be recognized as a test suite itself, with its own respective setup, teardown, or metadata. Robot Framework will treat tests/ as a test suite (collection of tests), but initial_test_suite.robot will also be a test suite. This is an important concept - the test suites can and will be nested..

The usual Robot Framework file extension is .robot. A .robot file can be treated by Robot Framework as either a test suite or a resource file. They are more or less the same, with the important difference being their purpose: test suites contain test cases and are meant for execution, whereas resource files contain only keywords for shared use in multiple test suites. Resource files can be referenced by other resource files, or by test suites, but test suites cannot be referenced by any other file.

The structure of the .robot file is understandable, but also very flexible. It contains one or more sections, those being:

Each section contains keywords, which you can treat like functions in other programming languages. You can either define keywords in the same file, or import either resource files or external libraries that provide you with additional keywords. A keyword may accept additional arguments, but we’ll investigate it deeper in next series installment.

Text written without indentation under Test Cases and Keyword headers is treated as test case name and keyword name respectively, and test case content and keyword content must be indented by one level - at least two spaces. Using this knowledge, we can write the simplest test suite (tests/initial_test_suite.robot) to get our setup going:

*** Test Cases ***
This test case does nothing
   #This is a comment.
   #Remember to indent the test case content with at least two spaces
   No operation

No operation is a keyword from BuiltIn Library, and - as the name suggests - we don’t have to import anything to use those keywords.

What’s going on here

This test case does nothing is the name of the test that we want to execute. As you see, it contains exactly one keyword to execute. This keyword, No Operation will do nothing, just pass the test execution.

Running the test and creating a test runner

In order to actually run the test suite, we can go one of two ways: install Robot Framework locally, or use docker image to create a container with our test execution. Let’s go the Docker way, simply because it is more robust (it won’t get affected by the works on my machine syndrome). Fortunately, most of the legwork has already been done for us in robotframework/rfdocker project, and we can (for now) call the latest rfdocker image:

docker run \
 --rm \
 --volume "$PWD/tests":/home/robot/tests \
 --volume "$PWD/results":/home/robot/results \
 robotframework/rfdocker:latest \
 tests

This is quite a lot to unload, and I’ll refer you to docker documentation for all possible options, but here’s a rundown:

I hate to repeat myself, so let’s just pack it into a run.sh script to avoid early onset of RSI:

#!/usr/bin/env bash

docker run \
 --rm \
 --volume "$PWD/tests":/home/robot/tests \
 --volume "$PWD/results":/home/robot/results \
 robotframework/rfdocker:latest \
 tests

Now for last adjustments, let’s make it executable, and in case you were using git for your version control (which you should), we’ll add the results/ directory into .gitignore file:

chmod +x run.sh
echo results/ >> .gitignore

We can finally call run.sh and enjoy the magnificent results of our first, unproductive test case:

> ./run.sh
==============================================================================
Tests
==============================================================================
Tests.Initial Test Suite
==============================================================================
This test case does nothing                                           | PASS |
------------------------------------------------------------------------------
Tests.Initial Test Suite                                              | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Tests                                                                 | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
Output:  /home/robot/results/output.xml
Log:     /home/robot/results/log.html
Report:  /home/robot/results/report.html

You will notice that the results/ directory has appeared in your working directory, and there are 3 files in it:

Congratulations, you have successfully executed your first Robot Framework Test Case!

Summary

In the next installment, we’ll enhance our test and add some parameters to it, via files stored in the project, and via environment variables.
If you've enjoyed this content, how about

<< Previous Post

|

Next Post >>