CircleCI 2.1 orb setup

There are many ways to incorporate Trakr into your CircleCI configurations. This guide focuses on using the CircleCI orbs to integrate visual testing into your CI pipeline. If you are interested to use Trakr testing without using orbs, please refer to this blog post.

Before you start

  • Make sure that you declare in your CircleCI config.yml as version 2.1.
  • Allow the use of third-party orbs under your (organization) security settings in CircleCI. (more info)
  • Setting up the TRAKR_API_TOKEN and TRAKR_PROJECT_ID environment variables under your CircleCI project build settings. You can find both information under your Trakr project settings > Developer/API

CC environment settings

The main concepts

Flow chart

Note that in or CircleCI we are executing both the build/tunnel job and the testing job in parallel since we need to keep the tunnel open for the test job.

1. Build
This is the general steps where you create the “build” environment of your web-based project. It likely includes checking out the code, load data into a database and running the webserver. The result should be some type of ad-hoc development environment that can be plugged into a variety of automated tests.

Most likely your build steps will be very different from our referenced sample.

2. Tunnel
Once you have a web-build, it is time to expose it externally to allow tools such as Trakr to access the build so that it can perform visual testing. This can be accomplished through port forwarding or tunneling from the CircleCi build to the outside world.

There are a number of tools available to do this, namely:

In this guide, we will be using localtunnel because it is completely free and does not need any signups or registration.

3. Test
Now that we have a tunnel opened, we can perform visual testing with Trakr.

Our sample config.yml

Below is a walkthrough of our sample CircleCI config.yml using an orb.

Let’s go through each part for a detailed walkthrough of our config.yml:

Code block sample

Here we are declaring the configuration version, import the Trakr testing orb into our configuration file. Finally, we are declaring an executor (environment) for our web project build.

For our web build, we are using a simple LAMP docker image. You can easily use different images and different web-development technology for your project (e.g. node, ruby, python, etc. See a list of pre-built images from CircleCI as well as adding other images for database connections, etc depending on your project complexity.

Code block

We also declare an environment variable in the configuration TUNNEL_PORT. This is the local port which your web build server runs on. For web-based apps, it can be 80, 8080, 443 (https) and in some cases 3000 (for Node JS). We will perform port forwarding in later steps from the local server port to the outside world.

Code sample

Lastly, our web build is extremely simple, we simply check out the project repository and copy them into the default Webhost root directory and start the webserver so the static website runs locally. You probably will have a more complex build process. However, the end result should be that you have your web-build running on the localhost at some port.

Code block
We install Node JS (LTS) and use the node package manager (NPM) to install localtunnel. However, if you are already using a Node JS based (executor) environment, you can simply run the last step to install localtunnel directly.

Code block

We start the tunnel and run it in the background so that we can move on to performing the test while the tunnel is established in the background.

Now your web-build will be accessible via https://trakr-project-$ where the $TRAKR_PROJECT_ID is an environment variable we set in CircleCI user interface (See before we start section)

(Note: we are using a shell script to run the tunnel in order to keep it persistent. Due to the unpredictable volume to the server, the tunnel can error out easily).

Code block

Finally, we run a script that waits for the Trakr test (in a separate job) to complete before we close the tunnel and discard the CircleCI build/job.

Code block

Trakr visual testing is defined in a separate job from the build/tunnel. Here we can pass in various additional parameters into the step to customize the test. All of the parameters can be found from the orb source.

  • delay_start: Since our build/tunnel job and the test job runs in parallel, you should set this to a reasonable guesstimate of time it will take for the build to complete and for the tunnel to be opened. Setting this to a small value such as 10 seconds might cause the visual test unable to fetch a screenshot from the build URL since the tunnel hasn’t been opened.

On Trakr

Screenshot collection

On Trakr, the screenshot collection triggered from CircleCI will have a name of CC WF $CIRCLE_WORKFLOW_ID pattern.

Visual test

A visual test will be generated between the last screenshot collection from the base environment you specified in your config.yml (defaults to production) and the screenshot collection captured from the CI build.

Additional notes

  1. The build/tunnel happens in the same job so that the tunnel can be opened on the same environment where the webserver runs on.
  2. can be unstable at times due to high traffic volume. This can cause some screenshots not being taken correctly. Please consider your own port forwarding service or use a paid service such as Ngrok.