Adventures in Automated Testing: Setting Out

Over the past few months, I’ve been considering a slight adjustment in my career focus. I’ve been an Engineer of no particular focus for around a decade; first in a design agency as the sole developer on staff, onto a full stack engineer, and now a senior software engineer. The strength of my roles has grown across my career, but I’m still left without much focus.

This lack of focus isn’t a problem; don’t get me wrong. I’ve been in this position for a long time, I enjoy it and I’m able to tackle a huge variety of tasks. However, as I’m growing as an Engineer, I’m noticing my professional interests pulling me in distinct directions.

Lately, I’ve been pining more and more for automated testing and quality assurance. I started a book about maintaining standards in a development team, I’m an advocate for reaching as high a code completion as possible in my testing environments and I’m now working on a project where the focus is load testing a large application. Whether I’ve intended it or not, I’ve been gravitating towards this.

Today, I’ve made the decision to fully explore this avenue, to really determine if this is indeed what I want. So here it is, “Adventures in Automated Testing: Setting Out” the first chapter in my journey of QA self-enlightenment.

What Do Already I Know

I’ve worked with automated testing in the past, though it’s always been use of the tools that are already present in the applications I’m working with. Below I’m going to list the tooling I’m familiar with, and honestly try to define them to the best of my knowledge. I’ll then research and write any errata beneath each entry. Why? Well, I need to determine what are the holes in my knowledge before I proceed to fill them.

• PHPUnit
• Behat
• Selenium
• Jenkins
• JMeter
• Travis
• CircleCI

PHPUnit

This is perhaps what I’m most familiar with. PHPUnit is a PHP-based unit testing library. Unit testing is one of the smallest and most granular testing systems on the spectrum, it’s about breaking down your application or classes into the the smallest parts (units) and testing them.

Errata: There isn’t any. I’m intimately familiar with this system.

Behat

Behat is a behavioural testing system that allows you to write tests in a simple and friendly syntax called Cucumber. In the background each line of these feature files are resolved to a Context, which uses a number of possible web drivers to execute them. The tool is for creating functional, behavioural tests. More for testing systems in unison, rather than separately. It is often used alongside something like PHPUnit as the tools complement each other.

Errata: The syntax is called Gherkin, the engine that executes it is called Cucumber; I foresee myself getting confused between these two often.

Selenium

Selenium is a web driver. It takes the commands from a range of test systems (for example, Behat feature contexts) and executes them within a virtual browser system. It’s aware of the DOM and as such can interact with it in a similar way to a user. This makes it useful for writing scripts that traverse your user journeys and test for expected outputs.

Errata: What I described here was simply the WebDriver part of Selenium, but there are other products too. Other than that, I was pretty spot on.

Jenkins

Jenkins is a stand-alone Continuous Integration system that will run all your tests for you. You can script, or bootstrap, all your testing systems, and execute them each time the product is built.

Errata: I’ve not described this in great enough detail to really give it the credit it needs, however, it’s factually correct.

JMeter

JMeter is for load and stress testing. It’s a language that allows you to configure complex testing scenarios to push your application to its limits.

Errata: JMeter isn’t just the language it’s the whole application that’s responsible for load testing your applications.

Travis/CircleCI

I’ve grouped these together because they both serve a similar purpose (regardless of the individual differences between the two) is similar in some ways to Jenkins, but whereas Jenkins is standalone, these ones is Cloud Based. They integrate with many services (such as GitHub) and can initiate builds and deployments when PRs are merged or branches are committed. Travis is free for Public Repositories with a limited feature set, but premium for Private repositories, whereas CircleCI charges based upon the capabilities of the build process.

Errata: None. I’ve used both Travis and CircleCI extensively. I suppose I should mention that CircleCI has a free plan also, but it’s limited to one concurrent build on a single container instance.

Finding the Knowledge Holes

As it turns out, I already know quite a bit about the tooling available. I’ve omitted some I know about, and no doubt have not yet discovered a number of excellent testing tools, and for this reason, my next step is to identify any holes in this knowledge and close them up.

I feel like, with the knowledge I do have, that I could successfully navigate the spectrum from the under the hood technologies, to the testing toolkits, to the continuous integration systems, however, I can only do this for a very specific set of tools. Ideally, I’d love to find something cross-platform and extensible that I can really dig into and grow with.