Adventures in Automated Testing: Locust

I’d like to have gone into this whole thing in a bit more of a steady manner, but as luck would have it, I’ve been thrown into the proverbial deep-end by happenstance. I’m currently part of a task force who will be attacking the concept of Load Testing for our company. We’ve had some shaky days with massive influxes of traffic in the past, so we’re aiming to go into this thing fully prepared for every eventuality.

First stage was selecting an appropriate software. There are lots of them around, so we set about defining a criteria for what we regarded to be ‘must-have’ features. For us, this criteria was as follows:

  • Can be integrated into our CI/CD pipeline
  • Easy to measure our system during a load test
  • Good reporting of results 
  • Ability to write complex scenarios
  • Ability to version control the tests

Since we wanted to version control the tests and integrate them into our delivery pipeline, this ruled out pretty much all SaaS Load Testing products. Which on some level, is a shame, since they offer some excellent reporting tools.

We reviewed a number of different platforms and the one we settled on was Locust. These guys have done a great job in making the codebase very easy to use. It’s all written in Python, and as such, you have access to a wealth of Python-based tooling, making the whole thing really flexible.

Installing Locust

This is where we had the most trouble. Getting locust installed was a bit of a job – I’ll preface this by saying that the issues we experienced at this stage, have now been resolved by locust from version 0.7.5 onwards.

We discovered that Locust didn’t support SSL. This wasn’t Locust’s fault – a dependency’s dependency had dropped some essential, breaking code into a minor version release. All guides we found suggested that this was due to Python dropping SSL support in their version too, so the recommended course of action was to rollback Python to a version < 2.7.8.

We used pyenv and virtualenv to try and get this to roll back effectively, and then other things began to break. It wasn’t easy.  Then I discovered that Locust’s dependency gevent had resolved this issue in their latest version, so I had to wipe references to pyenv and virtualenv reinstall python ( brew install python ) and upgraded gevent directly ( pip install gevent==1.1.1 ).

Everything began working smoothly.

The good news is that Locust engineers have now resolved this issue and are now packaging gevent 1.1.1 with the installer, so you’ll likely never encounter this issue.

Writing Tests

One of the biggest challenges I faced with Locust was discovering how to structure our project. Locust provides you with a set of tools and instruction on how to use them, but if you’re doing anything more substantial than the simple examples they offer you, then you’re working in an essentially freeform project environment.

Throw into the mix that I’d never worked in Python before, and you have a recipe for confusion.

We eventually separated our tasks from our tests themselves and created Page objects to manage the route retrievals. It went through a number of iterations before we decided on a solution.

Once that part was done, we realised that much of the hardest work was over and we could begin writing our tests in earnest.

Running The Tests

We opted for a series of Digital Ocean droplets so we could distribute the tests across a number of data centres. This approach allowed us to hit our staging server with a large amount of virtual users from different locations.

We used New Relic, Dataloop and Grafana to see in real time what the tests were doing to our system. We could see what the slowest database queries were, we could identify bottlenecks in the code, and we could track processing timings over time. We made some extremely interesting discoveries.

From this, we were able to make targeted hits of refactoring in the system to really focus down on the areas that weren’t fit for purpose. Before we knew it, we had worked out all the kinks and issues with the system and we were able to max our webserver’s CPU, forcing an auto-scale to continue serving content with negligible issues.

Adventures in Automated Testing: Locust

The Impostor Pattern

This last few weeks, I’ve been refactoring much of our legacy codebase; some components of it were in desperate need of replacing entirely, since they were no longer fit for purpose.

One of these components was a class we called MysqlX – a class that handled much of the website’s database communication. This was a crucial component to refactor; the idea was to remove dependence on the deprecated mysql_ functions and shift over to the Laravel’s Capsule – a component which is already extensively used across the newer portions of the system.

We considered a number of approaches; below are some of the constraints we were working under:

  • The MysqlX class is initialised a single time at the bootstrap stage.
  • global variable $mysqlx is utlised no less than 800 times throughout the codebase.
  • As with all projects, we are working until a time/resourcing constraint.

Obviously, with all this in mind, writing a decorator class, or replacing the class entirely, applying to all instances individually, was inconceivable. We had not the time, nor the in-depth knowledge of each use, to safely replace all 800 instances.

So, I came up with a method of refactoring that was deemed safer, and far quicker. This was to be a temporary refactoring to quickly improve the performance of the architecture, with minimal resource overhead.

I call this pattern, the Impostor Pattern.

Impostor Architecture

At present, our entire site architecture spoke directly to the MysqlX instance that was initialised at bootstrap level. This gave us a single point of initialisation; an important requirement of the Impostor pattern.

The simple description of this architecture is that instead of talking directly to the class you wish to refactor (the deprecated class), you can instead speak with an impostor class. This impostor class will route requests through to the deprecated class using PHP’s built-in magic methods, or via subclassing. Over time, you will replace each of these methods in the impostor class with newly refactored or reworked code. Eventually, once you have refactored each of the methods, your impostor will assume the role of your deprecated class.

There are several advantages to this. Firstly, you are able to refactor, without modifying the code in-situ, where the deprecated class is being utilised. Secondly, it’s a quick method of refactoring, allowing you to safely remove reliance upon a deprecated class.

Note: this method will not apply to all circumstances; especially those where the deprecated class is easily accessible within the codebase. There are much more effective strategies for this, such as using a decorator class.

There is a few key differences between the Impostor Pattern and the Decorator Pattern. The Impostor Pattern requires no modification of the in-situ codebase. The Impostor Pattern requires no dependence on the class to be deprecated.

Stages of Development

Learning

The learning stage occurs when the impostor has been implemented, but there are no refactored methods present within it. At this stage, you are provided with a unique opportunity to use logging or metrics to really learn about how your deprecated classes are used. This stage can be released onto production, since it’s relatively harmless.

Method Refactor

This stage occurs when you’re prepared to refactor a method of the deprecated class. You are to recreate this method on the impostor, which removes the reliance upon the original deprecated version.

Edge Cases

The edge cases stage is reached when you feel you’ve taken care of all the refactoring required by the project. Now is a time to look at what you’ve learned in the learning stage, to identify any edge cases for which you’ve not catered for.

Retiring

The retiring stage is when the deprecated class is finally ready to be removed.

TL;DR

  1. Create a class (the impostor) that sits between your code architecture and your class to be deprecated
  2. Utilise magic methods or subclassing to ensure that all requests are routed through to the original.
  3. Add logging or metrics to learn about how your deprecated classes are used.
  4. Create and refactor each of the deprecated methods on your impostor class. Testing at each juncture.
  5. Take care of any missed edge-cases.
  6. Retire the deprecated class.
The Impostor Pattern

Adventures in Automated Testing: Setting Out

Over the past few months, I’ve been considering a slight adjustment in my career focus. I’ve been an Engineer of no particular focus for around a decade; first in a design agency as the sole developer on staff, onto a full stack engineer, and now a senior software engineer. The strength of my roles has grown across my career, but I’m still left without much focus.

This lack of focus isn’t a problem; don’t get me wrong. I’ve been in this position for a long time, I enjoy it and I’m able to tackle a huge variety of tasks. However, as I’m growing as an Engineer, I’m noticing my professional interests pulling me in distinct directions.

Lately, I’ve been pining more and more for automated testing and quality assurance. I started a book about maintaining standards in a development team, I’m an advocate for reaching as high a code completion as possible in my testing environments and I’m now working on a project where the focus is load testing a large application. Whether I’ve intended it or not, I’ve been gravitating towards this.

Today, I’ve made the decision to fully explore this avenue, to really determine if this is indeed what I want. So here it is, “Adventures in Automated Testing: Setting Out” the first chapter in my journey of QA self-enlightenment.

What Do Already I Know

I’ve worked with automated testing in the past, though it’s always been use of the tools that are already present in the applications I’m working with. Below I’m going to list the tooling I’m familiar with, and honestly try to define them to the best of my knowledge. I’ll then research and write any errata beneath each entry. Why? Well, I need to determine what are the holes in my knowledge before I proceed to fill them.

  • PHPUnit
  • Behat
  • Selenium
  • Jenkins
  • JMeter
  • Travis
  • CircleCI

PHPUnit

This is perhaps what I’m most familiar with. PHPUnit is a PHP-based unit testing library. Unit testing is one of the smallest and most granular testing systems on the spectrum, it’s about breaking down your application or classes into the the smallest parts (units) and testing them.

Errata: There isn’t any. I’m intimately familiar with this system.

Behat

Behat is a behavioural testing system that allows you to write tests in a simple and friendly syntax called Cucumber. In the background each line of these feature files are resolved to a Context, which uses a number of possible web drivers to execute them. The tool is for creating functional, behavioural tests. More for testing systems in unison, rather than separately. It is often used alongside something like PHPUnit as the tools complement each other.

Errata: The syntax is called Gherkin, the engine that executes it is called Cucumber; I foresee myself getting confused between these two often.

Selenium

Selenium is a web driver. It takes the commands from a range of test systems (for example, Behat feature contexts) and executes them within a virtual browser system. It’s aware of the DOM and as such can interact with it in a similar way to a user. This makes it useful for writing scripts that traverse your user journeys and test for expected outputs.

Errata: What I described here was simply the WebDriver part of Selenium, but there are other products too. Other than that, I was pretty spot on.

Jenkins

Jenkins is a stand-alone Continuous Integration system that will run all your tests for you. You can script, or bootstrap, all your testing systems, and execute them each time the product is built.

Errata: I’ve not described this in great enough detail to really give it the credit it needs, however, it’s factually correct.

JMeter

JMeter is for load and stress testing. It’s a language that allows you to configure complex testing scenarios to push your application to its limits.

Errata: JMeter isn’t just the language it’s the whole application that’s responsible for load testing your applications.

Travis/CircleCI

I’ve grouped these together because they both serve a similar purpose (regardless of the individual differences between the two) is similar in some ways to Jenkins, but whereas Jenkins is standalone, these ones is Cloud Based. They integrate with many services (such as GitHub) and can initiate builds and deployments when PRs are merged or branches are committed. Travis is free for Public Repositories with a limited feature set, but premium for Private repositories, whereas CircleCI charges based upon the capabilities of the build process.

Errata: None. I’ve used both Travis and CircleCI extensively. I suppose I should mention that CircleCI has a free plan also, but it’s limited to one concurrent build on a single container instance.

Finding the Knowledge Holes

As it turns out, I already know quite a bit about the tooling available. I’ve omitted some I know about, and no doubt have not yet discovered a number of excellent testing tools, and for this reason, my next step is to identify any holes in this knowledge and close them up.

I feel like, with the knowledge I do have, that I could successfully navigate the spectrum from the under the hood technologies, to the testing toolkits, to the continuous integration systems, however, I can only do this for a very specific set of tools. Ideally, I’d love to find something cross-platform and extensible that I can really dig into and grow with.

Adventures in Automated Testing: Setting Out

Setting a Default User Role in FOSUserBundle

I had to dig for this information, so I thought I’d collect it all in one place for ease of use in the future.

So, FOSUserBundle doesn’t come with any default permissions for your users. It’s a good thing too, because the implementation may not be suited to your project; however, it does give you a very simple way to set it up.

Events

You need to set up an event subscriber which listens to FOSUserBundle’s REGISTRATION_SUCCESS event. You can configure this in your config/services.yml

services:
    # User Default Role
    app.exception_subscriber:
        class: AppBundle\Event\Subscriber\UserDefaultRole
        tags:
            - { name: kernel.event_subscriber }

Next you need to create your event subscriber

<?php

namespace AppBundle\Event\Subscriber;

use FOS\UserBundle\Event\FormEvent;
use FOS\UserBundle\FOSUserEvents;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;

class UserDefaultRole implements EventSubscriberInterface
{
    public static function getSubscribedEvents()
    {
        return [
            FOSUserEvents::REGISTRATION_SUCCESS => 'onRegistrationSuccess'
        ];
    }

    public function onRegistrationSuccess(FormEvent $event)
    {
        $rolesArr = ['ROLE_USER'];

        /** @var $user \FOS\UserBundle\Model\UserInterface */
        $user = $event->getForm()->getData();
        $user->setRoles($rolesArr);
    }
}

Modification to your User Entity

Unfortunately, your event isn’t capable of doing this by itself, because the default FOSUserBundle behaviour sneakily overrides it, so we’ll need to do some overriding of our own.

Within your User Entity, you’ll need to override the addRoles method to ensure that your new configuration can work as expected.

public function addRole($role)
{
    $role = strtoupper($role);

    if (!in_array($role, $this->roles, true)) {
        $this->roles[] = $role;
    }

    return $this;
}

Summary

What’s happening here in total is that we are setting up an event subscriber that’ll fire each time a user registers in your application – this fires prior to persistence. Within that event, we are setting the role to your default. As mentioned, all of this happens prior to persistence so by the time that comes around, the user object is loaded with the role you want to load.

Credit:

FOSUserBundle Hooking Into Controllers
StackOverflow. A combination between RayOnAir’s answer and andrew’s answer.

Setting a Default User Role in FOSUserBundle

What the hell is $x

I’m currently in the process of writing my first technical book. I chose to write it on a topic that is close to my heart; how to effectively work within a development team, on a code-based level. It’s going to be about what you can do to the way you develop, and the code itself to make the development environment easier for yourself, your colleagues and contributors. Below is a preview of my book’s introduction.


What the hell is $x?

This is a great question; what the hell is $x?

$x is of course, as you may recognise, a variable in the PHP syntax. It is anonymous, it is elusive, it is ire-inducing. $x is the poster-boy for bad variable naming. It is indicative of everything I despise about working in a development team where multiple developers share a code-base. Whatever language you develop with, you’ll find things like $x in almost every code-base you’ll see in your career. It tells us nothing.

It is a manifestation of an attitude to development that is deemed healthy from the product point of view, but unhealthy from the engineering point of view. It’s focus is on speed of delivery, rather than sustainability of the codebase, when in reality, a delicate balance needs to be struck between the two. Throughout this book, I will hope to illustrate why, and what we can do to reduce the appearance of artefacts such as $x.

Why do I care?

One thing I’ve learned, as I slowly make my way through this journey of life, is this: You cannot control what other people do, you can only control what you do. This, really, has served me well as part of my ethos for working. When working with others, you’ll undoubtedly come across developers you disagree with, product managers who make decisions that you wouldn’t have, managers who you feel don’t have your best interests at heart, CEOs who compel you to do things you’d be otherwise unwilling to do.

It is a fact of life that everyone is different. Why then, would a developer’s approach to development standards, or their ability to work effectively in a development team be any different? It too would suffer from the same variation in behaviour, or adjusted priorities that sit as a contrast to your own.

Throughout the decade of my work in various industries as a developer, I’ve noticed that I’ve become extremely opinionated about the way others develop. My co-workers develop differently, and (in my mind at least) it was by definition, wrong. It took me some time to realise that it was simply me with the wrong attitude, they didn’t do anything wrong, and they even took the time to ensure that their code adhered to the relevant PSR standards.

When I started work, I was somewhat alone – I was the sole developer in a fast-moving design agency. There were plenty of employees, most of them were print-based designers, others were 3D designers, motion graphic designers, there were administrative staff, production staff, directors, and then there was me. At the time of joining the company; website development was a somewhat new field for them, I wasn’t just their token developer, I was helping shape an emerging department, it was very exciting. It was perhaps this then, that in-turn shaped my own development ability, and my deeply biased opinion on style and standards.

I wanted so desperately for my new department to succeed that I became fervent in my pursuit for development perfection. I needed my code to be flawless (with the benefit of hindsight, I can tell you here, now, that my code was awful). I worked at the company for around four years in total, and it was only in the final year that the workload increased to such an extent to warrant an additional hand. At the time, I thought of my new colleague as beneath me; a superiority that I had cultivated with the manure of my own self-importance. In truth, he simply developed in a slightly different way than I. He knew the platform well, was effective at tackling solutions, he just didn’t prioritise code style and the ‘correct’ way to do things.

Maybe it was the three years of working in isolation from the criticisms of others that generated this attitude in me, who knows.

Upon leaving the design agency, I started working for an Ecommerce development house. I was part of a team now, a cog in the wheel. There were those who were far more advanced than I, and those that still had much to learn. I was bombarded with a veritable smorgasbord of development styles and alien ways of architecting solutions. It was into this cauldron of differing styles that really caused me to become more self-reflective. I needed to look at the way I was doing things and not focus on others.

I couldn’t control the Linux distro selected by the CTO for our deployment servers any more than I could control the maverick approach to tabbing and spacing in too-long JavaScript files that one of my peers had. I slowly began asking myself the question “What can I do, to make this environment better for everyone?”

Leading by example

I’d been working on a utility package for use with a Magento 1.8 extension. It was composer enabled, PSR-4 namespaced package that would slot seamlessly into the extension we were constructing. I’d built it from scratch, and it was a masterpiece – to my endless supply of self confidence.

Then, I took ill. My colleague was forced to pick up the remaining bits and pieces to complete the package. I was dreading it. His style was vastly different to mine, I was certain he’d mess it up. He didn’t. When I returned, I took some time to inspect the code he’d written. It was semantic, descriptive and succinct, stylistically matching the classes I’d written and his new methods were sitting alongside my own with pride.

This is where it hit me; in order to make my own development environment better, I need to lead by example. Stop pointing out flaws in other peoples plans and just focus on my own, make sure what I do is perfect and use it to inspire others to do the same.

About this book

So here we are in the opening of this book. This book isn’t meant to be an instruction manual of the standards you should be using in your projects – there’s plenty out there on that already, but instead it just presents the various tools and ideas at your disposal and shows you the how and why of using them to improve what you do. This is a clear way to improve the way that you develop – not so much from an achievement standpoint, but from a team-working perspective. We’ll be attempting to reduce the negative comments you encounter on code reviews, and trying to improve the re-usability of your code for years to come.

By the end, you should be able to effectively answer the following question:

What can I do, to my code, that’ll vastly improve the development environment of everyone around me?

This book will mostly be utilising examples and syntax from PHP, but that’s simply because it’s my language of choice. Most (if not all) of what’s discussed therein will apply to other languages.

You may find that certain examples or chapters will contradict others. This is intentional; development communities offer many conflicting schools of thought for how to do things, this book simply presents the arguments and provides you the ability to form an opinion of your own.


I’m hard at work completing this book in my spare time and I hope to complete before the end of Q1 2016, but we’ll see how it goes.

It will be released in eBook form on LeanPub, as well as some other avenues that I’ve not yet explored.

What the hell is $x

Career Update 2016

I was employed at the start-up JustPark. It was a London-based start-up, but with a Cardiff development team, thanks to the Cardiff based Head of Engineering. It was a fantastic job, great pay for the area, great team – I got on fantastically with everyone, in both offices, exciting work, the whole package. I made some unforgettable connections with my colleagues, those that I can safely count as close friends for the foreseeable future.

Everyone in the company really loved what they did, it’s an intoxicating environment. I quickly found that I even developed a passion for parking via the process of osmosis. Simply being around other passionate individuals made me passionate about it too, even though it’s parking, and parking is a pretty dry state of affairs when you actually take the time to think about it.

There were exciting projects, I was leading one of them. A tough third party integration project that I was trusted with, and delivered incrementally to great internal review. Things were looking good, progressing fast, the company was growing at a substantial rate. I had only been there 5 months and there was another 10 hires after I came on board. For a start-up, all indications pointed towards a grand future. They even took the whole company (48 employees) on an all expenses paid retreat to Greece.

I had worked tirelessly on some brand new functionalities for the company and made some serious progress on upgrading the legacy system from a spaghetti mess of tangles into a nice, clean, Laravel-based engine. Things were looking good… Except, they weren’t. Considering my whole time employed at the company I can’t say I ever felt stable. As someone who does suffer from mild anxieties and situations where my mind actively fights against me, I thought it was me. I’m the sole breadwinner in my family, and the month before I started, I became a father for the first time. It was this extra pressure on me that I’d attributed to the feeling of instability within the company. I felt like it wasn’t the company, it was me.

On the 12th of November, after having been employed for only five months, our whole office (the Cardiff one) were let go, along with a substantial portion of the London main office. The company was forced to pull back and downscale, as part of a ‘refocusing’ drive. Dismissing the Cardiff office was an ‘easy win’ for the company, since we were a remote office. Skills aside, for our office to survive, they’d need to keep paying for the office space, equipment costs and other expenses, so from that angle it was a simple decision. I don’t blame them for this, I feel like I would have done the same thing if I’d been the one forced to make cuts.

Then, whilst we were still working our notice for the company, I watched my project get released, one that was arguably my proudest achievement at the company to date. Whilst I was thrilled with it’s release, I couldn’t help but feel a little bitter that I wouldn’t be able to see it thrive.

The feeling in the Cardiff office was… strangely amiable. The ‘banter’ amongst us continued as normal and we were all party to a strange sense of camaraderie. It was eerie. We’d just had our careers at the company cut short, but we were laughing and talking as if nothing had happened. We were hearing reports of the atmosphere in the London office and how you could cut it with a knife, but here, in Cardiff, it was a different story.

Then I realised why. In Cardiff, we were all in the same boat; each of us had the same end of the stick. This sense of camaraderie was born out this situation that we all shared. Yeah we all bitched about the situation, but each rant was followed up by a joke, or a small amount of laughter. We were obviously dealing with out own emotional reaction to what had happened, but on the surface, it felt affable. A difficult time, but one we were all in together.

We would all use any contacts available to us over the next couple of days, shooting off to various recruitment agencies and trying to quickly find a way out. Luckily the climate for developers in Cardiff is weighted in favour of the candidate, rather than the company; a city not short on new opportunities.

Then, an opportunity came up, not just for myself, but for the whole team. Crowdcube, an Exeter-based crowdfunding business brought all of us on board. This was all thanks to some excellent connections that other members in my team had made through various conventions and events in the past. We were all extremely thankful to get the opportunity, and grateful for the ability to continue on with our careers with minimal disruption, financially and otherwise.

On the 4th of January, I started as Senior Software Engineer at Crowdcube.

Career Update 2016

Too many thoughts

Sometimes, I have too many thoughts,
so much that I cannot think.
How can you have too many thoughts,
that they actually shroud your ability to think?
Thoughts are thinking.

It’s when your thoughts have no composition,
and no real grounding in reality.
There’s no flow; no determination.
Just random thoughts, with random feelings, and random fears, random anxieties, random… whatever.

See, that’s what happens.
My train of thought just dissipates and becomes ‘whatever’.
It’s like the whirlwind of activity within my cranium only allows me but a glance at my thought before it fucks off for another rotation.
Focus dies.
Apathy rises.
Clarity dies.
Lethargy rises.

I feel like my mind is a queue into the X-Factor auditions,
each thought is a hopeless auditionee,
screaming their own brand of notes and rhythms
None are able to break free,
from the snaking queue to the front of the line
they are each eager for their chance to shine.

Then… I regress.
I do nothing. I sit back, and try not to stress.
I figure if I give my thoughts time to reach conformity,
then I can regain my mind’s tranquility.
As if there are too many thoughts in the queue and I just have to wait until the rush-hour is done and all my thoughts have reached their destinations.

It’s not like that though,
it’s not a rush hour,
it’s a busy street in the weeks approaching the holidays.
It’s not busy for a time,
it’s busy all the time.
It’s a state of mind,
Not a state of time.

I need release.

Too many thoughts