Four Recommendations to Boost Test Automation

According to DZone news, “Forrester recommends test automation rates of over 80% to meet the demands of two-week Agile sprints. However, only 12% of surveyed reference clients achieved that level of test automation.” While this gap may seem insurmountable, in today’s article we’ll share several approaches to boost the amount of automated testing you’re doing, and how to focus where it counts most.

1. Start Small

   There are quite literally thousands of tests that your organization could run. If you think of test automation as needing to automate them all, it will definitely feel overwhelming; in this situation, it’s hard to know where to even start!
   
   Don’t feel like you need to boil the ocean. Instead, start by identifying a bright spot. Taken from Dan and Chip Heath’s novel, a bright spot in this context is a project that can result in a quick win, illustrating the business value of automation. The idea behind a bright spot is not perfection, but rather “good enough” to prove applicability and success.
   
   To identify a bright spot for your organization, ask, “What is the problem and what are we trying to solve?”  For example, if you struggle with unit tests, that may be a good place to start. Or, another great place to start if you’re using Terraform could be with using Cookiecutter to make sure your infrastructure deployments match specifications.

2. Don’t Expect Perfection

   It won’t be perfect the first time and you will never be 100% done. Instead, embrace a culture of continuous learning and improvement. By taking small, steady steps to removing manual processes, the organization will, over time, reap greater dividends from automation. Launching off of a bright spot, and adding more automation, coupled with learnings the team has gleaned along the way, will help create a virtuous cycle.
   
   Consider that as automation is added, human error is reduced. The team saves time as processes are sped up and they don’t have to triage fat finger mistakes. With more time, the team can identify and execute new areas for automation, in turn, creating more time for the team. Capture milestones and metrics — such as pass/fail rates — as you go that illustrate the value of your automated testing efforts.

3. Find the Best Skills

   Take a look at where your in-house skills lie and deploy them to your benefit. For example, if your team can write YAML, you can run functional API and performance tests with the open source tool Artillery. Don’t be afraid to get creative as you look at ways that your team’s skills can marry up to create a bright spot.
   
   Conversely, if you don’t have a lot of automation skills in-house, do consider bringing in an expert who can help you identify a bright spot and teach you along the way the skills you’ll need to maintain and grow your efforts. For example, at Flux7 we help organizations identify bright spot candidates, and opportunities for automation services that will bring the greatest impact to achieving quality at speed throughout the CI/CD pipeline. We can help you build a foundation for automated tests that your team can build upon and expand.
   
   In addition, consider tools such as Selenium and Applitools that require very basic programming skills. They can provide quick wins for teams that are ramping up front end and cross-platform automated testing efforts.

4. Integrate Quality

   In 2001 Elisabeth Hendrickson published a paper, “Better Testing, Worse Quality?” in which she observed that as testing improved, software quality worsened. The cause was development over-reliance on QA, which caused developers to focus on sending code to test as quickly as possible, putting less focus on creating quality code. The result of this siloed approach was that bugs increased because developers were abstracted away from the effect of their actions — throwing buggy code over the wall to QA. The lesson from her paper is in lock-step with the goals of DevOps: to hold people responsible for product outcomes by breaking down silos.
   
   Clearly, siloing the quality team and treating them as a separate step in the process is counterproductive. It slows down the process at best and can also (counterintuitively, perhaps) decrease code quality. QA simply cannot be attached at the end of the process to meet sprint and quality output objectives. It is preferable to integrate your quality and test efforts as soon as possible.
   
   Indeed, many teams are blurring the line between development and QA. For example, developers and testers work together to make the UI more testable, or treat test automation as code. Test automation engineers work with DevOps engineers to ensure that the right tests are run in CI/CD pipelines. Security teams will often collaborate with DevOps or development teams to make sure security scans are occuring. This integrated approach not only streamlines the process, but it helps make people much more aware of their output, helping integrate myriad quality and test automation techniques very early in the process.

Example: Ansible and Cookiecutter

At Flux7, we created a templatized framework to create Ansible roles with testing built-in using Molecule. Molecule allows us to create different scenarios for different test cases, with which we can test Ansible roles against various platforms. Taking the solution one step further, we created standard templates for reuse with Cookiecutter. With our Molecule Framework for testing Ansible roles, we improve the quality of our Ansible roles using best practices, while ensuring integration with the CI/CD process. We’ve implemented this automated testing solution for several customers — one of which adopted it almost immediately across its teams in the US and Europe. 

Example: Artillery.io 

Flux7’s ECS Container Accelerator is a solution to simplify and speed up the cloud journey with a ready-to-deploy reference infrastructure, pipeline and sample container apps which implement common enterprise best practices. As part of the solution, an application pipeline is deployed to consistently build, test and promote an application from the lower to higher environments. Using Artillery.io, a load and functional testing solution that helps teams ship scalable backends, APIs and services, we stress test the applications. Performance testing is built in to the application pipeline using the artillery.io testing framework to ensure code promoted to production can scale.  

James Wade points out that, “testing as code is not so much about automated testing, but test automation,” a motto we incorporate into our test automation solutions for customers. For, in addition to streamlining processes and growing consistency, test automation can help reduce downtime and reduce errors or failures for end users — both of which can result in revenue impacting customer issues. Test automation decreases time to market by enabling teams to produce higher quality code, faster, and importantly, frees teams to focus on strategic work that has greater impact on the business.

Are you ready to begin experiencing these benefits, but need help identifying where to begin?