Legacy QA in the devops age: reskill them or release them?

As devops becomes more mainstream, are your legacy teams and processes holding you back from fully making the transition?

teach train direct learn formula stategy charts

The challenge

Devops is becoming mainstream—I know because I am contacted almost daily with unsolicited sales emails from companies trying to partner with me to implement devops. Many companies beginning their devops transition still have ingrained old-school processes, along with legacy QA and development teams.

Some companies believe or are told their QA teams are unskilled and should be eliminated and replaced with either a combined engineering team or an offshore testing staff. Swapping out staff is not as simple as it sounds. You won’t be able to hire or even locate replacements as fast as you want in today’s market, or the contractual prices for offshore talent in some cases could cost more than you are paying for your current QA team.

The solution

I believe there is a better choice and that is to take your existing staff and reskill them. This is an approach outlined by many, including Accenture CIO Julie Sweet (no relation to me). Given some roles stay open for six months or more, investing in your own staff makes smart business sense.

When there are areas of improvement in the QA space in a company, there are often areas of improvement in other groups such as software development, database and infrastructure. Rarely have I seen a development organization that had nearly perfect development processes paired up with a primarily manual-test focused QA team—the point being everyone needs to practice continual improvement.

Most will agree that to be successful at devops, long manual test cycles are not going to work when the goal is to release every day. One of the more obvious answers is implementation of test automation, but shifting testing left in the software development life cycle and reducing the amount of non-value-added work by using techniques such as value stream mapping also need to be included.

Many QA teams know the product functionality well and have been loyal employees for years, but lack basic programming or test automation skills. These skills can be taught. Next, you need to educate the larger team that “QA’s job is to help reduce the introduction of defects” not “QA’s job is to find the defects once they are added.” This means that the QA team is going to need to see the source code.

Areas to focus on

Start with ensuring your development teams perform dedicated code reviews using a tool such as Team Foundation Server or Crucible. Invite the QA team to participate. According to a graph in Predicting Software Assurance Using Quality and Reliability Measures, there is a 35 times higher defect removal density by performancing code review than by system testing. If your QA team is only performing black-box system testing, there is a lot of opportunity to reduce cycle times for your software by looping them into the code review process.

Static code analysis is the next area the QA team can contribute to devops success. QA teams after often metrics driven, though many of the metrics they are asked to produce don’t measure quality or value delivery to the customer. Turn your QA team loose on the results of your static code analysis tools and make the reports visible. There can be reticence to sharing a static code analysis report when it shows individual check-ins adding 30 days of technical debt, and the reports contain thousands of pages of errors, warnings, and security concerns, but hiding this does not make the code any better, nor help your clients. Static code analysis is an objective measure of structural quality and cannot be dismissed. If you hold the QA team accountable for reporting it and making it visible, you will be on a path to reducing the debt. Consider putting gates tied to static analysis results into your continuous integration system to reject check-ins that add technical debt.

Programmatic unit testing is important for any devops transformation, and I put the responsibility for writing those tests solely on the development team. I would like to see them reported on by the QA team, and added to a larger automated test and analytics dashboard.

Functional test automation written below the UI at the component or service layer, limited automated UI testing and performance testing are areas the QA can contribute. Ideally, most of these are written in the same development tools as the developers use, where possible. They should also be stored in the same source code repositories, and meet the same standards for code reviews and static analysis as the production code. Because the overall team all uses a code review process and continuous integration, there should no political argument to a shared code repository or full edit rights for the QA team. These tests should then be added to the appropriate place in the deployment pipeline, so tests are run at regular intervals with the goal of run the tests every build.

As your QA team continuous its devops journey, it can add some more advanced testing tools and techniques such as service virtualization, test data management, mobile device testing in the cloud, and testing in production. As it earns your trust, you will let the team make edits to the dev code, build some of the code for infrastructure-as-a-service, and help you build a truly cross-functional team. 


Woody, Carol, Ellison, Robert, & Nichols, William., (December 2014). Predicting Software Assurance Using Quality and Reliability Measures, CMU/SEI-2014-TN-026, CERT Division/SSD

This article is published as part of the IDG Contributor Network. Want to Join?