KPIs for managing and optimizing devsecops success

CISOs need to make devsecops a priority. These KPIs can help align both security and nonsecurity stakeholders as to where, why, and how security fits into the process

Back in 2012, in a report called “DevOpsSec: Creating the Agile Triangle,” Gartner identified the need for information security professionals to become actively involved in devops initiatives. Five years and more than 24 billion Docker image pulls later, that need is now a full-blown imperative. 

As I’ve written in InfoWorld before, devsecops needs to be led by the security team members because they are the ones ultimately responsible for the cyber security posture of the enterprise. Those tasked with implementing devsecops should expect a learning curve as they bond with devops teams, familiarize themselves with concepts such as continuous delivery and tools like Jenkins and Docker, and determine how to best implement devsecops into their organizations.

Devsecops leaders also need to adapt to the reality that devops is all about speed, agility, and automation, while devsecops is about updating and realigning security for today’s technologies, processes, and pace. Even the most security-friendly devops teams may perceive the inclusion of any security measures/controls as slowing the pace of application delivery. 

However, if devsecops teams seize the opportunity to “left shift” security, friction between security and devops (and hopefully, the rest of IT as well) should erode, quickly. That shift is foundational to devsecops success. One key way to justify the business case for devsecops and motivate those forging the righteous path that it’s good for everyone is to measure its effectiveness.

Here are some KPIs I’ve seen enterprises use to take their devsecops initiatives to the next level:

1. Reduction in security related tickets

Delays or hindrances caused by security are probably the No. 1 source of conflict between devops and security teams, which is why the subsequent sets of KPIs are dedicated to issues that cause delays. One of the main goals of devsecops is to minimize the Number of tickets opened to development teams due to security related issues. This KPI should align and gel with the KPIs listed further below and focus on:

  • The number of security-related tickets opened by developers in a given time frame
  • That number as a percentage of total tickets opened.

The desired result is, of course, the decline of these numbers over time. The percentage is especially important since an increase in code delivered or team growth could generate more security-related tickets; however, if that growth is smaller than overall ticket increase, the percentage will expose that to show that security issues are handled better.

2. Reduction in security-related build time delays

Where devops wants speed, security add their own checks: As security teams become increasingly involved in the planning stages of container initiatives, they will work with devops folks and application architects to design security into the actual specs of the application. However, these days, security teams are often unaware of the existence of container initiatives within their organizations until CI/CD process planning (and possibly implementation) is well underway. This requires security to come in ASAP and add their own checks and controls into the build process, slowing down image build times. Over time, things should speed up again, so measuring the reduction in security related build time delays is a very useful and practical KPI for devsecops.

3. Reduction in security-failed builds

One of the main goals of devsecops is to minimize the impact of security on the on speed of application delivery. The way to achieve that goal is to eliminate drag (that is, delays) created by measuring the number of builds that were not completed because they failed the security tests.

4. Reduction in security findings missed in the build process

If an image made it into production and it shouldn’t have, it means that it was missed during the dev process, and you have go back in your pipeline to fix it. We strive to continuously improve security tests to ensure as complete a coverage as possible in the early stages of development. The reasoning is that it’s less time-consuming to discover and fix issues early on than it is to catch and fix them later on. The way to achieve that goal is to ensure that all the required tests are automated within the pipeline, preventing bottlenecks. No one wants backtracking, so set up a KPI to measure the number of “late discoveries” that required sending things back upstream.

5. Reduction in hours spent resolving security issues

This KPI measures hours spent on security in the pipeline. They include:

  • How many people do you have in that take care of security in the devops environment? 
  • How much time do they spend on manual testing or validation? 

The more automated security becomes, the smaller these numbers will be. At the very least, they should confirm and align with the KPIs measuring and backtracking and delays —that is, time spent when things go wrong. If it doesn’t, it means you’re not automating enough.

6. Compliance KPIs

Early adopters of container technology still tend to be to be large, highly regulated companies, so they are usually tightly regulated via internal and external compliance mandates. So, put in place appropriate audit-oriented KPIs and benchmarks such as:

  • Time to compliance: The period it takes to achieve compliance for a new application
  • Percentage of audits passed: This should continuously improve, to 100 percent.
  • Number of corrective measures that needed to be taken post-audit: Aspire to zero.

As someone who builds security solutions for devops shops, I see the same dynamic all the time: CIOs, application architects, and devops folks want to pursue a container- and microservices-based development strategy but are being told they can’t move forward without addressing cybersecurity issues up front.

They are now required to approach cybersecurity strategically, and have the capacity to not only document their cybersecurity due diligence but explain to the CIO and have an action plan for addressing them. It’s that requirement—which is needed for compliance, and to ensure strong internal cybersecurity hygiene—that in great part creates the opportunity to “left-shift” security.

However, for that to happen (and to happen correctly), CISOs need to make devsecops a priority. I hope these KPIs can help align both security and nonsecurity stakeholders as to where, why, and how security fits into the process.

Copyright © 2017 IDG Communications, Inc.

How to choose a low-code development platform