Agile testing in regulated industries

Agile testing principles—such as BDD—align stakeholders around software quality and compliance

Last year, 49 percent of financial services organizations suffered a breach, according to the 2017 Thales Data Threat Report, Financial Edition. A fifth (21 percent) of organizations surveyed reported being breached more than once. And nine in ten respondents said they now feel more vulnerable to such incidents.

Security breaches, played out time and again in other sectors such as health care and retail, have placed millions of people at identity theft risk and dealt major blows to the public image and balance sheet of large companies.

In the world of medical devices, cyber security breaches are rare, but the potential impact is frightening. In 2017, the US Food and Drug Administration (FDA) recalled nearly half a million Abbott pacemakers to address a vulnerability that could allow hackers to run the batteries down or even alter the device’s pacing commands. Consider the power that a savvy, politically motivated hacker could have over the life of a VIP, such as a head of state, Fortune 500 CEO, or prominent member of law enforcement.

When the stakes are this high, prioritizing software quality is nonnegotiable. Many organizations are modernizing development and testing processes to sharpen their focus on quality. Agile and devops practices are gaining favor for testing in regulated businesses, despite the misperception that modern software development and testing methods don’t produce enough of a paper trail for audits.

On the contrary, agile minimizes unnecessary documentation and processes to create a rapid feedback cycle that can evolve with changing demands and requirements. Beyond promising faster time to market, many core agile practices are specifically designed to improve software quality. And this focus on quality offers distinct advantages to teams developing software in regulated industries.

Progressive development organizations in health care, banking, and other regulated industries are applying agile principles such as behavior-driven development (BDD) and acceptance-test-driven development (ATDD) to put quality at the forefront. I’ve seen many of our customers, including Fortune 500 financial services organizations, both streamline their development processes and strengthen audit trails with BDD.

1. BDD forces collaboration on requirements

BDD is an agile development method that helps teams efficiently deliver the features that matter most to the business. It involves tight collaboration and communication among business stakeholders, product owners, developers, and testers to discover, understand, and translate business needs into technical requirements before development begins.

BDD test scenarios differ from traditional test scripts in that they are written using a syntax that is similar to the English language. That means it’s easier for business people to verify that developers and testers understand their needs, and there’s less risk for requirements to get lost in translation as they are passed from business to engineering. These factors are especially important in highly regulated environments where noncompliance can result in hefty fines.

If your organization faces regular audits, applying the concepts of BDD may help your team maintain traceability throughout the development cycle, which has simplified the audit process. BDD prescribes that tests are written early and that they are based on specific requirements. This traceable link between test and requirement helps prove that the software has been adequately tested in a way that is easy for auditors to understand.

2. ATDD offers precision business acceptance  

When you build new features for the business, your goal is from them to be accepted by the business. However, the stage of acceptance testing, or validation, from the business usually occurs later in the development life cycle. Even if the development accomplished functional testing earlier, you still tend to force the business stakeholders to test near the end of your development. This is a risk you don’t want to take because it could delay your releases moving to production.

Combining ATDD with BDD can be a surefire way to ensure that requirements are indeed accomplished in the finished product. ATDD brings together business stakeholders, testers, and developers to define automated acceptance criteria before code is written. Therefore, when the automated tests fail, they provide feedback that the requirements are not being met. When they pass, this indicates that acceptance had been met at the beginning rather than at the end of your development cycle.

3. BDD crowdsources domain expertise

Shifting to a more collaborative process has enabled testers to come forward as user experts, which in turn bridges the gap between business and technical requirements. There is tremendous value in recognizing testers’ unique insight into user experience. Out of everyone involved in the software development life cycle, testers arguably spend the most time interacting with the software the way that a user would. And usability becomes extremely important when consumers are interacting with your applications to access health information or perform financial transactions.

ScienceSoft suggests that acquiring domain knowledge is one of the most important jobs of testers in regulated industries. Testers must know enough about the regulations governing their industries to understand which parts apply to the software they are testing and ensure proper test coverage.

In addition to leveraging testers’ domain expertise, BDD and ATDD crowdsource domain knowledge from stakeholders across the business—including regulatory experts, business leaders, product owners, and developers. This makes it easier for regulated businesses to ensure that the correct rules are included in the test case design to guarantee proper test coverage, regulatory compliance and user needs.

4. BDD increases traceability

Regulatory clearance often requires demonstrating traceability between requirements, tests and software artifacts. BDD tools, such as the open source Cucumber, also offer ways to document automated testing, a process that isn’t always the de facto standard in nonregulated industries. If you are automating tests in BDD, test cases, which are written in plain English, become documentation directly tied to the automated tests that were written for it. This makes it easy for nontechnical stakeholders to understand exactly what each script was testing for, and to quickly compare the test cases to both requirements and results. Tools that integrate with open source automation frameworks can simplify this process by providing a single view of all testing activity, giving teams a simple way to report that the software has satisfied each requirement.

BDD and ATDD can streamline development by aligning stakeholders in advance, crowdsource domain knowledge to ensure both regulatory compliance and a positive user experience, and create a thorough audit trail. Yet switching to these agile methods means retraining testers to write test cases in a different way. Leaders must understand and make room for a learning curve and provide the appropriate technology to support it.

Copyright © 2018 IDG Communications, Inc.