At the start of any new software project, you need to focus on formalizing the vision and gathering all relevant requirements. This goes double for business intelligence initiatives -- mostly because one of the main reasons BI projects fail is due to poor requirements definition.
What's so special about gathering requirements for BI projects? Consider that reports, dashboards, and real-time notifications have little or no value unless they enable someone in the organization to make a better decision. Ideally, all decision-making should support the organizational strategy, so an explicit link between BI initiatives and that strategy is essential to success.
BI projects that lack this vital link are likely to wind up with "pretty, but useless" reports and dashboards. All participants must be confident that the right tools will be in place to monitor the effectiveness of the overarching strategy and link it to operational metrics.
Seeing the big picture
The need for strategic alignment is a key reason why BI projects must have support from C-level executives. After all, BI projects tend to be expensive, complex, and cut across multiple departments and hierarchies. Regulatory compliance, data integration (both inside and outside the organization), and the potential need for solutions that are diagnostic and/or predictive in nature may all come into play.
To understand the strategy-driven requirements for a BI project, an organization may employ any number of approaches and methodologies. Ideally, the strategy has been expressed in a formalized manner, preferably based on an established framework for strategic analysis. The gold standard in this regard is a Porter five forces analysis, but competing approaches are legion. I am partial to the “value disciplines” defined in "The Discipline of Market Leaders," but there are many other excellent choices.
Once you have some idea of the organization’s strategy, you can start to ask meaningful questions about what decisions will need to be made to most effectively implement that strategy. Once you know what decisions will need to be made and by whom, you are well on your way to understanding a large portion of your BI requirements.
Here's a simple example of strategic alignment in practice. Let's say a manufacturing firm has chosen the “operational excellence/low cost” value discipline (per "The Discipline of Market Leaders"). As a result, the key decisions will involve reducing costs through supply chain optimization, defect reduction, and increasing productivity.
In this scenario, the firm would establish KPIs (key performance indicators) for inventory levels, cycle times through the manufacturing line, defects, machine uptime, and the like. On the other hand, a different firm that has chosen “customer intimacy” as its core value discipline will be more interested in decisions that affect the ability to learn more about customers, engage more deeply with customers using co-creation, and optimize the lifetime value of the customer through cross-selling, up-selling, and professional services. This firm will establish KPIs based on such factors as repeat sales, net promoter score, customer contacts, and profitability per customer.
A comprehensive BI initiative will also encompass an analysis of how data and insights gleaned from the BI solution are used to validate the chosen strategy. These issues fall into the domains of performance management and organizational development. For example, a firm might decide to use an approach such as balanced scorecard, growth-share matrix, or applied information economics for evaluating strategy and determining the value of improved information.
Cross-cutting concerns and stakeholders
The next major point in dealing with the unique requirements of BI is addressing the “cross-cutting concern” factor. Simply put, to achieve maximum value from a BI project, you must ensure that all stakeholders have been identified and each perspective has been taken into account, regardless of where those stakeholders fall on the org chart.
Let's return to the manufacturing company example. The primary data need of a foreman may be metrics about maintenance time for a series of machines on the manufacturing line because their key role in implementing the corporate strategy is to keep productivity up by reducing downtime. A division-level VP in the same firm may be more concerned with an aggregate productivity calculation on a per-plant basis. In this case, we have two employees working toward the same goal, but with quite different data needs. Developing a BI solution without carefully gathering needs from both stakeholders -- among others -- would be a mistake that diminished the value of the project.
A corollary to this issue is access control. Many firms tightly restrict access to data and enforce an overly rigorous “need to know” policy. While it's important to protect sensitive information, it's equally important to ensure that a BI system serves the needs of all users, regardless of their rank in the organization. Line-level workers, who are closest to problems when they develop, are often best positioned to address those problems and make key decisions. No one is served by preventing them from accessing crucial data.
Regulatory compliance is another concern. This is especially important in companies that are publicly traded and/or in heavily regulated industries such as finance, pharmaceuticals, medical devices, and so on. For these firms, it's essential to consult the compliance department, the CFO, legal counsel, and other stakeholders who deal with regulatory issues. Failing to do so can result in fines, product recalls, or worse.
Finally, BI projects also depend on a thorough approach to data integration. Needed data may reside in multiple operational data stores, in existing analytical databases, or even outside the firm in the form of open data.
In the worst-case scenario, a firm may not have a “single version of the truth” for important records. How many companies still have a shipping and logistics database with different data for a given customer than the Customer Relationship Management system does? In these environments, it would be ideal to implement Master Data Management before building BI systems, but this is not always practical.
In such a case, the onus falls on the BI team to gain a complete understanding of the existing data schemas and carefully manage the ETL flow into the data warehouse/data lake, to ensure that reports and dashboards can be trusted by all users across the enterprise. Nobody wants to walk into a meeting with a report that says one thing about an important issue, while somebody with a similar report from a different system tells a completely different story.
Keys to success
BI projects have a tendency to be bigger in scope and expense than initial forecasts. The requirements phase often determines whether a BI project will help move the business forward or simply be viewed as a waste of time and money. To sum up, here are the keys to gathering requirements for a successful BI project:
- Ensure a clearly defined corporate strategy is in place and develop an understanding of how the BI initiative will serve that strategy.
- Develop an understanding of what decisions will be made based on the BI system.
- Evaluate the needs of all stakeholders, regardless of their rank or level on the organization chart.
- Establish KPIs and metrics that support decision-making among the stakeholders.
- Consider the overlap between BI and performance management.
- Take regulatory compliance into account and ensure the appropriate stakeholders are involved in the discussion.
- Understand the current data environment and its impact on the BI solution. If possible, implement MDM before BI. Otherwise, work diligently to address data quality in the data warehouse/data lake.
Addressing these issues as you gather requirements won't ensure success, but failing to do so will almost certainly guarantee that you won't enjoy the maximum return on your investment. A properly realized BI initiative can pay massive dividends for your enterprise, so make sure to nail the requirements and build a system that supports your strategy across, up, and down the organization, while serving the needs of everyone on your team.