The drive toward real-time business intelligence

Turning business intelligence into a real-time proposition is a key to unlock digitalization. A number of challenges abound on that path

internet speed fast runners running relay race
Credit: Shutterstock

In 5 steps for transforming your business using data, we have identified real-time business intelligence as one of the first steps toward the digitalization of business. But what does real-time (or right-time) business intelligence really mean, and what does it entail?

Real-time data collection

Real timeliness in business intelligence starts with a timely collection of the inputs and stimuli to the business intelligence processes. Of course, not all inputs need to be collected in real time -- for reference information like product catalogs, customer details, batch processing (on a daily basis for example) remains perfectly suitable. However certain data points need to be updated much more often, in order to have an impact on operations. Depending on the type of business, this may include customer orders, geolocalization data, website actions, financial transactions, etc.

This requires that jobs used to collect and transform data from the sources, be modified to support faster data velocity. If for your business, the right-time upload of customer transactions means they must be processed every five minutes, you should be able to use traditional high-performance batch ETL technology with jobs running frequently on smaller data sets. If however you must collect geolocalization information with a 10-second maximum lag, then streaming or messaging technology will become a requirement.

Real-time processing

Beyond the technical challenges of running smaller/faster batches and streaming/messaging technology, the major difficulty of real-time business intelligence resides in the differences in velocity between the different types and sources of data. Because not all data enters the data warehouse/mart/lake at the same time, and because it is not refreshed at the same frequency, updating the data analysis and reference structures becomes very challenging.

With the requirement for real-time processing, reports, graphs, and other outputs often need to be refreshed on-demand. Not every data set can be pre-calculated, and drill-down requests from users are expected to run much faster than what is usually the case.

Another challenge real timeliness brings is the unstability of the data set. Between the moment a report is produced and the moment a drill-down is requested, the underlying data may have been updated, rendering the two outputs inconsistent.

Real-time insight availability

Unlike reports-based business intelligence, where pre-defined reports, graphs and charts are produced and published at predefined time intervals, real-time business intelligence must make its outputs available on-demand. Users, but even more often programs and applications, need to be able to request specific information by making calls to the business intelligence platform. Historically built on a cumbersome Service Oriented Architecture and using SOAP Web Services, most platforms evolve toward the REST architectural style: Web APIs running over a standard Web infrastructure, that invoke platform-side programs and routines and return their output as JSON or XML data sets -- to be consumed by the requesting application.

Real-time use of insight

Of course, the key to success in real-time business intelligence is not the underlying technology, but how the resulting insight will be used.

Reports and graphs, the traditional insight provided by business intelligence and consumed in the boardroom, is suddenly a lot less relevant. Of course, they remain essential to measure the performance of the business. But the individuals most impacted by this new, faster insight, are the knowledge workers and the people in the field, who will be able to either make faster and more informed decisions, or whose tasks/routes/actions/etc. will be modified in real-time based on these insights: the truck driver who gets rerouted to collect a shipment from a tier-one client, the manufacturing line manager who places a resupply order based on real-time order information.

Which poses the next challenge of digitalization: How do you inject this insight, these real-time analytics, into business processes. Stay tuned for more on this topic.

This article is published as part of the IDG Contributor Network. Want to Join?

From CIO: 8 Free Online Courses to Grow Your Tech Skills
Notice to our Readers
We're now using social media to take your comments and feedback. Learn more about this here.