Bringing embedded analytics into the 21st century

Analytic tools haven't kept up with the revolution in how software is built, especially in embedded analytics. But the tide is starting to turn

carbon black streaming dashboard
Credit: Carbon Black

Software development has changed pretty radically over the last decade. Waterfall is out, Agile is in. Slow release cycles are out, continuous deployment is in. Developers avoid scaling up and scale out instead. Proprietary integration protocols have (mostly) given way to open standards.

At the same time, exposing analytics to customers in your application has gone from a rare, premium offering to a requirement. Static reports and SOAP APIs that deliver XML files just don't cut it anymore.

And yet, the way that most embedded analytics systems are designed is basically the same as it was 10 years ago: Inflexible, hard to scale, lacking modern version control, and reliant on specialized, expensive hardware.

Build or Buy?

It's no wonder that today's developers often choose to build embedded analytics system in-house. Developers love a good challenge, so when faced with the choice between an outdated, off-the-shelf solution and building for themselves, they're going to get to work.

But expectations for analytics have increased, and so even building out the basic functionality that customers demand can sidetrack engineers (whose time isn't cheap) for months. This is to say nothing of the engineer-hours required to maintain a homegrown system down the line. I simply don't believe that building it yourself is the right solution unless analytics is your core product.

So what do you do?

Honestly, I'm not sure. Given the market opportunity, I think it's inevitable that more and more vendors will move into the space and offer modern solutions. And so I thought I'd humbly lay out 10 questions embedded analytic buyers should ask about the solutions they're evaluating.

  1. How does the solution scale as data volumes grow? Does it fall down or require summarization when dealing with big data?
  2. How does the tool scale to large customer bases? Is supporting 1,000 customers different than supporting 10?
  3. Do I need to maintain specialized ETLs and data ingestion flows for each customer? What if I want to change the ETL behavior? How hard is that?
  4. What's the most granular level that customers can drill to?
  5. Do I have to pay to keep duplicated data in a proprietary analytics engine? If so, how much latency does that introduce? How do things stay in sync?
  6. Can I make changes to the content and data model myself or is the system a black box where every change requires support or paid professional services?
  7. Does it use modern, open frameworks like HTML5, Javascript, iFrame, HTTPS and RESTful APIs?
  8. Does the platform offer version control? If so, which parts of the platform (data, data model, content, etc.) are covered by version control?
  9. How customizable is the front-end? Can fonts, color palettes, language, timezones, logos, and caching behavior all be changed? Can customization be done on a customer-by-customer basis or is it one template for all customers?
  10. How much training is required for admins and developers? And how intuitive is the end-user interface?

No vendor that I know of has the "right" answer to all these questions (yet), but they should be taking these issues seriously and working toward these goals.

If they're not, you can bet your engineers are going to start talking about how they could build something better in a week. HINT: They actually can't, but good luck winning that fight ;-)

This article is published as part of the IDG Contributor Network. Want to Join?