Iteration is the best way to find the insight in your data

Iteration is key to success. There is simply no way you can open a dataset and know, ahead of that exploration, how best to display it

Christina’s World, by U.S. artist Andrew Wyeth, is considered an icon of American painting. Do you think the painting arrived in Wyeth’s mind fully formed? Of course not. He sketched his vision over and over again to get the lines of the subject just right. As any artist would with their work, Wyeth honed in on the best articulation of his subject. How can data analysts learn from this process? In the field of data analysis, the same principles of drafting and iteration apply to helping you find and present the best insights in your data.

One dataset, multiple creative articulations

If you provide people with the same dataset, what would their differing iterations look like? How would they explore the data, and how would they present it?

Late last year, as part of the MakeoverMonday project, we asked people to log their exploration of a common dataset (the FAA Wildlife Strikes dataset) by recording a snapshot of their view each time they made a change. The logging process was automated by the software being used.

Like Wyeth’s own creative process, the results show that iteration is a fundamental ingredient to data analysis. Every person created something unique, but the common strand was iteration. Here are the three key lessons we extracted from the project, which apply to all successful data analysis.

Sketching with your data

Having time as a metric in your data doesn’t mean you should make a line chart. And a dataset with countries and cities, doesn’t necessarily mean you need a map. The story in your data needs to be explored and teased out, and then represented in the most efficient way possible.

In the example above, the author tries mapping at the start of their process, but quickly abandons that path in favor of time-based analysis. They explore timelines, heat maps, and connected scatter plots before moving from exploration to formatting. The correct choice only becomes apparent through exploration.

Be open to tangents

You might begin an analysis with one question, but soon get diverted to a more interesting tangent as the data reveals itself. Often, those tangents are more appealing, and lead to some of the biggest cost-saving discoveries for enterprises. It’s what you didn’t expect to find that can be the most valuable. In the exploration above, you can see the final chart emerge, but only after many attempts to find other stories.

There is no distinction between exploration and formatting phases

When you are in the flow, you are at one with your data and your tools—you move freely from exploration mode, into formatting mode, and back again. Data analysis and presentation are different, but it must be possible to switch from one mode to another in an instant. In the example above, the author is mostly focused on getting the timeline just right, but keeps jumping out on tangents, looking at the data this way, then that way, then another, before returning to the timeline. This is flow.  Achieving this rhythm should be the goal of any analyst’s exploration of data.

Iteration is key to success. There is simply no way you can open a dataset and know, ahead of that exploration, how best to display it. Boilerplate templates aren’t feasible because the insights you need are unique to your situation—and only iteration can tease out the best results.

This article is published as part of the IDG Contributor Network. Want to Join?