Typically, when we work with big data, we think in terms of millions of rows. Conventional business intelligence tools consume data stacked this way to generate charts, reports, and dashboards -- often dealing with fewer than two dozen columns.
But in many areas, particularly those related to scientific research, new insight can be gained by interacting with hundreds of columns and pivoting to various views of the same data sets in three dimensions. That's what cancer researchers are doing at the Institute for Cancer Research using the latest 3D big data visualization technology. According to researchers there, the potential for accelerating analysis is profound.
[ Download the Big Data Analytics Deep Dive by InfoWorld's David Linthicum for a comprehensive, practical overview of this booming field. | Harness the power of Hadoop with InfoWorld's 7 top tools for taming big data. ]
A private nonprofit institution within the Department of Oncological Sciences at the University of Turin, the Institute for Cancer Research contributes to the fight against cancer by understanding the basic mechanics of the disease and by providing optimal diagnostic and therapeutic services. The institute performs both basic research and clinical research, placing it at the intersection of molecular biology and medicine.
Researchers have struggled to make sense of the sheer volume of their research data points. For example, a genomic data set has a matrix of values with thousands of rows (gene values) and hundreds of columns (single samples/patients), normally represented as a heat map.
Traditional visualization approaches have proved unwieldy and limited, because they offer only partial views of the data sets. The problem multiplies when several data sets must be compared and correlated. As a result, it took several weeks of highly specialized personnel, including experienced bio-software analysts, to extract valuable information from data sets and related annotations.
But such work is essential. Based upon the results, doctors and biologists develop hypotheses of causes of the disease, which ultimately require investment of resources for in-depth research in order to identify therapeutic targets.
Visualizing cancer data. The two walls (floor and right side) are variables in a hierarchy (genes on the floor, patients on the wall). When a researcher selects one or more items on a wall, the mean/average values for the active data set of all on all items in the other wall are computed. For example, a researcher could select a given group of patients and instantly get the average value for each gene (and group of genes) in those patients. It works both ways: Select one or more genes, and the average value for each patient and group of patients appear.
Researchers sought a way to slash the timelines of analysis and allow unsupervised, fast data analysis. They found a solution in a product from Kairos3D called GenomeCruzer, powered by its Gilgamesh big data 3D visualization engine. GenomeCruzer provides an environment where the whole data set can be visualized and explored, together with its data patterns and relations. "The amazing thing is the speed at which we are exploring huge datasets and discovering features we never noticed before, and this is certainly a confirmation of the tool's potential," says researcher Dr. Enzo Medico.
Ultimately, the tool represents a shortcut to data interpretation, bringing tasks that took weeks down to minutes. At the same time, interactive visualization makes the analysis process accessible to a wider range of researchers, even those with no bio-software skills, such as biologists and physicians. By expanding the range of potential analysts to study the cancer data while slashing analysis time, 3D big data visualization has the potential to dramatically increase the volume of cancer research and shorten the path to cures.
This article, "Fighting cancer with 3D big data visualization," was originally published at InfoWorld.com. Read more of Andrew Lampitt's Think Big Data blog, and keep up on the latest developments in big data at InfoWorld.com For the latest business technology news, follow InfoWorld.com on Twitter.