Study documents data boom

WASHINGTON - If you're feeling overwhelmed by information overload, you may not be alone. The amount of new information stored on various media such as hard drives has doubled in the past three years, to five exabytes of new information produced in 2002, according to a study released Tuesday by the University of California, Berkeley.

That's exabytes, as in one byte with 18 zeros behind it, six zeros more than a terabyte. The amount of information put into storage in 2002, five exabytes, was equal to the contents of a half a million new libraries, each containing a digitized version of the print collection of the entire U.S. Library of Congress, according to the study by professors Peter Lyman and Hal Varian of the UC Berkeley School of Information Management and Systems. The professors estimated that between two and three exabytes of information was generated in 1999.

Most of that data -- 92 percent of it -- was stored on magnetic media, primarily hard drives, the study estimates.

The study, a follow-up to a 2000 study by UC Berkeley, doesn't dwell on how people and companies process these massive amounts of information coming at them, Lyman said, but his next goal is to produce a study examining that very issue. "I'm going to spend the next year on the consumption of information," he said. "How do people make sense of this? How do they cope?"

The current study doesn't address the quality of information and how people choose good information sources, he added. Significant differences exist in the "accessibility and usability and trustworthiness" of information between various sources, Lyman noted. "We treated it all the same, simply to understand how much there was ... but when you get into consumption, the discrimination over the quality of information, and how you make that decision, really becomes important," he added.

With the amount of stored information growing at a rate of about 30 percent a year, a "real change in our human ecology" is taking place, said Lyman, who presented the study at a conference in Florida Tuesday. "Everything is public," he said. "Everything is on the record."

One problem with all this information being stored is that it's not always accurate, he added. As information passes through multiple hands, it can be condensed or mischaracterized. So commentaries or reports on a speech or a paper Lyman gave 20 years ago sometimes contain distortions, he said.

"There are multiple renditions, only one of which I remember," he added.

The study underscores the need for companies to smartly manage their information, said Gil Press, director of corporation information at EMC Corp., an information storage vendor and a sponsor of the study. But IT solutions aren't the only answer, because humans still need to look at information with a critical eye, he added.

"We are getting swamped, and we need better ways to organize and manage information," Press said. "Hopefully, information technology will never replace smart thinking and the human analytical thinking."