The result? Individual companies ended up in the database 700 or 800 times, making the system even slower and less accurate.

Unfortunately, the application was so deeply embedded in the company's other systems that management was reluctant to spend the money to rip and replace. Finally, the carrier's IT department made the business case that the company's aging data app would ultimately prevent it from being able to add new customers, costing it $750,000 a day in new premiums.

At that point, the company used SSA-Name3 by Identity Systems to clean the data, ultimately weeding out 36,000 duplicate records.

Dupes are one of those problems that keep IT managers up at night. The larger your database, the worse the problem usually is, says Ramesh Menon, a director at Identity Systems, which provides identity searching and matching software for organizations such as AT&T, FedEx, and the Internal Revenue Service.

Unfortunately, nobody knows how big their problem is, he says. “If anybody tells you 'I have exactly 2.7 percent duplicates in my customer database,' they are wrong.”

There's no magic bullet, either. Menon says the solution lies in using data matching technology to isolate “the golden record,” a singular view of information across multiple data repositories. Even then, the hardest part may be getting all the vested parties in an organization to agree on what data they're willing to share, as well as what constitutes a match.

“Two different sections of the same organization may have completely different definitions of what a match or duplicate contact is,” he says. “These kinds of integrations fall apart because people can't agree about who owns the data or what information can be exchanged with others.”

4. When data decays
Remember text-based adventure games such as Zork? Apparently, somebody somewhere is still making these things. Worse, they're using data that's equally ancient.

MailChimp co-founder Ben Chestnut tells the story of an old-school games developer that used MailChimp’s e-marketing service to contact 10,000 previous customers, alerting them that he'd finally finished version two. Most of the addresses were at least 10 years old -- some of them Hotmail accounts discarded so long ago that Microsoft was using the addresses as spam traps. Within a day, all MailChimp e-mail was blacklisted by Hotmail's spam filter.

Fortunately for MailChimp, the developer had kept pristine records, down to the IP address each customer had used to download his games. That's what saved them, says Chestnut. “We fired off a quick note to Hotmail's abuse desk -- proved they were legitimate customers, just old. The next day we got delisted. That's pretty rare.”

All data ages quickly, but contact data ages faster than most.

“You have to make the assumption that data decays like a radioactive sample,” says Informatica's Parthasarathi. “You have to go into every system and periodically update it.”