The researchers make the point that most estimates of damage are reached via surveys. Using surveys seems like a good strategy until you realize that researchers start with what appears to be a hard number provided by respondents, then extrapolate to a larger population: "Suppose we asked 5,000 people to report their cyber crime losses, which we will then extrapolate over a population of 200 million. Every dollar claimed gets multiplied by 40,000. A single individual who falsely claims $25,000 in losses adds a spurious $1 billion to the estimate. And because no one can claim negative losses, the error can't be canceled" through averaging, as happens somewhat when people choose from ranges.
They go on to say the cyber crime surveys they've examined "exhibit exactly this pattern of enormous, unverified outliers dominating the data. In some, 90 percent of the estimate appears to come from the answers of one or two individuals," Florencio and Herley state.
If you've ever done testing, you know it often makes sense to discard outliers in your results -- a practice you should've learned in introductory statistics classes. I have to assume that the people who conduct the self-servingly skewed surveys probably know it too, but choose not to bother. As we used to joke in our newsrooms: Never let facts spoil a good story.
How common is this upward bias in surveys? "Among dozens of surveys, from security vendors, industry analysts, and government agencies, we have not found one that appears free of this upward bias. As a result, we have very little idea of the [actual] size of cyber crime losses."
In case you're wondering why "sex" appears in the report title, it wasn't just to sensationalize the survey. Florencio and Herley liken the reporting of cyber crime to the reporting of the number of sexual partners claimed by survey respondents. "Cyber crime, like sexual behavior, defies large-scale direct observation and the estimates we have of it are derived almost exclusively from surveys," they say. And both topics lend themselves to exaggeration.
None of this is to suggest that cyber crime is not a problem. It is, of course. But the researchers note that in most cases, stolen passwords and other data are sold for pennies on the dollar, which is to say they're hard to monetize.
Even if they don't translate into the losses claimed by the self-interested security industry, there is a real price to be paid from the misuse of these surveys: Exaggerated stories of the size of profits dervived from cyber crime not only scare users unnecessarily, they fool novice hackers into to thinking they'll get rich quick. So they try.
This article, "Bad stats sink cyber crime costs claims," was originally published by InfoWorld.com. Read more of Bill Snyder's Tech's Bottom Line blog and follow the latest technology business developments at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.