What's unclear, though, is how effective these programs have been in identifying and stopping potential terrorist threats such as this latest bombing attempt in New York.
Critics of such programs argue that data mining for terrorists is essentially an exercise in futility given the vast amounts of data that would need to be sifted through on a daily basis, the lack of historical data upon which to base predictions, and the lack of information on patterns that point to terrorist activity.
Bruce Schneier, a noted security guru and chief security technology officer at BT, has long argued that using data mining approaches to search for potential terrorists is akin to searching for a needle in a haystack.
"Data mining works best when there's a well-defined profile you're searching for, a reasonable number of attacks per year, and a low cost of false alarms," Schneier wrote in a 2006 blog post. It's an approach that works well in areas such as fighting credit card fraud where fraudulent patterns are fairly easily discernable, he said.
In the terrorism context, a data mining program can be vital in searching for more information and context on a specific, already identified individual such as Shahzad. But the much larger volumes of data that would need to be sifted through on a daily basis to identify the rare, potential terrorist greatly increase the possibility of false positives and negatives, Schneier said.
Schneier calculates that even the most accurate and finely tuned data mining system will generate one billion false alarms for every real terrorist plot it uncovers.
Similar concerns prompted the National Research Council to issue a report in 2008 calling pattern-seeking data mining tools too unreliable for identifying potential terrorism suspects.
The continued and unchecked use of such tools poses potential privacy problems for uses, the NRC had noted in its 376-page report which was prepared partly at the request of the U.S. Department of Homeland Security.
James Lewis, director and senior fellow at the Center for Strategic Center for Strategic and International Studies, and leader of a team that developed a set of cyber security recommendations for President Obama said its hard to pass verdict without more information.
"We'd have to know if data mining had missed all plots or just this one," Lewis said. " If it catches some but not all, then the question is does it catch enough to justify itself.," he said.
"One case isn't enough to tell and my problem with the critics is that they have an agenda - usually privacy - and see everything through that lens," Lewis said. "There is no single solution but data mining might be a useful part of the package. We don't have enough data to know," he said.
Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld . Follow Jaikumar on Twitter at @jaivijayan or subscribe to Jaikumar's RSS feed. His email address is firstname.lastname@example.org .
Read more about BI and analytics in Computerworld's BI and Analytics Knowledge Center.