Verity, ClearForest add structure to data

Verity this week will unwrap a software add-on to its search system, designed to make unstructured content more usable in corporate applications. The announcement follows activity from ClearForest, which last month introduced Version 6.0 of its Text Analytics platform for systematically structuring unstructured data so it can be processed with enterprise data in business intelligence systems.

Free IT resource Open Source Business Conference (OSBC) May 22-23, 2007 Sponsored by OSBC Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft

The Verity Extractor 1.0 can identify and pull out patterns and concepts in unstructured content, thereby increasing the relevance of enterprise search and the accuracy of automated classification, according to David How, senior product manager at Verity.

One of the biggest challenges in search and data management is making full use of unstructured data, How said.

According to some estimates, unstructured content comprises nearly 80 percent of corporate data. “Of all the data out there, most of it is unstructured, which means you aren't taking full advantage of the data. By pulling out the structure in unstructured [information], you are adding value,” How said.

Extractor 1.0 identifies patterns, concepts, and entities in unstructured data, and by adding structure, allows this type of information to be used in a database, enterprise application, or content management system.

Out of the box, the software can recognize 14 entities, according to Verity officials, including people, places, phone numbers, dates, prices, metric measures, ZIP codes, or addresses. Custom entities can also be built using Extractor.

Because entity extraction creates additional metadata around the context of a search, "it improves the richness and accuracy of the search process," said Hadley Reynolds, vice president and director of research at Delphi Group.

“The problem has always been that traditional keyword search didn't provide a very rich core of meaning, because all it knew about it was the occurrence of words," Reynolds said.

Extractor 1.0 features C and Java APIs for integrating the software with Verity's search, classification, and recommendation technologies.

ClearForest's Text Analytics 6.0 lets companies analyze unstructured text and enterprise data simultaneously, thereby allowing both types of data to be used in business intelligence applications.

New features in Version 6.0 include integrated extraction categorization capabilities, new system management software, an administrative dashboard, and upgrades to the development tools. The enhanced extraction features, when used in combination with the ClearForest's Analytics product, provide increased depth and accuracy of text analysis, so unstructured content can be used as a source for enterprise analytics, according to ClearForest officials.