Speeding  up enterprise search

There’s no overstating the importance of search to your public Web or internal servers. Even the best-architected sites have grown so complex that more than two-thirds of visitors use site search as their primary form of navigation, research company IDC reports. 

Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld Verity Ultraseek 5.3.1 Verity, verity.com Very Good  8.6 criteria score weight Ease-of-use 8 20% Integration 9 20% Management 8 20% Performance 9 20% Scalability 9 10% Value 9 10% Cost: Starts at $10,000 for 50,000 documents; CCE starts at $4,995 Platforms: Suse Linux 8.0; Red Hat Enterprise Linux 3; Red Hat Linux 7.1; Solaris 7 and 8; Windows Server 2000, 2003 Server Bottom Line: Ultraseek requires little administration and delivers excellent results when compared with other search products. A simple browser-based GUI helps IT staff manage crawls and remote-search servers. The system is very scalable, integrates well with other Verity applications, and combines searches from back-end enterprise systems. About our Reviews and Scoring Methodology

Yet making the right search choice can be equally complicated. There are dozens of options, ranging from low-end products that yield imprecise results to intricate solutions that crawl all types of enterprise data repositories and take years to implement.

In fact, major search vendors declined to submit their enterprise products to InfoWorld for testing, saying setup would be too complex. Among the no-shows was Verity’s K2 Enterprise with its intricate taxonomy and classification functions.

Nevertheless, I was able to get a good sense of Verity’s enterprise search approach with a three-month test of Verity Ultraseek, a lighter version of the technology found in the high-end K2 Enterprise. My conclusion: It’s a very good choice for Internet, extranet, and intranet searches. 

Version 5.3.1 improves search results and their display with new options, including Page Expert, which filters out irrelevant page content. The updated Ultraseek is a good fit for large deployments because IT can delegate to business managers admin tasks such as revising search forms to match the requirements of different sites.

Deep searching

After you get Ultraseek running -- a trivial process -- you can almost forget about touching the software. The latest version now supports Suse Linux and Red Hat Enterprise Linux; my test platforms employed both Red Hat Enterprise Linux 3 and Windows Server 2003. 

Customization is performed via a Web-based UI divided into functional sections. Although Ultraseek provides a lot of options for you to customize, creating a basic search isn’t difficult: I merely entered a starting URL and completed a second form with the types of documents to crawl.

I was pleasantly surprised with the usability and depth of Layout Manager, a great GUI for designing the search interface. For example, I employed the same CSS style sheet across any given site to maintain a consistent look for search results, and I could select default collections and categories.

Unfortunately, my initial results were not as good as those I got from search appliances. For example, on identical site crawls, Ultraseek returned 20 percent false hits compared with none for both the Google and Thunderstone appliances. The good news is that I matched those results after configuring two new Ultraseek features, Page Expert and Secure Result Filtering.

The Page Expert function helped me exclude noncontent text (such as navigation and menus), which seemed to confuse Ultraseek’s search algorithms. Next, I used Secure Result Filtering, which checks each hit against the current user’s permissions and then presents only allowable results. This works with basic HTTP, HTTPS, and NTLM (NT LAN manager) security, so it should be adequate for portals and other similarly customized sites.

As do most search software solutions, Ultraseek provides editorial control. For example, editors can allow keywords to be associated with specific URLs that appear above the normal search results. Making these connections with Ultraseek, however, is much easier than it is with Thunderstone.

One interesting addition in Ultraseek 5.3.1 is Weblog Update. From a preference setting in Six Apart’s Movable Type 3.12, I easily notified my Ultraseek server that blog sites had been updated and triggered it to immediately crawl the changes. With blogs becoming increasingly important as a source of information, this feature ensures that visitors’ searches will include the most recent posts.

Corralling configuration

This release also improves Ultraseek’s already strong enterprise emphasis. For example, I set up several connections to a Web server with the Multi-Spider tool, which resulted in faster spidering of hosts with lots of pages. (The previous version permitted one connection per host.) Another useful feature allows you to crawl development or staging servers to optimize crawl settings. You can then import that information into Ultraseek to quickly create a new collection, reducing administration effort.

Right out of the box, Ultraseek groups results by site, which helps users isolate information they need. Although it’s not part of the standard configuration, administrators may want to consider adding Ultraseek CCE (Content Classification Engine), which puts search results into easy-to-navigate topic hierarchies.

This step isn’t automatic, as it is with Vivisimo Velocity, but CCE will allow you to take an existing site map and use that to build topic trees, for example. Along the same lines, Ultraseek gets a nod for incorporating searches of enterprise Web applications. Integration with portals such as BEA and Vignette is quickly accomplished using prebuilt portlets.

Ultraseek’s easy implementation and customization, combined with its relevance tuning, mean you can quickly implement a Web search solution. The software provides a range of competitive functions, including spelling suggestions, secure search, and multilanguage support.

Integration is equally well-executed: Near the end of testing, I upgraded Ultraseek to the new Verity Enterprise Web Search software, which allows users to simultaneously search internal repositories and all relevant Web content indexed by Yahoo Search.

Moreover, Ultraseek’s indexes can be employed by K2 Enterprise. This should reduce rollout time when IT managers want to deploy a full-scale Verity enterprise search solution that melds Web search results with those from databases, CRM systems, and other in-house systems.