Google rolls out semantic search capabilities

Google takes a first step toward the future of search engine technology

Google has given its Web search engine an injection of semantic technology, as the search leader pushes into what many consider the future of search on the Internet.

The new technology will allow Google's search engine to identify associations and concepts related to a query, improving the list of related search terms Google displays along with its results, the company announced in an official blog on Tuesday.

For example, Google's search engine, upon encountering a query like "principles of physics," now understands that "angular momentum," "special relativity," "big bang" and "quantum mechanics" are related terms, the company said.

Ori Allon, technical lead of Google's Search Quality team, said in an interview Tuesday that the search improvement involves a dollop of semantic search technology mixed in with a big helping of lightning-fast, on-the-fly data mining.

"This is a new approach to query refinement because we're finding concepts and entities related to queries while you do a search, so everything is happening in real time and not [pre-assembled]," he said. "Because we're doing it in real time, we're able to target many more queries."

The use of semantic search isn't more broad at this point because full conceptual analysis of documents would slow down the process of generating query refinements on the fly, Allon said. "If we want to get it all done in a matter of milliseconds, there's a lot of innovations we still have to do. A full semantic search would be very hard to do in this limited amount of time," Allon said.

This is a big stumbling block that semantic search engines often run into: scaling their technology to the speed and volume of a massively used service like Google's.

"We're working really hard at search quality to have a better understanding of the context of the query, of what is the query. The query isn't the sum of all the terms. The query has a meaning behind it. For simple queries like 'Britney Spears' and 'Barack Obama' it's pretty easy for us to rank the pages. But when the query is 'What medication should I take after my eye surgery?', that's much harder. We need to understand the meaning," said Allon, who came to Google in 2006 when the company hired him after his 'Orion' Ph.D. project on search engine technology caught Google's attention.

Offering query-refinement suggestions is but the first application of the technology behind these enhancements, so users can expect other concrete improvements applied to things like search ranking, he said.

"The main Google infrastructure now is able to have a better sense of what is the context of the query and what are its related concepts, and how they relate to each other," he said. "So this is the first of what we hope will be many other applications that we're working hard to incorporate into search quality."

Google has often been criticized for using what is considered an aging approach to solving search queries based primarily on analyzing keywords and not on understanding their meaning.

Google executives over the years have acknowledged that semantic search technology will be an important component of search engines in the future.

"Right now, Google is really good with keywords and that's a limitation we think the search engine should be able to overcome with time," Google Vice President of Search Products - User Experience Marissa Mayer said in an interview with IDG News Service in October 2007. "People should be able to ask questions and we should understand their meaning."

She cautioned, however, that Google sees semantic search technology as part of the algorithmic mix, not as a replacement to its traditional keyword-analysis approach.

"I think the best algorithm for search is a mix of both brute-force computation and sheer comprehensiveness and also the qualitative human component," she said.

In January of this year, during Google's fourth-quarter earnings conference call, CEO Eric Schmidt touched briefly on this topic, hinting that the company is getting more serious about semantic search technology. "Wouldn't it be nice if Google understood the meaning of your phrase, rather than just the words that are in the phrase? We have [done] a lot of discoveries in that area that are going to roll out [soon]," Schmidt said.

There is an entire field of Google competitors that are busy developing and perfecting semantic search engines, betting that they will be able to deliver on the promise of this technology: to let users type in queries in natural language and have the search engine understand their meaning and intent.

Microsoft last year acquired Powerset, one of these companies, in order to improve its Web search engine with semantic search technology.

Google also rolled out on Tuesday another enhancement to its search engine: longer "snippets," which are the text excerpts Google extracts from Web sites to show in search results where the query keywords appear.

Critics have often pointed out that these excerpts aren't very useful in previewing enough context so that users can decide whether to click over to the Web site.

Now, when people enter queries that are three words or longer, Google will deliver longer snippets in order to provide users with a better view as to how their query keywords appear on the Web site.

It remains to be seen if Web site publishers will cry foul over longer snippets. In the past, publishers have sometimes complained that search engine abstracts that are too long give away too much of their sites' content. This in turn, they say, could cause potential visitors to not click over to the page, particularly if the abstract, or snippet, gives them the information they're looking for.

This is an area where search engines have to strike a delicate balance between fulfilling their mission -- giving their users the most precise information possible related to their query -- and not violating the copyrights of Web site publishers.


Copyright © 2009 IDG Communications, Inc.