Information Week
The new search indexing system is faster and provides 50% fresher results, according to Google.
Google has switched to a new search indexing system that the company claims is faster than the previous technology and provides 50% fresher results.
Google announced last August that it was overhauling its search technology. Dubbed "Caffeine," the new architecture was introduced a month after Microsoft upped the ante in the search war by extending its new Bing search engine to all of Yahoo's Web properties. The Yahoo Web portal is the number two player in search.
Google announced last August that it was overhauling its search technology. Dubbed "Caffeine," the new architecture was introduced a month after Microsoft upped the ante in the search war by extending its new Bing search engine to all of Yahoo's Web properties. The Yahoo Web portal is the number two player in search.
Google said Wednesday it built Caffeine to bring users more up-to-date, relevant search results from the fast-growing Web.
To better understand how Caffeine works, a person must first know that Google doesn't search the entire Web to answer user queries, but rather its index of the Web. The quality of results depends on how well a search engine can keep its index up-to-date.
Under the old system, Google would crawl the entire Web to update large batches of Web pages in its index. Updates of individual pages in a batch could not be made available until the entire batch was updated, which meant there was a significant delay between when Google found or updated a page and made it available to the user.
With Caffeine, Google crawls the Web in smaller portions and updates its index on a continuous basis.
"As we find new pages, or new information on existing pages, we can add these straight to the index," Google engineer Carrie Grimes said in the company's blog.
Caffeine analyzes hundreds of thousands of Web pages each second in parallel and adds new information to the index at a rate of hundreds of thousands of gigabytes per day, according to Google. Caffeine takes up nearly 100 million GB of storage in one database.
"You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles," Grimes said.
Early testers of Caffeine gave the search platform rave reviews shortly after the new system was first introduced. Testers said it yielded more results with better accuracy than the existing technology.
Google remains the dominant search engine, accounting for 71.4% of Web searches in May, according to Experian Hitwise. Yahoo is a distant second with a 14.96%, followed by Bing, 9.43%.
However, Microsoft in May made significant strides in four major categories, a reflection of the company's focus on vertical markets. The number of searches on Bing related to automotive, health, shopping, and travel soared by 95%, 105%, 100%, and 71%, respectively, compared to the same month a year ago, Hitwise said.
In March, Microsoft announced that it would release a number of enhancements to Bing. Most notably, Microsoft improved Bing's Quick Tabs feature, which delivers results based on what the search engine believes is the intent of the user's query.
