Google’s custom search engine (CSE) announcement has significant implications for the search engine industry and Google. This blog post will discuss some of these, but I am sure we will be learing about it for some time to come. Here are the major implications:
Combining Human Editing with Algorithmic Search
1. Google has recognized the inherent limitations in link based ranking systems. CSEs provide a method for incorporating expert human editor input into search results. There is a spark of brilliance in the way Google has done it.
Other human edited concepts, such as DMOZ, the Yahoo directory, and the various tagging sites that have emerged have all run into serious limitations.
DMOZ has run into problems because of its all volunteer model. Staffing is inconsistent, so many categories are uncovered, and it’s known that many categories are covered by people with competitive interests in the content of their part of the directory.
The Yahoo directory has the benefit of being staffed by paid editors. This ensures their motivations are managed by their employer. But in real terms, no human edited directory can truly categorize the entire web
Tagging sites, such as del.icio.us provide a voting mechanism which allows users to vote on sites. However, these tagging sites are no substitute for search. They are a very effective mechanism for uncovering the hottest new sites.
CSE’s provide a mechanism for subject matter experts (SMEs) to provide direction and guidance to their own personal version of Google search. Equally important, there is also a business model which provides economic motivation for the site owners / SMEs to do so.
Google is providing site owners with a revenue share of the ad revenue generated on their CSEs. In addition, the work done by the SMEs represents an asset which is owned by the site owner. This is a powerful combination.
The Distributed Search Model
Google’s CSE program are a major step in the direction of a distributed search model. As we said above, the notion of getting the human editorial input from economically motivated SMEs is a master stroke.
You can see that this may drive a whole new philosophy of search. End users may begin to migrate towards performing their search on CSEs implemented by SMEs that they trust. The long term winners will be those with the foresight to build CSEs with the purest editorial intent.
These types of CSEs will provide the highest possible quality results. Editors will remove spam sites, and tweak results to provide a high quality search result in their area of expertise.
As the model matures, CSE quality will improve significantly. More and more users will gravitate to the better versions of the search engines over time.
Improved Core Algorithms
So who will have access to all this wonderful data? Google. It is our understanding that they will not make use of the data in the short term, but in the longer term, one has to believe that they will begin to incorporate the wisdom of the market place into their core algorithms.
This could well be how they close the loop. Motivated, distributed human editors, driving the core algorithms. Sounds pretty sexy doesn’t it?