Somewhere during the days reading I came across this report on the Emerging Opportunity in Vertical Search. The report pulls stats and data from a report called Vertical Search Delivers What Big Search Engines Miss by Outsell, Inc.
What is fascination is that the Outsell’s data suggests that the B2B vertical search market will reach $1.0 billion in revenue by 2009. So it’s no wonder so many companies have launched vertical search platforms. Some of the most significant platforms, listed alphabetically, are:
- Eurekster Swickis
- Google Custom Search Engines
- Microsoft Live Search Macros
- Yahoo! Search Builder
All of these are vertical search platforms, that permit you to build your own vertical search engine. All of these vendors handle the crawling of the web, and the building of the index for you. You simply need to enter in lists of sites for your search engine, specify rules for its operation, and other related data through a web GUI of some sort.
Included in the Slack Barshinger report is some data from a Jupiter Research Survey done on search in 2004. The data shows that 41.2% of users in indicate that “Results are often not directly relevant to my query”. Frankly, I suspect that given a different question, such as “Some of the results in the top 10 results are not directly relevant to my query”, that the percentage would be far higher.
And this lines up with one of the conclusions, that there is an overall Internet search failure rate of 31.9%. What’s at the core of this problem?
Context. The search engines do not understand your context. In the Slack Barshinger report they provide the example of a dentist searching on ceramics. The result page shows pottery sites as the primary results. And even if the dentist refines their search, to include the word dentist, the results are not good.
This is because there are two context problems:
- The search engine does not know the searcher’s context
- The search engine cannot easily determine the intended context of web pages (i.e. is this page targeted at dentists of pottery buffs?)
Net-net, this is a BIG issue for search engines. They know that search has flaws, and they want to fix them. These problems are driving lots of innovation. There are hundreds of vertical search engine start ups, focused on a variety of narrow niche markets.
These companies perform their own crawls, build their own indexes, and even make use of non-crawlable data, such as purchased databases of information to provide more focused results. This space is going to be really big, and represents a big new frontier.