Images In Blended Search

This post is going to take a look at how far the top 4 search engines have gone with integrating images into their blended search results. I’ve put it together as part of my background research for my presentation at SMX West on the panel on images and blended search.

What I did was take 10 different search terms and tried them out. One of the search terms ended with the word “pics”, four ended with “pictures” and five ended with the word “images”. So let’s start by looking at the raw data:

Search Engine Search Phrase Results Source Comments
Yahoo! white mountain pics 3 images up top Flickr  
Google white mountain pics 0 images N/A  
Microsoft white mountain pics 0 images N/A  
Ask white mountain pics 3 images up top Ask image search videos and news on right, image results same as white mountains pictures, do not match the image search results for white mountain pics
Yahoo! white mountains pictures 3 images up top Flickr pictures are from California, the Alps, and upstate NY
Google white mountains pictures 0 images N/A  
Microsoft white mountains pictures 5 images up top Live Image search  
Ask white mountains pictures 3 images up top Ask image search encyclopedia, videos and news on right
Yahoo! acne pictures 3 images up top Flickr (2), Medlist.com  
Google acne pictures 0 images N/A  
Microsoft acne pictures 5 images up top Live Image search  
Ask acne pictures 3 images up top Ask image search encyclopedia, videos and news on right
Yahoo! acne vulgaris pictures 3 images up top acne.republica.pl, scf-online.com, cosmetic.medicine.jp  
Google acne vulgaris pictures 3 images up top history.amedd.army.mil, library.med.utah.edu, aafp.org  
Microsoft acne vulgaris pictures 5 images up top Live Image Search  
Ask acne vulgaris pictures 3 images up top Ask image search encyclopedia, videos and news on right
Yahoo! psoriasis pictures 3 images up top dogflu.ca, dermnetnz.org, dermatology.co.uk  
Google psoriasis pictures 0 images N/A  
Microsoft psoriasis pictures 4 images up top Live Image Search  
Ask psoriasis pictures 3 images up top Ask image search encyclopedia, videos and news on right
Yahoo! Moon images 3 images up top smm.org (2), us.movies1.yimg.com  
Google moon images 3 images up top hoax-slayer.com. carpecrustulum.wordpress.com, nssdc.gsfc.nasa.gov  
Microsoft moon images 5 images up top Live Image Search 3 are of cartoon (anime) character Sailor Moon
Ask moon images 3 images up top Ask image search videos and news on right, First item is a picture of a moon snail
Yahoo! stanford images 3 images up top d.yimg.com (2), us.movies1.yimg.com  
Google stanford images 2 images up top star.stanford.edu, serc.carleton.edu  
Microsoft stanford images 4 images up top Live Image Search  
Ask stanford images 3 images up Ask image search videos on right
Yahoo! mit images 3 images up top Flickr (2), cs.virginia.edu  
Google mit images 3 images up top images.google.com  
Microsoft mit images 4 images up top Live Image Search  
Ask mit images 3 images up top Ask image search videos on right
Yahoo! forest images 3 images up top ucmp.berkeley.edu 2 are basically half size for some reason
Google forest images 3 images up top mass.gov (2), hickerphoto.com  
Microsoft forest images 4 images up top Live Image Search  
Ask forest images 3 images up top Ask Image Search videos and news on right
Yahoo! trees images 3 images up top Flickr  
Google trees images no image results N/A  
Microsoft trees images 5 images up top Live Image Search  
Ask trees images 3 images up top Ask Image Search videos and news on right

Let’s try to summarize this. Here are my observations about the above data:

  1. Yahoo and Ask picked up on the phrase “white mountain pics” and rendered images for that search. Google and Microsoft did not.
  2. Microsoft did offer inline image results up above the web search results in all 9 cases other than the “pics” case.
  3. Google provided image results only 50% of the time, in spite of the leading words “images” and “pictures”.
  4. Yahoo and Google provide a substantial portion of their results directly from third party web sites, as opposed to their image search properties.
  5. Microsoft and Ask provided 100% of their results from their respective image search engines.
  6. There were only two anomalies across all 40 searches performed:
    • Ask provided the same results for “white mountain pics” as it did for “white mountains pictures”. Interestingly enough, even though they provide all their results from their image search engine, “white mountain pics” and “white mountains pictures” do NOT provide the same results in their image search.
    • Microsoft shows two pictures of an Anime character, Sailor Moon, in response to “moon images”. I’d think that they would only call up Sailor Moon images if someone actually typed in “sailor moon images”.

Another interesting aspect of all this is that if you dig in a bit deeper and repeat these searches on the image search engines, you find that the results integrated into web search are NOT, generally speaking, the first 3 results that come up for the exact same search within image search.

For example, if you look at the “acne vulgaris pictures” results from Microsoft, the five images presented in the web search results are the 1st, 5th, 7th, 9th, and 10th results if you search on the exact same phrase in image search.

Overall the result quality was high. When someone provides such a leading indicator of what they want (i.e. “images” or “pictures” is part of the search query) most vendors picked up on that cue (C’mon Google!) and presented reasonable results.

Comments

  1. Holly says

    This is interesting. Is it safe to assume they are getting the images based off of the name associated with the image?

  2. says

    Nice findings!

    I think you’ll find that the Google results are from their image search, not from a third party. It’s not obvious, because most do not rank well for the same keyphrase in the image search.

  3. says

    Hi Holly – There are many factors in how images are picked. Alt tags and image file names are big ones. However, surrounding text on the web page, the web page title and content are also factors that search engines look at. But certainly the alt tag and the file names are big factors.

    Patrick – The above data is meant primarily to provide some data on the differences between the integration of image search into the web search engines across each engine.

    It also shows that most search engines will value the use of a leading word like “pictures” in searches performed by users and are very likely to include image search results in web search when users are that specific in their search query.

    For the time being, the path to success is still based on the same principles of image optimization (alt tags, file names, surrounding web page elements), but with an increase emphasis on including the word pictures (or images) in your alt tag for each image.

Leave a Reply

Your email address will not be published. Required fields are marked *

*