Implementing Custom Meta Descriptions in WordPress

Recently, we did a post about Google’s snippets stating that out site will soon be the most read site in the SEO World. Not many people can claim that Google says that about their site.

As I pointed out in the discussion, this results from the way that Google builds it’s snippets. Google first looks to find text on your page that contains the actual user search query, or something quite close to it. If it does not find such a string of text, it will consider using an ODP description for your page (if there is one) or your meta description tag.

If Google cannot find either one of these, then it creates the best snippet it can using text it finds on your page. So then I received a link from a post by Steve Mertz where he clearly explains that the way to avoid Google using bad descriptions for your web page is to write a compelling meta description.

And Steve is right. As long as Google does not decide to build its description of your page based on the search query itself (usually because it cannot find the exact phrase), your meta description can be the message that people see about your site.

So this set me to thinking about implementing meta description tags within WordPress. Turns out that it is not easy. I went to the WordPress plugins directory and started checking out all the plugins in the “metadata” section.

I was looking for one that would allow me to custom write my own meta description. Most of the plugins don’t support this. They will auto construct the meta description from the beginning text of your post. Since I had gotten up the energy to solve the problem at all, I decided that I wanted to solve it well. This meant that I was committing to writing a custom meta description for each post.

After about an hour or so of research, I settled on Another WordPress Meta plugin. This was the only one I found that would allow me to write a custom meta description for each post. So I installed it.

Installation was easy. Operation is easy too. Just below the text for each post is a box that provides a text edit box for the meta description, and another one for meta keywords. A minute or two of extra work for each post. Piece of cake.

One downside though. If you don’t write a custom meta description the plugin will use the same meta description as you have for the home page for your blog. Great to have a fallback that is probably a decent description of the blog as a whole, but it potentially adds to duplicate content concerns.

In addition, it automatically implements this meta description on all your historical posts. So I spent a couple of hours today going through all my old posts writing a custom description for them. But what the hell, when you get around to addressing problems like these, you might as well address it completely.  Probably means that I will soon lose that great description from Google soon though …

UPDATE: I received a comment from Uberdose (see below), the publisher of “Another WordPress Meta Plugin” that they have removed the feature of automatically using the home page post on all of the historical posts, thus eliminating the duplicate content problem I eluded to above.  This is a great improvement on this tool, making it even better.

NoODP, NoYahoo

Barry Schwartz reports that Yahoo is on the verge of adding support for the NoODP metatag. This emerged from an interview with Yahoo’s Tim Mayer.

This is a good step forward, as Yahoo does use the descriptions for sites found in the Open Directory Project on some occasions, and sometimes these descriptions are not that good.

Sadly, Yahoo is not implementing a NoYahooDir tag. This is a mistake, as pointed out by Danny Sullivan in this post.

What Danny demonstrates is a few examples where directory descriptions are not very good. One way this happens is when a description is written that uses data that changes over time. Danny provides the example of Tony Knowles, who is shown by one search engine to be running for governor in Alaska, and by another to be running for Senator in Alaska.

What happened here is that description data was pulled from different sources, and in the case of Yahoo, the description is that used in the Yahoo directory.

Pulling data from static sources, such as a directory, as a description for a dynamic object (a web site) is bad. There just isn’t a simple algorithmic solution for precisely recognizing when a directory description is inaccurate. The description will get out of date, and a directory source should only be used as a last resort.

So hopefully, the search engines, including Yahoo, will get on board with this soon.

The NoIndex Metatag Debate

Matt Cutts has just put out a posting on how search engines handle the “noindex” metatag. He makes some interesting observations about the results as follows:

  • Google doesn’t show the page in any way
  • Ask doesn’t show the page in any way
  • MSN shows a url reference and Cached link, but no snippet. Clicking the cached link doesn’t return anything.
  • Yahoo! shows a url reference and Cached link, but no snippet. Clicking on the cached link returns the cached page.

Matt goes on to note that it would be great if all the search engines treated the noindex metatag, well, like a noindex request (translation, “DON’T index it” – my words, not Matt’s). I think that this is an excellent idea.

Danny Sullivan, still of Search Engine Watch, adds his thoughts to the noindex discussion. Danny observes that Google treats the robots.txt file in a way that many webmasters may not expect. In particular, Google may still index pages that are marked as “don’t crawl” in the robots.txt file, may still be indexed by Google.

The one case I have seen where this happens is when pages are marked in robots.txt as “disallow”, yet third party pages link to those pages. In the past, Google has reserved the right to still index these pages.

In fact, Google may still index a page even if it is marked with the noindex metatag. This can happen if it is also listed in the robots.txt file as “disallow”. The reason is that Google treats the robots.txt as being a higher priority than the noindex metatag, and ignores the metatag if the page is also called out in the robots.txt file.

Now it turns out that this is pretty simple to address. All you need to do is not use the robots.txt file, and rely solely on the noindex metatag. In this event, Google will reliably not index the page, in all cases (that I know of).

Danny repeats a call I have seen him make in the past for imporved standardization of search engine behavior. More standardization by the search engines sounds like another excellent idea. We all recognize that there are things that the search engines need to keep close to the vest, and treat as proprietary. But making life easier on SEOs and search engines by following agreed upon standards will benefit us all.

You can read more about metatags and SEO here.

Robots.txt and Robots Metatags

Telling a search engine to not index a page sounds relatively easy. According to the specs, all you need to do is implement the NOINDEX parameter in your Robots Metatag. Sounds like you’re all set. But you might not be.

Why? It turns out that if you have also used your Robots.txt file to tell Google to not crawl the pages you have “NOINDEX”ed that Google will ignore the metatags. This is because in Google’s algorithms Robots.txt takes precedence over the Robots metatag. Translation: the Robots Metatag is ignored if you exclude the crawling of the page via Robots.txt.

So even if the page is not crawled, your page can still be indexed if other pages link to it. So if you want to prevent a page from being indexed, use only the Robots Metatag, and you should be off to the races.