I wrote this absurdly long email the other day about the proper use of tags. Specifically, it discussed tags being indexed by the site’s search engine and Google. I realized that for the most part, none of the recipients would fully read it, so I decided to strip out the site-specific and share it with the rest of the world. It was initially about twice this long. At least now I don’t feel like I wasted my time.
Searching tags on the site
Tags are a filter mechanism. They provide an easy way to list all articles that are tagged with the same word. In this way, they serve as an ad-hoc extension to categories that allow websites to add additional category-level organization without altering the existing architecture. This is handy because sites generally do better with a condensed taxonomy. You don’t want to have it any more complex than necessary, so tags do a great job of adding the useful function of category-level navigation without mucking everything up. Because this is their intended use, in order to ensure that tag pages don’t alter Google’s understanding of the site’s taxonomy, tag pages are othe sister siteen intentionally excluded from Google’s index by the use of a noindex meta tag or the robots.txt file. You can’t search for tags in the search box for the same reason you generally don’t include the category in the search index.
Search algorithms are complex, and while I can’t speak intelligently on the existing algorithm’s logic, I’m sure it’s difficult to modify it do properly weight and rank for tags. My assumption, however, would be that in most cases the articles is tagged with a word that exists in the body, so that it will still show up in search, weighted appropriately.
Searching tags on the sister site
There are multiple reasons that you cannot search for the site’s tags on the sister site. This first is that the sister site uses a very intricate, custom built search mechanism. It’s fairly fragile and extremely complex. Modifying this to also index content from the site would be very difficult and risky. Search has been a contentious issue for years on the site, and the community is happy with the current setup. I would not want to risk disturbing the current search mechanism. The second reason has to do with the specific separation between the sister site and the site. Since the forum is so large, search is one of the primary means of navigation. If we insert site content into the sister site’s search results, we may likely receive pushback from the community. Finally, if we included site results in the sister site’s search results, we’d have to reciprocate. This would undoubtedly end up favoring the sister site in results over the site, for example, if I do a search for “[keyword]”, there might be a dozen site posts in the results versus 1,000 results from the sister site threads. Also, reciprocation would require rebuilding the search mechanism of the site. The current implementation can’t handle the mass of the sister site.
Searching for tags on Google
Getting our pages to rank in Google is something that we go rather well, however, when it comes to getting specific terms to rank well in Google, it really depends on the term. For example, it’s easier to rank for “Idaho Scotch-Tape distributors” than it is for “Cheap Las Vegas Hotels”. One is just a more competitive space. Depending on what keywords we wish to target, we may or may not be able to affectively rank for them. Regardless, there are few people out there with the knowledge and understanding of search engine optimization out there then our team here.
We need to make a distinction between an individual article ranking for a specific term, and our tag page ranking for a specific term. Generally, tag pages don’t rank. This is because people don’t link to tag pages, they link to articles—and links drive rankings. This isn’t a bad thing, as we othe sister siteen optimize our articles for conversions, but rarely optimize our tag pages. The thought being “why would we optimize a ‘portal’ page?”.
My final though on Google ranking for keywords relates to the nature of tags. They are typically what we call “head terms”, specific one or two word phrases. Targeting head terms is always more difficult then targeting “tail terms”, which are phrases with three or more words in them. We generally see better results when targeting tail terms, as you can target multiple variants of a tail term in a single article, tail terms are less competitive, and visitors who enter for tail terms are often more engaged.
