Short Answer: There is no straight forward yes/no answer to this question. If you are running a typical Blog using WordPress, where your category and tag pages only contain excerpt from belonging posts, then the answer is no, you shouldn't allow search engines to index your category and tag pages. If you have unique content to those category and tag pages, then the answer is yes, you should index them for better SEO.
Noindex Category & Tag pages if they contain only post excerpt & no unique content
Here is why you shouldn't index tag/category pages with no unique content & only excerpts from posts:
Search engines rank your pages using internal and external linking to and from your pages: higher the incoming link, higher the rank + higher the rank of the incoming links higher the rank of your link.
Each search engine has it's own ranking algorithm, but the basic idea is the same. You may not have any control over the external linking, so I'm giving an example only using internal linking. say you have three blog entries:
http://YourSite.com/bing-search-engine/, http://YourSite.com/yahoo-search-engine/ and http://YourSite.com/google-search-engine/
and these blog entries are under the category named Site Review: http://YourSite.com/site-review/
Now notice that, the Site Review category page will have linking from all three blog posts, but each of these posts will be linked only once from the category page itself. So, search algorithms will give the category page higher priority!
So in essence, if users search something and find your site in the result, in most cases they will find your category page instead of the post that contains the original information. This is a typical case. Because in a typical installation, Category pages of your blog contain only excerpts (i.e. a short description) from the original posts.
So, for a typical WordPress Category SEO, where you have no other unique content for that category, it's safe to index your blog entries only & make category pages noindex
The same applies for WordPress Tags SEO as well. Because in a typical installation, Tag pages of your blog also contain only excerpts from the blog posts. So you almost always gain nothing by indexing your tag pages that only contain post excerpts.
So make the typical tag pages noindex as well (if the tag pages contain no unique content), for better blog tags SEO.
Same applies for blogs other than WordPress as well. However, if you run a different type of site (something other than blogging), then this decision will solely depend on the type of your site and the content they have.
Index Category & Tag pages if they contain unique content
Do your category or tag pages have additional content other than the post excerpts or do they have added value to your site in any other meaningful way?
If the answer is yes, then go ahead and index them. If the answer is no, then don't index your category and tag pages.
There is a very good WordPress plugin named Yoast SEO that can manage all these for you. So install this plugin and configure everything you need from the admin panel of the plugin.
Remember that search engines will still crawl your category and tag pages, regardless of your indexing in the XML sitemap or search bot setting (either in meta tag or in robot.txt file). The only affect of not indexing them (where appropriate) is: they (category and tag pages) will not have higher ranking over the individual blog entries and pages.
Also, note that while giving the above suggestion, I've assumed that in your WordPress installation (or any other blog), the category and tag pages have a higher degree of incoming internal links. This can differ from theme to theme, but in most cases they do have higher incoming links than individual posts.
To go a step further: you better have unique content in all of your category and tag pages and then index them to get ranking for terms that are specific to those category and tag pages only.
Let's revisit the example: lets say your posts on Google, Bing and Yahoo contains information on how good or bad these search engines are and specific tips on how people can get most benefit from the specific search engines. We've already assumed that they are under site review category, now let's also assume that the posts have two common tags named Advanced Search Tips & Tricks (http://YourSite.com/topics/advanced-search-tips-tricks) & Search Engine Comparison (http://YourSite.com/topics/search-engine-comparison). If you were following my post carefully, you already know what I'm doing here. Yes, you've guessed it right!
At the beginning, you'll only have post excerpts on those tags and category pages. So you'll give them noindex, so that if someone search for "Yahoo Search pros and cons" in google, your post http://YourSite.com/yahoo-search-engine/ will show up, not the category page http://YourSite.com/site-review/ or the tag page http://YourSite.com/topics/search-engine-comparison.
However, as you can see yourself, those category and tag pages are a great opportunity to rank on a different set of keywords! So write at least a 300 words article on the site review category page, for example, what is a site review, how did you reviewed the sites, whether or not you'll review a site on request from your visitors etc. etc. So your category page will now have unique text content on site review, and the belonging excerpts from your Google, Yahoo and Bing posts will only come after that content. Same applies for the tags. Write at least 300 words articles (Yoast SEO plugin will allow you to write articles on tag pages) on Advanced Search Tips & Tricks and Search Engine Comparison tag pages. At this point, go back and make these category and tag pages to index.
Now search algorithms will correctly rank your posts on specific related terms and at the same time, your category and tags pages will be ranked for keywords like "site review", "advanced search tips", "search engine comparison" etc. etc. but none of them will compete with each other, because they don't have duplicate content now. Isn't that Great!
To summarize: Index category and tag pages only when you have unique content in those pages, until then, keep them with noindex. If some of them have unique content and some of them don't, then give them index and noindex accordingly.
Happy blogging 🙂
Funny, but doesn't this post link to your tags (look on the right hand side menu)?
Actually it's not 🙂
Please read the post carefully, since you've missed the whole point.
The point is about indexing in Google, not linking or following. In fact linking a post to different tag pages may be good!
However, if you go to any of these tag pages and view the source, then you'll see that in all the tag pages, there is a small piece of HTML CODE:
<meta name="robots" content="noindex,follow" />
This CODE asks Google not to enlist the tag pages in search results, but follow the links within.
In fact, follow doesn't have to be mentioned, since it is implied. So the CODE can be:
<meta name="robots" content="noindex" />
I hope you've got the point now. Cheers 🙂
Hi
I'm building a voucher website and I have some questions regarding tags & seo. Can you please help?
Let's say I have 5 voucher codes for 5 different websites that sell holidays, hotels, flights & car hire and the vouchers can buy any of the products. On each voucher, I want to make sure each product has tags relevant to it.
Now, each product/keyword will have 5 different keywords extensions as tags. For example hotels tags will be: hotel voucher, hotel voucher code, hotel discount voucher, hotel discount voucher code, hotel discounts. So, altogether there will be 20 tags on each voucher.
Question 1: Since each of the 20 tags has its own URL, but they all have the same content(same voucher excerpts/preview). Would that be considered as duplicate content?
Question 2: Since each voucher will have 20 tags(more in some categories), would that be considered as tag spamming.
I'm hoping each tag will be indexed and it will make it easy for people to find vouchers on my site when they search those keywords.
Hi, This is your answer:
Ans-1: Try to put some difference in the tag pages. For example, I'm assuming you put partial content (excerpt) in those tag pages. In that case if you put the excerpt like Google does, i.e. put the text content that is around this specific tag, then you'll be able to avoid duplicate content.
If that's not possible, then try to put some description related to the tags on the tag pages. Otherwise it may be considered as duplicates.
Asn-2: As long as your voucher description/title etc. (that is visible on the page) contains the tag, it'll not be considered as tag spamming.
Try to minimize the tags that are not present within the text content of the site.
I hope it answers your question.
Cheers 🙂
Great i will adjust tag and category crawl settings using robots.txt
Great! Thanks for the share! I just bookmarket your blog and wanted to say that I have really enjoyed reading your blog posts.
Very nice tips, Thanks for the help:)
The problem is that I've had it set to index tag categories for a long time with All in One Seo and I'm afraid that if I un-index them now I will lose search rankings and thus get less traffic. What do you think?
I noticed that it happens exactly as you describe, my tags show up in search ahead of the actual articles. And since it is just a tag, it's not as likely to get clicked as often because the title is just a word or two, and then when people do clickthrough they have to scroll all through the list of articles to find what keywords they were looking for instead of just going to the specific article.
Nathan, I had the same confusion when I first did this. The thing is, Just because you decide to use no-index for your tag pages doesn't mean Google will stop crawling those pages and stop giving you credit for those pages.
You'll still get SEO values from those pages, but Google will simply stop showing those tag/category pages in the search result. Also, your original blog page will in turn get the additional values from the related tag/category pages since those pages still link to your original post.
This is what happened in my case: This post: http://www.fayazmiraz.com/wordpress-blog-url-change/ was in the first page in some search results (it still is 🙂 ), however the related tag page was in 1st position and the original post was in 4th. After I decided to put the no-index meta tag, magically my tag page was removed from the same search result and the original page jumped to position 1 🙂
If you're still not confident, then try it yourself in one of your pages manually, and put the following meta tag in the head section of your tag page:
<meta name="robots" content="noindex,follow" />
If your result is a success, then use "All in One Seo" to do the same for all the pages.
Happy Blogging 🙂
Thanks for your advice. I went ahead and turned off indexing for tags last night. I'll keep my fingers crossed...
Interesting. I was trying to determine if I should link the category or tags. Good to have a different view on this. Considering how much time and money Google spends developing their search technology, you wouldn't think it would matter that much anyway!
yup..changes made! thanks dude :-0
Oh now its clear to me, my site has more than 8k index pages and it includes tags and categories, now I will set it to noindex, thank dude.
Thank you. I had been looking for a good explanation for this for a while now.
Thanks for the tips, couldnt figure out why my tags out rank posts
A very very thanks, i was really very much confused on this topic weather to index tag and category or not, every time i do some RND and my site google search result was decreased from first page to 4th page.
That's exactly what I was looking for... I'm on my way to change the meta tags on my Blog !
Thanks
@Gilles_Vitu
This is great thanks, I really appreciate it.
A very good post. Do you send newsletters as well ?
Nop, no newsletters. Not yet anyway. But you can subscribe to RSS Feed if you want to 🙂
http://feeds.feedburner.com/fayazmiraz
Uhh, this is exactly happened in on of my client's site. I've Googled for it and thanks to you i've found the reason. Thanks for these free seo tips 🙂
Ya it makes sense, that's why their category pages were coming up more often in Google search than the original posts. So, many of the intended pages got lesser traffic and eventually it became a reason of lesser conversion rate.
I made the main menu link to categories and have google index them.
Since my site is fairly new I can't say which is better compared to my other old site which I noindex'ed the category and tags.