x-robots, invisible directives

A little-known feature of search engines that could amaze webmasters because it does not appears in the content of the page while acting as meta tags.

It has become possible to turn nofollow external links in a completely invisible manner, since it can even be selective and work only with Googlebot!
What also makes nofollow detection tools, as the SeoQuake extension for Firefox, ineffective and misleading.

Nofollow

Remember that the nofollow attribute is intended for an external link ignored by crawlers. This attribute can be added to the tag of a link:

<a href="xxxx" rel="nofollow">Lien</a>

It can also be applied to the whole page in a meta tag:

<meta name="robots" content="nofollow">

We use this tag for pages that must not be indexed by search engines, such as registration forms, legal information and such pages.

x-robots

This is what makes the importance of the X-Robots directive!

In PHP it can be written:

header(’X-Robots-Tag: noindex, nofollow’, true);

And in the .htaccess file it can be included in a rule:

Header set X-Robots-Tag "nofollow"

In both cases the operation is completely undetectable to the user, or software reporting nofollows.

Implications

Exchange of links in footer or in the content allows sites to establish an artificial popularity. Google could penalize it.

Under a link exchange scheme, the value of PageRank transmitted tends to be neutralized when many links are reciprocal, but if one partner links the other site under nofollow, he takes an advantage.

A reason to erely on natural links, those that are really useful to visitors because they provide a supplement to the article.

Google supports X-Robots, including noindex, noarchive, nosnippets properties and the availability period.

See also the SEO Manual for more infos about directives to search engines.