> Main > Products and Services > Purchase > Company
English / Russian

Documentation

CNSearch 1.5.1

© "CN-Software" Ltd. 2002-2008
support@cn-software.com
www.cn-software.com
Previous | Table of Contents  | Next

Using "Robots" Meta-tags

Besides the Exclusions standard, there is also a possibility to control search robots behavior by means of 'META' HTML-tag.

Unlike 'robots.txt' files which describes site indexing as a whole, 'META'  tag controls indexing of a certain web-page. Besides, it is possible to cancel indexing not only of the document itself, but of the document links as well.

Indexing parameters should be defined in the 'content' field of a source code of each page. The following parameters can be used:

Default value: <meta name="Robots" content="ALL">.

Note: Values should not be separated by a comma.

Incorrect variant:

<META name="ROBOTS" content="noindex, nofollow">

Correct variant:

<META name="ROBOTS" content="none">

In this example the indexer allows analyzing a document without indexing its links:

<META name="ROBOTS" content="nofollow">

The name of the tag, as well as the names and values of fields are not register sensitive. As a matter of fact, the indexer checks only for 3 values: NOINDEX, NOFOLLOW, and NONE, because FOLLOW and INDEX are default values.


Previous | Table of Contents  | Next