> Main > Products and Services > Purchase > Company
English / Russian

Documentation

CNSearch 1.5.1

© "CN-Software" Ltd. 2002-2008
support@cn-software.com
www.cn-software.com
Previous | Table of Contents  | Next

Examples

Example 1:

# robots.txt for http://www.site.com

User-Agent: *

# this is an infinite virtual URL space

Disallow: /cyberworld/map/

Disallow: /tmp/ # these will soon disappear

The content of '/cyberworld/map/' and '/tmp/' is protected in this example.

Example 2:

# robots.txt for http://www.site.com

User-Agent: *

# this is an infinite virtual URL space

Disallow: /cyberworld/map/

# Cybermapper knows where to go

User-Agent: cybermapper

Disallow:

In this example, the search robot 'cybermapper' is granted full access, while the rest do not have access to content of '/cyberworld/map/'.

Example 3:

# robots.txt for http://www.site.com

User-Agent: *

Disallow: /

Access to the server is denied to any search robot in this example.


Previous | Table of Contents  | Next