YaCy '#[clientname]#': Local robots.txt

#%env/templates/metas.template%#
#%env/templates/header.template%# #%env/templates/submenuPortalConfiguration.template%#

## Exclude Web-Spiders

Here you can set up a robots.txt for all webcrawlers that try to access the webinterface of your peer. robots.txt is a voluntary agreement most search-engines (including YaCy) follow. It disallows crawlers to access webpages or even entire domains.

#(error)# ::

Unable to access the local file: #[msg]#

::

Deletion of htroot/robots.txt failed

#(/error)#
Deny access to

Entire Peer
Status page
Network pages
Surftips
News pages
Blog
Wiki
Public bookmarks
Home Page
File Share
Impressum

#%env/templates/footer.template%#