Add a robots.txt URL to the site root

Adds a robots.txt file to the site root. Defaults defined by
'robotstxt' section of config. New events StartRobotsTxt and
EndRobotsTxt to let plugins add information. Probably not
useful if path is not /, but won't hurt anything, either.

1 files changed, 4 insertions, 0 deletions

diff --git a/lib/default.php b/lib/default.php
index 1337a9633..2bedc4bf0 100644
--- a/lib/default.php
+++ b/lib/default.php
@@ -270,4 +270,8 @@ $default =
         'singleuser' =>
         array('enabled' => false,
               'nickname' => null),
+        'robotstxt' =>
+        array('crawldelay' => 0,
+              'disallow' => array('main', 'settings', 'admin', 'search', 'message')
+              ),
         );