block bot rogerbot for all URLs
authorWolfram Schneider <wosch@indexdata.dk>
Fri, 27 Jan 2012 11:58:48 +0000 (11:58 +0000)
committerWolfram Schneider <wosch@indexdata.dk>
Fri, 27 Jan 2012 11:58:48 +0000 (11:58 +0000)
web/htdocs/robots.txt

index fed2b3a..a0fdc6c 100644 (file)
@@ -10,3 +10,6 @@ Disallow: /raw.html
 Disallow: /stats.html
 Disallow: /upload.html
 
+User-agent: rogerbot
+Disallow: /
+