# more information about this file can be found at # http://www.robotstxt.org/wc/robots.html # a list of known bots # http://www.robotstxt.org/wc/active/html/index.html # see http://www.sxw.org.uk/computing/robots/check.html # for syntax checking User-agent: * Crawl-Delay: 10 Disallow: /includes/ Disallow: /submit/