# robots.txt for mklab.ph.rhul.ac.uk, T.Crane 31th May 2002 User-agent: * Disallow: /manual/ # Don't index the local default copy of the Apache manual #The IRL crawler is compliant with the /robots.txt file. You can use the #following commands to prevent it from accessing your website: User-agent: IRLbot Disallow: /