# robots.txt - control Web robots # see e.g. http://www.google.com/support/webmasters/bin/answer.py?answer=40360 # this is /var/www/html/robots.txt # the TRIUMF iGoogle appliance User-agent: gsa-crawler-itriumf Disallow: /docushare/dsweb/Services # the public Google crawler User-agent: Googlebot Disallow: / # everything User-agent: * Disallow: /