# www.robotstxt.org/ # http://code.google.com/web/controlcrawlindex/ User-agent: * Crawl-delay: 3 Disallow: /search Disallow: /assisted-pathway Disallow: /policies-and-programs Disallow: /?sq_content_src= Disallow: /_recache Disallow: /_edit Disallow: /_admin Disallow: /_login Disallow: /_performance Disallow: /designs Disallow: /_designs Disallow: /_web_services User-Agent: Funnelback Crawl-delay: 0