# robots.txt User-agent: MSIECrawler Disallow: / User-agent: SiteSnagger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: WebReaper Disallow: / User-agent: * Disallow: