# Allow Google, Yahoo and Bing to crawl all beside of /admin/ User-agent: Googlebot User-agent: Yahoo! Slurp User-agent: msnbot User-agent: AhrefsSiteAudit Disallow: /admin/ Disallow: /analitics/ Disallow: /class/ Allow: / # Disallow all other to crawl everywhere User-agent: * Disallow: /