# /robots.txt file for GlennDoten.net. # Point to the Sitemap file. The Sitemap file tells search engines what # site content should be indexed. This file (robots.txt) tells search # engines the site content that should not be indexed. Sitemap: http://GlennDoten.net/sitemap.xml # Allow all robots. User-agent: * Disallow: # Diallow Internet archiver (web.archive.org, aka way back machine). User-agent: ia-archiver Disallow: / # Directories to disallow. User-agent: * Disallow: /aspnet_client/ User-agent: * Disallow: /admin/ User-agent: * Disallow: /ads/ User-agent: * Disallow: /bin/ User-agent: * Disallow: /Classes/ User-agent: * Disallow: /_database/ User-agent: * Disallow: /error/ User-agent: * Disallow: /images/ User-agent: * Disallow: /lib/ User-agent: * Disallow: /obj/ User-agent: * Disallow: /portal/ User-agent: * Disallow: /scripts/ User-agent: * Disallow: /secure/ User-agent: * Disallow: /upload/ User-agent: * Disallow: /stats/