## Enable robots.txt rules for all crawlers User-agent: * ## Magento sitemap: uncomment and replace the URL to your Magento sitemap file # Sitemap: http://www.example.com/sitemap/sitemap.xml ## GENERAL MAGENTO SETTINGS ## Do not crawl Magento admin page Disallow: /admin/ ## Do not crawl common Magento technical folders # Disallow: /app/ # Disallow: /downloader/ # Disallow: /errors/ # Disallow: /includes/ # Disallow: /lib/ # Disallow: /pkginfo/ # Disallow: /shell/ # Disallow: /var/ Disallow: /hippy/