User-agent: ia_archiver Disallow: / User-Agent: * Disallow: /recommend # End robots.txt file