User-Agent: * Disallow: /client-area/embedded-login Disallow: /clients/domainchecker.php Allow: /