User-agent: * # match all bots Disallow: / # keep them out Crawl-delay: 10 # maximum rate is one page every 10 seconds Request-rate: 1/10 # maximum rate is one page every 10 seconds Visit-time: 1800-1815 # only visit between 18:00 and 18:15 UTC (GMT)