# Search Engine Robots Control File # # Further information at: # http://www.searchengineworld.com/robots/robots_tutorial.htm # # EXAMPLES: # # 1) The following allows all robots to visit # all files because the wildcard "*" specifies all robots. # # User-agent: * # Disallow: # # 2) This one keeps all robots out. # # User-agent: * # Disallow: / # # 3) The next one bars all robots from # the cgi-bin and images directories, and # the cheese.htm file: # # User-agent: * # Disallow: /cgi-bin/ # Disallow: /images/ # Disallow: cheese.htm User-agent: * Disallow: