# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-agent: * # Disallow: / User-agent: * Allow: /search/location/sample Disallow: /zipcode/ Disallow: /polygons/ Disallow: /match_append_requests/ Disallow: /search/ Disallow: /xmlrpc.php Disallow: /iframe_states/ Disallow: /iframe_cities/ Disallow: /iframe_zip_codes/ Disallow: /iframe_neighborhoods/ Disallow: /iframe_autocomplete/ Disallow: /iframe_location/