#robots.txt file specified initially on 10/10/05 #pretty much allowing any bot to spider the root, and not much else...mseguin #last updated on 3/4/06 to explicitly and completely ban Propsmart completely...mseguin #last updated on 3/14/06 to block Trulia completely...mseguin #last updated mseguin /2/16/08 to allow googlebot to crawl hot folder User-agent: * Disallow: /admin/ Disallow: /board/ Disallow: /cgi-bin/ Disallow: /committees/ Disallow: /contact_us/ Disallow: /css/ Disallow: /dev/ Disallow: /errors/ Disallow: /government/ Disallow: /help/ Disallow: /images/ Disallow: /js/ Disallow: /libraries/ Disallow: /members/ Disallow: /mgmt/ Disallow: /mysql/ Disallow: /news_and_info/ Disallow: /public/ Disallow: /recip/ Disallow: /rules/ Disallow: /search_for_REALTOR/ Disallow: /staff/ Disallow: /survey/ Disallow: /Templates/ Disallow: /webservices/ User-agent: Googlebot Allow: /hot/ User-agent: PropsmartCrawler Disallow: / User-agent: Alexa Disallow: / User-agent: TruliaBot Disallow: /