User-agent: * Disallow: /administrator/ Disallow: /cache/ Disallow: /components/ Disallow: /editor/ Disallow: /help/ Disallow: /images/ Disallow: /includes/ Disallow: /language/ Disallow: /mambots/ Disallow: /media/ Disallow: /modules/ Disallow: /templates/ Disallow: /installation/ ################################### # YouPosted.com Smart Robots v2.01 ################################### # I have implemented a smart Robots.txt due to the incompatibility issues between different # robots and whether they support wildcards (*), multiple user-agents and end-anchors ($). # And it helps to keep the size of the robots down without duplication. # This means that I am now serving a spider-specific robots.txt to Google/Yahoo/MSNbot # and a generic one to every other bot/person. # # Detected Spider/Bot: None # Serving Robots.txt: Generic # A basic robots.txt without the use of wildcards and end-anchors. # It excludes less, but at least every spider/bot can understand them. # # Headers Sent: # Content-Type: text/plain # Expires: Tue, 15 Apr 2008 19:35:09 GMT (24 hour validity) # # My Sitemap - I don't provide it just for the fun of it Sitemap: http://www.youposted.com/sitemap.xml User-agent: * # All Spiders should understand these # Default SMF Folders Disallow: /attachments/ Disallow: /Packages/ Disallow: /Smileys/ Disallow: /Sources/ Disallow: /Themes/ # Default SMF Actions Disallow: /index.php?action=activate Disallow: /index.php?action=admin Disallow: /index.php?action=calendar Disallow: /index.php?action=emailuser Disallow: /index.php?action=findmember Disallow: /index.php?action=help Disallow: /index.php?action=helpadmin Disallow: /index.php?action=login Disallow: /index.php?action=logout Disallow: /index.php?action=mlist Disallow: /index.php?action=modifykarma Disallow: /index.php?action=pm Disallow: /index.php?action=post Disallow: /index.php?action=printpage Disallow: /index.php?action=profile Disallow: /index.php?action=recent Disallow: /index.php?action=register Disallow: /index.php?action=reminder Disallow: /index.php?action=search Disallow: /index.php?action=unread Disallow: /index.php?action=unreadreplies Disallow: /index.php?action=verificationcode Disallow: /index.php?action=who Disallow: /index.php?theme # SMF Mod Related Disallow: /archive.php Disallow: /index.php?action=blog Disallow: /index.php?action=viewblog Disallow: /index.php?action=chess Disallow: /index.php?action=comment Disallow: /index.php?action=downloads Disallow: /index.php?action=links Disallow: /index.php?action=reporttm Disallow: /index.php?action=recenttopics Disallow: /index.php?action=mm Disallow: /index.php?action=sitemap Disallow: /index.php?action=staff Disallow: /index.php?action=tags Disallow: /index.php?action=thankyou Disallow: /index.php?action=viewkarma Disallow: /index.php?action=viewers Disallow: /index.php?f= Disallow: /index.php?filter Disallow: /index.php?referredby Disallow: /Games/ Disallow: /Downloads/ Disallow: /index.php?action=arcade;favorites Disallow: /index.php?action=arcade;sa=highscore Disallow: /index.php?action=arcade;sa=play;random Disallow: /index.php?action=arcade;category Disallow: /index.php?action=arcade;sort Disallow: /index.php?action=arcade;stats Disallow: /index.php?action=stats;collapse Disallow: /index.php?action=stats;expand # Spammers - Passing links doesn't work, but stop the requests as well Disallow: /index.php?action=http Disallow: /index.php?board=http Disallow: /index.php?topic=http # Non-SMF related/Junk Disallow: /runsearch/ Disallow: /runsearch Disallow: /freelinks.php Disallow: /No%20advertising Disallow: /rc3/ Disallow: /rc3/index.php?action=links