User-agent: * Disallow: /cgi-bin/ User-agent: Slurp Disallow: / User-agent: ROBOTS Disallow: / User-agent: All Spiders Disallow: / User-agent: Nutch Disallow: / User-agent: SBIder Disallow: / User-agent: crawl Disallow: / User-agent: bot Disallow: / User-agent: bot/ Disallow: / User-agent: Powermarks Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: findlinks Disallow: / User-Agent: BaiDuSpider Disallow: / User-Agent: psbot Disallow: / User-Agent: Walhello appie Disallow: / User-agent: ia_archiver Disallow: / User-agent: ArchitextSpider Disallow: / User-agent: Slurp.so/1.0 Disallow: / User-agent: Slurp/2.0j Disallow: / User-agent: Slurp/2.0-KiteHourly Disallow: / User-agent: Slurp/2.0-OwlWeekly Disallow: / User-agent: Slurp/3.0-AU Disallow: / User-agent: UltraSeek Disallow: / User-agent: MP3Bot Disallow: / User-agent: Lycos_Spider_(T-Rex) Disallow: / User-agent: NationalDirectory-SuperSpider Disallow: / User-agent: Atomz Disallow: / User-agent: httrack Disallow: / User-agent: NPBot Disallow: / User-agent: SurveyBot Disallow: / User-agent: Sqworm Disallow: / User-agent: FAST-WebCrawler Disallow: / User-agent: psbot Disallow: / User-agent: teoma_agent1 Disallow: / User-agent: Teradex_Mapper Disallow: / User-agent: WSCbot Disallow: / User-agent: url check Disallow: / User-agent: deepindex Disallow: / User-agent: 68.142.249.47 Disallow: / User-agent: lj2384.inktomisearch.com Disallow: / User-agent: hyper-decontextualizer Disallow: / User-agent: scooter Disallow: / User-agent: webcopy Disallow: / User-agent: imagelock Disallow: / User-agent: w3c_validator Disallow: / User-agent: LinkScan Server Disallow: / User-agent: Linkbot Disallow: / User-agent: LinkVerify Spider Disallow: / User-agent: NetLookout Disallow: / User-agent: LinkSweeper Disallow: / User-agent: NetMechanic Disallow: / User-agent: NetMonitor Disallow: / User-agent: SpurlBot Disallow: / User-agent: WebTrends Link Analyzer Disallow: / User-agent: online link validator Disallow: / User-agent: Weblink Scanner Disallow: / User-agent: Robozilla Disallow: / User-agent: Netprospector JavaCrawler Disallow: / User-agent: SiteBar Disallow: / User-agent: WebPictures Downloader Disallow: / User-agent: WebLeacher Disallow: / User-agent: X-Uploader Disallow: / User-agent: wget Disallow: / User-agent: W3CRobot/ Disallow: / User-agent: ExtractorPro Disallow: / User-agent:ParaSite Disallow: / User-agent: SuperBot Disallow: / User-agent: Webrobot Disallow: / User-agent: Website eXtractor Disallow: / User-agent: URL2File Disallow: / User-agent: WebSnatcher Disallow: / User-agent: WebStripper Disallow: / User-agent: WebTwin Disallow: / User-agent: WebZIP Disallow: / User-agent: WWWOFFLE Disallow: / User-agent: Xaldon WebSpider Disallow: / User-agent: CyberNavi_WebGet Disallow: / User-agent: EmailWolf Disallow: / User-agent: Convera Internet Spider Disallow: / User-agent: Expired Domain Sleuth Disallow: / User-agent: Excalibur Internet Spider Disallow: / User-agent: InternetArchive Disallow: / User-agent: ipiumBot Disallow: / User-agent: metabot Disallow: / User-agent: Mister Pix II Disallow: / User-agent: NetZippy Disallow: / User-agent: ORA_checksite Disallow: / User-agent: Oxxbot1 Disallow: / User-agent: NZBot Disallow: / User-agent: PICgrabber Disallow: / User-agent: Phoaks Disallow: / User-agent: SQ Webscanner Disallow: / User-agent: SpotOn Disallow: / User-agent: Steganos Internet Anonym Disallow: / User-agent: SurfControl Disallow: / User-agent: UtilMind HTTPGet Disallow: /