at main 1.6 kB view raw
1User-agent: AI2Bot 2User-agent: Ai2Bot-Dolma 3User-agent: aiHitBot 4User-agent: Amazonbot 5User-agent: anthropic-ai 6User-agent: Applebot 7User-agent: Applebot-Extended 8User-agent: Brightbot 1.0 9User-agent: Bytespider 10User-agent: CCBot 11User-agent: ChatGPT-User 12User-agent: Claude-SearchBot 13User-agent: Claude-User 14User-agent: Claude-Web 15User-agent: ClaudeBot 16User-agent: cohere-ai 17User-agent: cohere-training-data-crawler 18User-agent: Cotoyogi 19User-agent: Crawlspace 20User-agent: Diffbot 21User-agent: DuckAssistBot 22User-agent: FacebookBot 23User-agent: Factset_spyderbot 24User-agent: FirecrawlAgent 25User-agent: FriendlyCrawler 26User-agent: Google-CloudVertexBot 27User-agent: Google-Extended 28User-agent: GoogleOther 29User-agent: GoogleOther-Image 30User-agent: GoogleOther-Video 31User-agent: GPTBot 32User-agent: iaskspider/2.0 33User-agent: ICC-Crawler 34User-agent: ImagesiftBot 35User-agent: img2dataset 36User-agent: imgproxy 37User-agent: ISSCyberRiskCrawler 38User-agent: Kangaroo Bot 39User-agent: meta-externalagent 40User-agent: Meta-ExternalAgent 41User-agent: meta-externalfetcher 42User-agent: Meta-ExternalFetcher 43User-agent: MistralAI-User/1.0 44User-agent: NovaAct 45User-agent: OAI-SearchBot 46User-agent: omgili 47User-agent: omgilibot 48User-agent: Operator 49User-agent: PanguBot 50User-agent: Perplexity-User 51User-agent: PerplexityBot 52User-agent: PetalBot 53User-agent: QualifiedBot 54User-agent: Scrapy 55User-agent: SemrushBot-OCOB 56User-agent: SemrushBot-SWA 57User-agent: Sidetrade indexer bot 58User-agent: TikTokSpider 59User-agent: Timpibot 60User-agent: VelenPublicWebCrawler 61User-agent: Webzio-Extended 62User-agent: wpbot 63User-agent: YouBot 64Disallow: /