User-agent: * Disallow: Crawl-delay: 10 User-Agent: adidxbot Disallow: / User-Agent: AdsBot-Google Disallow: / User-Agent: AhrefsBot Disallow: / User-Agent: AhrefsSiteAudit Disallow: / User-Agent: Amazonbot Disallow: / User-Agent: anthropic-ai Disallow: / User-Agent: Applebot Disallow: / User-Agent: AwarioRssBot Disallow: / User-Agent: AwarioSmartBot Disallow: / User-Agent: Bytespider Disallow: / User-Agent: CCBot Disallow: / User-Agent: ChatGPT Disallow: / User-Agent: ChatGPT-User Disallow: / User-Agent: Claude-Web Disallow: / User-Agent: ClaudeBot Disallow: / User-Agent: cognitiveSEO Disallow: / User-Agent: cohere-ai Disallow: / User-Agent: DataForSeoBot Disallow: / User-Agent: Diffbot Disallow: / User-Agent: dotbot Disallow: / User-Agent: FacebookBot Disallow: / User-Agent: GPTBot Disallow: / User-Agent: ImagesiftBot Disallow: / User-Agent: magpie-crawler Disallow: / User-Agent: Mediapartners-Google Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: omgili Disallow: / User-Agent: omgilibot Disallow: / User-Agent: OnCrawl Disallow: / User-Agent: peer39_crawler Disallow: / User-Agent: peer39_crawler/1.0 Disallow: / User-Agent: PerplexityBot Disallow: / User-Agent: redditbot Disallow: / User-Agent: rogerbot Disallow: / User-Agent: SemrushBot-BA Disallow: / User-Agent: SemrushBot-COUB Disallow: / User-Agent: SemrushBot-CT Disallow: / User-Agent: SemrushBot-SI Disallow: / User-Agent: SemrushBot-SWA Disallow: / User-Agent: SiteAuditBot Disallow: / User-Agent: SplitSignalBot Disallow: / User-Agent: Twitterbot Disallow: / User-Agent: yandex Disallow: / User-Agent: YouBot Disallow: / User-Agent: Google-Extended Disallow: / User-Agent: Screaming Frog SEO Spider Disallow: / User-Agent: voltron Disallow: / User-Agent: 008 Disallow: / User-Agent: Slurp Disallow: / User-Agent: CCBot Disallow: / User-Agent: ChatGPT-User Disallow: / User-Agent: GPTBot Disallow: / User-Agent: Pinterestbot Disallow: / User-Agent: Googlebot-Image Disallow: / User-Agent: AdsBot-Google Disallow: / User-Agent: NPBot Disallow: / User-Agent: SlySearch Disallow: / User-Agent: TurnitinBot Disallow: / User-Agent: The Knowledge AI Disallow: / User-Agent: 360Spider Disallow: / User-Agent: aiHitBot Disallow: / User-Agent: netEstate NE Crawler Disallow: / User-Agent: Google-Extended Disallow: / User-Agent: Omgilibot Disallow: / User-Agent: FacebookBot Disallow: / User-Agent: AwarioRssBot Disallow: / User-Agent: AwarioSmartBot Disallow: / User-Agent: ClaudeBot Disallow: / User-Agent: Claude-Web Disallow: / User-Agent: cohere-ai Disallow: / User-Agent: DataForSeoBot Disallow: / User-Agent: ImagesiftBot Disallow: / User-Agent: magpie-crawler Disallow: / User-Agent: omgili Disallow: / User-Agent: peer39_crawler Disallow: / User-Agent: peer39_crawler/1.0 Disallow: / User-Agent: PerplexityBot Disallow: / User-Agent: YouBot Disallow: / Sitemap: https://thomasrigby.com/sitemap.xml