Sitemap: https://schembri.me/robots.txt User-agent: * Allow: / # Block AI crawlers, from: # https://netfuture.ch/2023/07/blocking-ai-crawlers-robots-txt-chatgpt/#New-challenges # Used for many other (non-commercial) purposes as well User-agent: CCBot Disallow: / # For new training only User-agent: GPTBot Disallow: / # Not for training, only for user requests User-agent: ChatGPT-User Disallow: / # Marker for disabling Bard and Vertex AI User-agent: Google-Extended Disallow: / # Speech synthesis only? User-agent: FacebookBot Disallow: / # Multi-purpose, commercial uses; including LLMs User-agent: Omgilibot Disallow: /