# ============================================================================== # Robots.txt for Negombo Water Plus # Optimized for Global Search Engines & Generative AI Crawlers (GEO) # ============================================================================== # Search Engine Crawlers (Google, Bing, Yahoo, DuckDuckGo, Yandex, Baidu) User-agent: Googlebot User-agent: Googlebot-Image User-agent: Bingbot User-agent: Slurp User-agent: DuckDuckBot User-agent: Baiduspider User-agent: YandexBot Allow: / Allow: /images/ Allow: /uploads/ Allow: /style.css Allow: /script.js Allow: /site.webmanifest Allow: /llms.txt Allow: /llms-full.txt Allow: /sitemap.xml Disallow: /admin.php Disallow: /logout.php Disallow: /config.php Disallow: /functions.php # Generative AI & LLM Search Crawlers (ChatGPT, Claude, Perplexity, Gemini, Apple Intelligence) User-agent: GPTBot User-agent: ChatGPT-User User-agent: OAI-SearchBot User-agent: ClaudeBot User-agent: Claude-Web User-agent: anthropic-ai User-agent: PerplexityBot User-agent: Google-Extended User-agent: Applebot User-agent: Applebot-Extended User-agent: cohere-ai User-agent: Amazonbot User-agent: Meta-ExternalAgent User-agent: Bytespider Allow: / Allow: /images/ Allow: /uploads/ Allow: /llms.txt Allow: /llms-full.txt Disallow: /admin.php Disallow: /logout.php Disallow: /config.php Disallow: /functions.php # Default rule for all other crawlers User-agent: * Allow: / Allow: /images/ Allow: /uploads/ Allow: /style.css Allow: /script.js Allow: /llms.txt Disallow: /admin.php Disallow: /logout.php Disallow: /config.php Disallow: /functions.php # XML Sitemap and Host declarations Sitemap: https://negombowaterplus.com/sitemap.xml Host: negombowaterplus.com