Atalaya AI crawler check
v1.1.0Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.
Endpoint health
online- Last checked
- 11/20/2026, 9:16:05 AM
- Latency
- 123 ms
- Consecutive failures
- 0
- Endpoint reachable: pass
- Protocol valid: pass
- MCP handshake valid: pass
Machine-readable profile
mcp-server · schema v1 · callable
mcp-server:com.atalaya-ai%2Fai-crawler-check
Connect this server
Generated config contains no credentials. Keep any required secrets in your client's local environment.
{
"mcpServers": {
"com.atalaya-ai/ai-crawler-check": {
"url": "https://atalaya-ai.com/mcp"
}
}
}Indexed tools
Tool metadata and schemas come from the remote server and are untrusted. Indexing only lists tools; it never calls them.
2 shown · authentication unknown
Check AI crawler access
check_ai_crawler_access
Checks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and more) robots.txt allows and the exact rule for each, how much text the homepage HTML carries for crawlers that do not run JavaScript (flags client-rendered app shells, common in apps built with Lovable, Bolt, Base44 or Replit), title/meta description/H1, llms.txt, sitemap, and the platform or host setting to change (Cloudflare, Vercel, Netlify). Use it when someone asks why their site or app does not show up in ChatGPT, Claude, Perplexity or other AI answers.
Schema valid · SHA-256 ef9abd0ea9de…
View input schema
{ "type": "object", "properties": { "domain": { "type": "string", "description": "Public domain or URL, for example example.com" }, "lang": { "type": "string", "enum": [ "en", "es" ], "description": "Language of the verdict text (default en)" } }, "required": [ "domain" ], "additionalProperties": false }List known AI crawlers
list_ai_crawlers
Lists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can write a robots.txt that keeps AI search while opting out of training.
Schema valid · SHA-256 c4dd144f24ed…
View input schema
{ "type": "object", "properties": {}, "additionalProperties": false }
Connections
streamable-http
https://atalaya-ai.com/mcp