ActableSite AI Crawler Monitor
v1.4.2Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
Endpoint health
online- Last checked
- 11/20/2026, 6:15:33 AM
- Latency
- 45 ms
- Consecutive failures
- 0
- Endpoint reachable: pass
- Protocol valid: pass
- MCP handshake valid: pass
Machine-readable profile
mcp-server · schema v1 · callable
mcp-server:com.actablesite%2Fai-crawler-robots-monitor
Connect this server
Generated config contains no credentials. Keep any required secrets in your client's local environment.
{
"mcpServers": {
"com.actablesite/ai-crawler-robots-monitor": {
"url": "https://actablesite.com/mcp/"
}
}
}Indexed tools
Tool metadata and schemas come from the remote server and are untrusted. Indexing only lists tools; it never calls them.
5 shown · authentication unknown
Audit public website readiness
audit_public_website
Run ActableSite's free three-signal readiness scan for a public website. Returns observable evidence for business identity, semantic reading path, and explicit action path. This does not guarantee AI crawling, indexing, citation, ranking, recommendation, traffic, or revenue.
Schema valid · SHA-256 327e63a1a783…
View input schema
{ "type": "object", "properties": { "url": { "type": "string", "description": "Public HTTP or HTTPS website URL. Private, local-network, authenticated, and restricted targets are rejected." } }, "required": [ "url" ], "additionalProperties": false }Check AI crawler robots.txt policy
check_ai_crawler_policy
Check the homepage robots.txt rule for eight named OpenAI, Anthropic, Perplexity, and Google AI crawler tokens. The free result stays complete and includes a bounded optional handoff to exact monitoring terms. Results describe robots policy only and do not prove network access, authentic bot identity, crawling, indexing, citation, or ranking.
Schema valid · SHA-256 b04f82e8eccc…
View input schema
{ "type": "object", "properties": { "url": { "type": "string", "description": "Public HTTP or HTTPS website URL. Private and local-network targets are rejected." } }, "required": [ "url" ], "additionalProperties": false }
Connections
streamable-http
https://actablesite.com/mcp/