Marketing

crawlreadyai.com

An SEO and AI crawl readiness auditing tool that checks website access for AI crawlers, robots.txt, llms.txt, sitemaps, and indexability, and compares sites against competitors.

WebMCP1 endpoint4 known toolsFirst detected September 12, 2026Last detected September 12, 2026

ENDPOINT 1

https://crawlreadyai.com

No auth detected

MCP server metadata

Name: WebMCP

Capabilities

declarativeimperative

Known tools 4

auditWebsite

Runs a free AI and SEO crawl readiness audit on a website URL. Returns AI crawler access, robots.txt status, llms.txt presence, sitemap health, and indexability results.

Inferred read-only

run_audit

Navigate to the CrawlReady AI free audit page so the user can check a website for AI crawler access, robots.txt, llms.txt, sitemap, and SEO issues.

Inferred read-only

compare_websites

Navigate to the website comparison tool to compare AI and SEO crawl readiness between your site and competitors.

Inferred read-only

view_free_tools

Navigate to the CrawlReady AI free SEO and AI-readiness tools directory.

Inferred read-only

CONNECT WITH APPROVAL

Client installation

Review this server and its permissions before adding it. Secret placeholders must be set locally.

Codex

~/.codex/config.toml

[mcp_servers.webmcp]
url = "https://crawlreadyai.com"
enabled = true

Claude Code

.mcp.json

{
  "mcpServers": {
    "webmcp": {
      "type": "http",
      "url": "https://crawlreadyai.com"
    }
  }
}

Claude Desktop

Settings → Connectors → Add custom connector

Name: webmcp
Remote MCP URL: https://crawlreadyai.com

Add this remote URL as a custom connector in Claude Desktop. Availability depends on the user plan and workspace policy.

Cursor

.cursor/mcp.json

{
  "mcpServers": {
    "webmcp": {
      "url": "https://crawlreadyai.com"
    }
  }
}

Visual Studio Code

.vscode/mcp.json

Add to Visual Studio Code

{
  "servers": {
    "webmcp": {
      "type": "http",
      "url": "https://crawlreadyai.com"
    }
  }
}

Generic MCP

Client-specific MCP configuration

{
  "name": "webmcp",
  "transport": "streamable-http",
  "url": "https://crawlreadyai.com"
}

MCP Inspector

Run the official MCP Inspector locally and enter the indexed Streamable HTTP endpoint.

TRUST AND VERIFICATION EVIDENCE

Loading Trust v2 evidence…

Checking the associated registrable domain. The BuiltWith key remains server-side.

Indexed

Evidence is source-attributed and does not guarantee that a third-party server is safe. Risk labels are conservative metadata heuristics.