aihub.dev

A curated index of the tools developers actually use to build with AI. No models — just the tooling around them: frameworks, vector databases, inference, observability, agents and more.

← All tools

XCrawl

Freemium

Data & Agents

Web data API to scrape, crawl, map and search for AI builders.

What it’s for

XCrawl turns web pages into clean Markdown or JSON through one API. It covers scraping, crawling whole sites, mapping a domain's URLs and search results, with proxies and JavaScript rendering handled for you. It also ships an MCP server and an n8n node, so agents and automation workflows can pull web data directly.

How to use it

Create a free account and copy your API key from the dashboard. Install the Python (pip install xcrawl-py) or Node SDK, or call the REST API, to scrape a URL into Markdown, crawl a site, or run a search. To use it from an agent, add the MCP server to Claude or another MCP client.

Pricing

1,000 free credits to start, no credit card required. Paid plans add more credits; see the pricing page for current rates.

Alternatives & related tools