LurkAPI
Docs · Web API

Web API

Any public web page as HTML, markdown or plain text, even past bot checks and captchas.

Overview

Send a URL and get the page back, even from sites that try to block bots. There are three ways to call it.

Auto (scrape) tries the cheapest step first and steps up only when the site blocks it. You pay for the step that worked.

One step at a fixed price: a plain request (fetch), a real browser (render), a browser on separate IPs (unblock), a residential IP (residential) or a browser on a residential IP (render-residential).

Your own steps (custom): list the steps to try, in order, and pay for the one that got the page.

The web scraping guide explains each step, what it costs and how failures work.

Source: any public URL.

Endpoints

Scrape a page (Auto)GET /v1/web/scrape · 1 credit; +3 if a browser on separate IPs gets it, +4 if it needs a residential IP, +13 if a browser renders it on a residential IP, +10 if a captcha is solvedAny public page, trying the cheapest step first and stepping up only if the site blocks it. 1–24 credits.
Fetch a page (plain request)GET /v1/web/fetch · 1 creditA page from one plain request, without running JavaScript. The cheapest step. 1 credit.
Render a page (browser)GET /v1/web/render · 1 credit; 11 if a captcha is solvedThe page as a real Chrome browser shows it, after its JavaScript runs. Gets past JavaScript challenges. 1 credit.
Unblock a page (separate IPs)GET /v1/web/unblock · 4 creditsPages from sites that block our usual IPs (DataDome and the like), fetched by a browser on separate IPs. 4 credits.
Fetch a page from a residential IPGET /v1/web/residential · 5 creditsA page from a residential IP in the country you pick, without running JavaScript. 5 credits.
Render a page from a residential IPGET /v1/web/render-residential · 14 credits; 24 if a captcha is solvedThe page as a real Chrome browser on a residential IP shows it, for sites that block every other step. 14 credits.
Scrape a page with your own stepsGET /v1/web/custom · 1 credit; +3 if a browser on separate IPs gets it, +4 if it needs a residential IP, +13 if a browser renders it on a residential IP, +10 if a captcha is solvedA page from the steps you choose, tried in your order. You pay only for the one that got the page.

Call it

Each endpoint is a web address. Request it with GET, put your API key in the x-api-key header, and you get JSON back. For example, Scrape a page (Auto):

curl
curl "https://api.lurkapi.com/v1/web/scrape?url=https%3A%2F%2Fsiteproxy.ruqli.workers.dev%3A443%2Fhttps%2Fbooks.toscrape.com%2F&format=markdown" \-H "x-api-key: YOUR_API_KEY"

Use it in Claude

Connect LurkAPI to Claude and ask in plain English. Each endpoint is a tool Claude can call:

From the blog

All posts