> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapeunblocker.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Errors

> HTTP status codes returned by ScrapeUnblocker and what they mean.

ScrapeUnblocker uses standard HTTP status codes. `2xx` means the page was delivered, `4xx` points at the request or at the target site, and `5xx` means a problem on our side or a request that ran out of time. Only a few responses are **billed** - they are marked in the table; everything else is free.

## Status codes

| Code | Meaning | Billed | Where to look |
| - | - | - | - |
| `200` | The page was delivered. Also used for a dead origin's own `5xx` error page and for the target's own `401`, `451` or empty `500` - then the `X-Origin-Status` header carries the target's status. A target that answers `204` (no content) comes back as an empty `200`. See [Dead URLs](#dead-urls-x-origin-status) | Yes | Response body; `X-Origin-Status` if present |
| `400` | Bad request: the `x-scrapeunblocker-key` header is missing, the URL is malformed or not `http`/`https`, an unsupported `proxy_country`, or `Hostname cannot be resolved` (the domain does not exist) | Only `Hostname cannot be resolved` | Check the URL and the key header |
| `401` | Authentication problem - the key is not recognised, or the account behind it has no valid subscription | No | See [401 unauthorized](#401-unauthorized) below and [Authentication](https://docs.scrapeunblocker.com/authentication.md) |
| `402` | Billing problem - quota exceeded, credit limit exceeded, repeated payment failures, or your own monthly or daily budget limit reached | No | See [402 payment required](#402-payment-required) below |
| `403` | Either the target blocked us on every bypass path, **or** the URL is a login / authentication / payment page we do not fetch (`X-SU-Error-Code: restricted_target`), or a government site (`X-SU-Error-Code: government_target`), or a secret / server configuration file such as `/.env` (`X-SU-Error-Code: secret_file_target`), or a bare IP address rather than a domain name (`X-SU-Error-Code: ip_target`), or a URL the site serves only to search-engine crawlers it verifies by IP, such as Facebook's sitemaps (`X-SU-Error-Code: crawler_only_target`), or a `steps` action tried to fill a password or payment-card field (`X-SU-Error-Code: restricted_action`), or your API key is suspended for abuse (`X-SU-Error-Code: key_suspended`, every call, until support lifts it) | No | `X-SU-Error-Code` header; [restricted targets](#restricted-targets) or [handling failures](https://docs.scrapeunblocker.com/guides/handling-failures.md) |
| `404` | With `X-Origin-Status: 404`: the target site itself says the page does not exist; the body is the target's page. See [Dead URLs](#dead-urls-x-origin-status). Without the header: a plugin found no such item, or the API path is wrong | Only with `X-Origin-Status` | `X-Origin-Status` header, then the body |
| `410` | The target site itself says the page is gone (`X-Origin-Status: 410`); the body is the target's page. See [Dead URLs](#dead-urls-x-origin-status) | Yes | Drop the URL |
| `413` | `/getImage`: the file is larger than 50 MB | No | - |
| `415` | `/getPageSource`: the URL is not an HTML document (an image, PDF or file download) - use `/getImage` for images. `/getImage`: the URL is not an image (for example a page); `X-Resource-Content-Type` names what it is | No | Fix the URL or the endpoint |
| `422` | Validation error - missing required field or wrong type. With `steps`, also `step_failed` when an action could not run. (With `parsed_data=true`, a page that rendered but held no structured data is not an error: it is a `200` with `data_extracted: false` and the page's `html` - see [parsed data](https://docs.scrapeunblocker.com/guides/parsed-data.md#when-nothing-can-be-extracted)) | Only `step_failed` (the page was rendered) | The response body pinpoints the problem field or step |
| `429` | Concurrent requests limit exceeded (body `Concurrency limit exceeded`, headers `X-SU-Deny-Reason: concurrency` and `Retry-After: 1`) | No | Wait `Retry-After` seconds for a running request to finish, then retry. See [concurrency limits](https://docs.scrapeunblocker.com/rate-limits.md) |
| `500` | Internal error on our side | No | Retry with backoff |
| `502` | Our gateway lost the connection to the scraping backend (HTML body). Dedicated plugin endpoints also return `502` with a plain-text message when the target returned no usable results or the call did not finish in time (`... timed out.`) | No | Retry with backoff |
| `503` | No API capacity right now (`No server is available to handle this request`, or `Render unavailable.`) - a problem on our side, not the target's | No | Retry with backoff |
| `504` | The request reached its time cap: `Request timed out.` (60 s on `/getPageSource`, 90 s with `steps`, 75 s on `/getImage`) or a SERP timeout. Usually a slow target | No | Retry. If persistent for one domain, contact support |
| `523` | `Origin unreachable` - the target answered nobody over any of our exits (down, or refusing connections in bursts). Not a bot block. A target that accepts the connection but is slow to answer is not a `523`: we keep waiting for it within the time cap | No | Retry after a pause |

## Dead URLs: `X-Origin-Status`

When the target site itself answers that the page does not exist (`404` or `410`), that answer **is** the result for your URL - we reached the site, the URL is just dead. ScrapeUnblocker returns the target's own status (`404` or `410`) with the page exactly as the target served it (the body can be empty; with `get_cookies=true` it is the usual JSON, and with `parsed_data=true` the body is `{"data": {"page_type": "not_found", "data": {}}}` - there is nothing to extract from a not-found page) and adds:

| Header | Value |
| - | - |
| `X-Origin-Status` | The target's own status, e.g. `404`, `410`, `502` |
| `X-ScrapeUnblocker-Note` | A short human-readable explanation |

These calls are billed like any delivered page, and **retrying will not change the answer**. `X-Origin-Status` is what tells the target's `404` apart from a `404` of our own (a wrong API path) - branch on it. On `/getImage` a dead image comes back the same way, with an empty body:

```python theme={null}
r = requests.post(
    "https://api.scrapeunblocker.com/getPageSource",
    params={"url": url},
    headers={"x-scrapeunblocker-key": "YOUR_API_KEY"},
    timeout=125,
)
if r.status_code in (404, 410) and r.headers.get("X-Origin-Status"):
    mark_as_dead(url)  # the page no longer exists on the target site
```

When the target's origin server returns its own `5xx` error page (for example nginx `502 Bad Gateway`), you get a `200` with that page, `X-Origin-Status` carrying the origin's status and `X-Upstream-Unavailable: true`, also billed. It stays a `200` on purpose: most HTTP clients retry `5xx` automatically, and every retry would be billed.

The target's own `401` (the page needs a login), `451` (unavailable for legal reasons) and a `500` with no body at all come back the same way, once a second exit has seen the same answer: a `200` with the body exactly as the target served it (often empty) and `X-Origin-Status: 401`, `451` or `500`, billed. They are never passed through as the status itself - our own `401` always means a problem with your API key, and our own `500` an error on our side.

> [!NOTE]
>
> Until 2026-09-30 a target's `404` / `410` also came back as a `200` with `X-Origin-Status`. Code that checks `r.status_code == 200` first should now handle `404` / `410` with the header as a dead URL, not as an error to retry.

A URL whose **domain** does not exist gets `400 Hostname cannot be resolved` instead, also billed.

## Served variant: `X-SU-Served-Url`

A few sites refuse automated access to a page but serve the same content through an alternate rendering of it, such as the page's AMP version. For those, `/getPageSource` fetches the alternate rendering and says so:

| Header | Value |
| - | - |
| `X-SU-Served-Url` | The URL that was actually fetched, e.g. `https://www.olx.co.id/amp/motor-bekas_c200/q-vespa` for `https://www.olx.co.id/motor-bekas_c200/q-vespa` |

The body is that page, billed like any delivered page. The alternate rendering carries the same listing but can be lighter than the regular page (fewer items, less markup). The header is absent on every other call.

## Is it the target, or is ScrapeUnblocker down?

If you run failover to another provider, only these mean a problem on our side: connection errors, read timeouts, `500`, `502`, `503`, and `504` when it hits many different domains at the same time. All other codes above describe the target site or the request.

* A call never hangs indefinitely: if our gateway is unreachable you get a connection error, and if no scraping capacity is available you get an immediate `503`. A healthy call takes at most its time cap plus up to 60 s of queueing when all capacity is busy. Use a connect timeout of about 10 s and a read timeout of about 125 s.
* Decide on a rate, not on a single response - one failed call is often one difficult domain. For example, switch over when at least half of the calls in the last minute (at least 20 calls, across at least 3 domains) end with one of those signals, and switch back once probe calls succeed.
* Responses our gateway generates itself (`502`, `503`, `504`) are never billed.

## 401 unauthorized

A `401` means the request was rejected at our edge, before it ever reached the scraping engine. Because nothing was scraped, a `401` does not count against your quota and is never billed.

Unlike validation errors, `401` responses have a plain-text body (`Content-Type: text/plain`), not JSON. There are two distinct messages, and they mean different things:

> [!NOTE]
>
> The first line of a `400`, `401` or `402` body is the message, and it does not change. Some bodies add a line or two after it with where to get a key or more requests. Match on the first line (or check whether the body starts with it), not on the whole body.

| Response body | Meaning |
| - | - |
| `Unauthorized` | The key you sent is not recognised |
| `No valid subscription` | The key is recognised, but the account behind it has no active subscription |

### `Unauthorized`

```
HTTP/1.1 401 Unauthorized
Content-Type: text/plain

Unauthorized
```

The value of your `x-scrapeunblocker-key` header does not match any known key. Common causes:

* **A typo, or a truncated copy-paste.** Keys are long; make sure the whole value was copied.
* **Trailing whitespace or a newline** in the header value - especially when the key is read from a file rather than an environment variable.
* **An empty header value.** Sending `x-scrapeunblocker-key` with nothing after it counts as an unknown key, not as a missing header.
* **A rotated or revoked key.** After you generate a new key in the [dashboard](https://app.scrapeunblocker.com), the old one stops working once its short grace period ends.
* **Environment mismatch.** Production keys only work against `api.scrapeunblocker.com`. A key issued for one environment sent to another is an unknown key there.

### `No valid subscription`

```
HTTP/1.1 401 Unauthorized
Content-Type: text/plain

No valid subscription
Pick a plan: https://www.scrapeunblocker.com/pricing
AI agent? With your user's OK, buy credits yourself: https://agents.scrapeunblocker.com
```

The key itself is valid, but the account it belongs to currently has no subscription period covering today - for example the free trial has ended and no plan was chosen, or a plan lapsed and was not renewed. Pick a plan in the [dashboard](https://app.scrapeunblocker.com) and access resumes within about a minute; no key change is needed.

> [!NOTE]
>
> A billing problem on an **active** subscription returns `402`, not `401` - quota exceeded, credit limit exceeded, or a card that failed repeatedly. `401` is strictly about who you are, `402` about what you owe.

### Missing header returns `400`, not `401`

If you omit the `x-scrapeunblocker-key` header entirely, the response is `400 Bad Request` with a body that starts with `Missing x-scrapeunblocker-key`. This is deliberate: it separates "you forgot to authenticate" from "you authenticated, and it was rejected", so client code can tell a wiring bug from a credential problem.

```
HTTP/1.1 400 Bad Request
Content-Type: text/plain

Missing x-scrapeunblocker-key
Sign up free (500 requests): https://www.scrapeunblocker.com
AI agent? With your user's OK, buy a key yourself: https://agents.scrapeunblocker.com
```

## 402 payment required

A `402` means your key and account are recognised and in good standing as credentials - the request was stopped for a **billing** reason. Like `401`, it is refused at our edge before anything is scraped, so a `402` consumes no quota and is never billed.

The body is plain text (`Content-Type: text/plain`), not JSON. There are four messages:

| Response body | Meaning | Fix |
| - | - | - |
| `Quota exceeded` | You have used every request your plan allows this billing period | Upgrade your plan, or wait for the period to reset |
| `Credit limit exceeded` | Your unpaid balance has grown past your account's credit limit | Pay the outstanding invoice |
| `User set budget exceeded` | This billing period's spend has reached the monthly budget limit you set, or today's spend has reached your daily budget limit | Raise or remove the limit in your profile, or wait for the next billing period (monthly limit) or 00:00 UTC (daily limit) |
| `Payment failed - update payment method` | A card payment for an open invoice has failed three times in a row | Update your card, then pay the invoice |

If more than one applies, the most serious wins: payment failure outranks credit limit, which outranks quota, which outranks your own budget limit.

> [!NOTE]
>
> All four clear themselves. Our gateway refreshes key statuses about once a minute, so once you upgrade, the invoice is paid or you raise your budget limit, access comes back within roughly a minute - no key change, no support ticket, no redeploy.

### `Quota exceeded`

```
HTTP/1.1 402 Payment Required
Content-Type: text/plain

Quota exceeded
Upgrade or add requests: https://www.scrapeunblocker.com/pricing
AI agent? With your user's OK, buy credits yourself: https://agents.scrapeunblocker.com
```

Your usage for the current billing period has passed your plan's quota. If your plan allows overages, this only fires once you are past quota **plus** the overage allowance - inside that band requests still succeed and the extra usage is invoiced. Any active coupon credit is spent before plan quota, so a key with remaining credit is never quota-blocked.

The counter resets at the start of your next billing period, which starts on your subscription's anniversary day, not on the first of the month. To get moving sooner, upgrade in the [dashboard](https://app.scrapeunblocker.com) - the new quota applies on the next status refresh. Current usage against quota is visible in the dashboard, so this is the one `402` you can see coming.

### `Credit limit exceeded`

```
HTTP/1.1 402 Payment Required
Content-Type: text/plain

Credit limit exceeded
```

This applies to accounts that accrue usage-based charges. We add up what you currently owe - the amount remaining on your open invoices, plus metered usage already consumed on active subscriptions but not yet invoiced - and compare it against your account's credit limit. Past the limit, the key is paused.

When this triggers we also finalise and attempt payment on the outstanding invoices automatically, so in the common case where your card is good it settles itself and access returns within about a minute. If payment does not go through, pay the invoice from the [dashboard](https://app.scrapeunblocker.com). A higher credit limit can be arranged through [support](https://scrapeunblocker.com/contact).

### `User set budget exceeded`

```
HTTP/1.1 402 User set budget exceeded
Content-Type: text/plain

User set budget exceeded
```

You set a **monthly budget limit** in your [profile](https://app.scrapeunblocker.com/dashboard/profile), and this billing period's spend has reached it. Spend is counted the way your invoice is: your plan's fixed monthly fee, if it has one, plus the requests billed on top of it, excluding VAT. Requests paid from coupon credit do not count. We also email you once when the limit is reached.

On a plan with a fixed monthly fee the limit cannot be lower than that fee, and the requests the fee already includes are never blocked - a limit equal to the fee simply stops the key where overage billing would begin. Leave the field empty or set it to `0` for no limit.

You can also set a **daily budget limit** on any plan that bills usage - pay-as-you-go, or a plan with a fixed monthly fee and overages. It counts today's spend (the UTC day) the same way. On a plan with a fixed fee only the requests billed above the plan's included requests count, because the fee already pays for the rest. Once today's spend reaches the limit, the key returns the same `402` until 00:00 UTC or until you raise or remove the limit, and we email you once that day.

The key works again at the start of your next billing period, or within about a minute after you raise or remove the limit. Usage is counted with a delay of a few minutes, so the period's spend can end up slightly above the limit.

### `Payment failed - update payment method`

```
HTTP/1.1 402 Payment Required
Content-Type: text/plain

Payment failed - update payment method
```

An invoice on your account is open and its payment has been attempted and declined three times. Those attempts are our payment provider's automatic retries spread over several days, so reaching this state means a card has been failing for a while - typically expired, cancelled, or short of funds.

Update your payment method in the [dashboard](https://app.scrapeunblocker.com) and settle the open invoice. As soon as the invoice is paid the block lifts on the next status refresh, within about a minute.

> [!WARNING]
>
> Subscribing to a new plan does **not** clear this on its own. The old unpaid invoice stays open, so the block stays in place until that specific invoice is paid, even if the new subscription is active and paid for.

### `402` versus its neighbours

| Code | It means |
| - | - |
| `401` | We do not accept your credentials - unknown key, or no subscription at all. See [401 unauthorized](#401-unauthorized) |
| `402` | We accept your credentials; the account owes money, is out of quota, or hit the budget limit you set |
| `429` | Nothing is wrong with the account - you have more requests in flight at once than your plan's concurrent requests limit. See [concurrency limits](https://docs.scrapeunblocker.com/rate-limits.md) |

## 422 validation error shape

When you send an invalid request body, `/getPageSource`, `/serpApi`, and `/getImage` all return a structured validation error:

```json theme={null}
{
  "detail": [
    {
      "loc": ["query", "url"],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}
```

`loc` is the path to the problem field. `msg` is human-readable. `type` is a stable machine-readable identifier.

## Restricted targets

ScrapeUnblocker does not fetch sign-in or payment surfaces, government sites or
secret files, and it does not fill credential or payment forms. Four refusals
enforce this. A fifth answers at once a URL that no client but a verified
search-engine crawler can fetch, instead of failing it after slow retries.
All are returned at our edge (or before any action runs), are **never billed**,
and carry an `X-SU-Error-Code` header so you can tell them apart from an
ordinary bot block.

| `X-SU-Error-Code` | When | Endpoints |
| - | - | - |
| `restricted_target` | The URL is a login, authentication or payment / checkout page - a known identity or payment host (for example `accounts.google.com`, `checkout.stripe.com`), or a path whose segment is one of `login`, `sign-in`, `oauth`, `checkout`, `payment`, and similar. The request is refused before a browser is launched | `/getPageSource`, `/getImage` |
| `government_target` | The URL is a government, military or intergovernmental site (for example `*.gov`, `*.mil`, `*.int`, `gov.uk`, `gouv.fr`, `go.jp`), or a URL that only wraps one: a redirect link whose query carries the government URL, a `*.translate.goog` proxy page, or an archive copy such as `web.archive.org/web/.../https://www.irs.gov/`. The request is refused before anything is fetched | `/getPageSource`, `/getImage` |
| `secret_file_target` | The URL asks for a secret or server configuration file: a path segment that is `.env` or a variant (`.env.local`, `.env.production`, ...), a `.git`, `.svn`, `.aws` or `.ssh` directory, `.htpasswd`, `.npmrc`, `wp-config.php` or a copy of it, a private key (`id_rsa`, `*.pem`, `*.p12`), `phpinfo.php` or a Spring Boot `actuator/env` / `heapdump`. Also when a redirect or archive URL wraps one. Pages that only mention such a name (`/blog/what-is-a-dotenv-file`) are fetched as usual. The request is refused before anything is fetched | `/getPageSource`, `/getImage` |
| `ip_target` | The URL's host is an IP address in any notation, an alias of one on a wildcard-DNS service such as `nip.io`, or a URL that only wraps one. Use the site's domain name | `/getPageSource`, `/getImage` |
| `crawler_only_target` | The site serves this URL only to search-engine crawlers it verifies by IP address and answers every other client, any proxy or browser included, with an empty `403`. Today: the sitemap files under `facebook.com/sitemap/` that Facebook's `robots.txt` lists. The site's other pages are fetched as usual. The request is answered before anything is fetched | `/getPageSource` |
| `restricted_action` | A [`steps`](https://docs.scrapeunblocker.com/guides/browser-steps) `type` action is about to fill a field the page marks as a password or a payment-card field. The run stops on that step | `/getPageSource` with `steps` |

A `restricted_target` refusal is a plain-text `403`:

```
HTTP/1.1 403 Forbidden
Content-Type: text/plain
X-SU-Error-Code: restricted_target

This target is a login, authentication or payment page. ScrapeUnblocker does not fetch sign-in or payment surfaces...
```

A `restricted_action` refusal is a JSON `403` naming the step:

```json theme={null}
{
  "error": "restricted_action",
  "detail": "This step would enter a password or payment-card field. ScrapeUnblocker does not fill credential or payment forms.",
  "step_index": 1,
  "field": "password"
}
```

Matching is deliberately narrow: a whole path segment must match, so an article at
`/payment-policies` or `/how-to-login-safely` is fetched normally - only the login
or payment surface itself is refused. Ordinary `steps` into search boxes, filters
and forms are unaffected. If you have a legitimate need for such a target, contact
[support](https://scrapeunblocker.com/contact).

## 403 - blocked vs. invalid key

A `403` from ScrapeUnblocker **never** means your API key is wrong. Invalid keys return `401`. Otherwise a `403` is one of two things, which the `X-SU-Error-Code` header tells apart: a [restricted target](#restricted-targets) (the header is set), or the target site blocked us on every bypass route we tried (no such header).

When you see a `403` with no `X-SU-Error-Code` (a bot block):

1. **Try a different `proxy_country`.** Some sites geo-fence or geo-rotate their bot protection. A US site may be unreachable from EU IPs and vice versa.
2. **Wait and retry.** Rate-based blocks expire after a few minutes.
3. **Contact support** if the same URL repeatedly fails - we may need to add a custom plugin for that domain.

A `403` **with** an `X-SU-Error-Code` is a policy refusal, not a bot block - retrying or changing `proxy_country` will not help. See [restricted targets](#restricted-targets).

More detail in the [handling failures guide](https://docs.scrapeunblocker.com/guides/handling-failures.md).

## Retries and idempotency

All three endpoints are safe to retry. Requests are idempotent in the sense that retrying with the same parameters does not double-charge or create duplicate state on your account. We recommend exponential backoff for transient `5xx` errors and `523` (failed calls are free, so a retry costs nothing extra):

```python theme={null}
import time
import requests

def fetch_with_retry(url, max_attempts=3):
    for attempt in range(max_attempts):
        r = requests.post(
            "https://api.scrapeunblocker.com/getPageSource",
            params={"url": url},
            headers={"x-scrapeunblocker-key": "YOUR_API_KEY"},
            timeout=125,
        )
        if r.status_code == 200:
            return r
        if r.status_code in (500, 502, 503, 504, 523) and attempt < max_attempts - 1:
            time.sleep(2 ** attempt)
            continue
        r.raise_for_status()
    return r
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.