Skip to main content

Trigger a collection

POST /datasets/v3/trigger Error codes related to the "Trigger a collection" endpoint

Download snapshot

GET /datasets/v3/snapshot/:snapshot_id Error codes related to the "Download snapshot" endpoint

Calculate the number of parts for delivery

GET /datasets/v3/snapshot/:snapshot_id/parts Error codes related to the "Calculate the number of parts for delivery" endpoint

Fetch the input file from a triggered snapshot

GET /datasets/v3/snapshot/:snapshot_id/input Error codes related to the "Fetch the input file from a triggered snapshot" endpoint

Cancel a collection

POST /datasets/v3/snapshot/:snapshot_id/cancel Error codes related to the "Cancel a collection" endpoint

Monitor progress

GET /datasets/v3/progress/:snapshot_id Error codes related to the "Monitor progress" endpoint

Monitor delivery

GET /datasets/v3/delivery/:delivery_id Error codes related to the "Monitor delivery" endpoint

Deliver collected data to storage

POST /datasets/v3/deliver/:snapshot_id Error codes related to the "Deliver collected data to storage" endpoint

Get snapshot list

GET /datasets/v3/snapshots Error codes related to the "Get snapshot list" endpoint

How to handle 429 errors (and avoid IP blacklisting)

This guide shows you how to handle HTTP 429 (Too Many Requests) from Bright Data Datasets API and avoid automatic IP blacklisting.

What Bright Data does (from our side)

  • If your IP gets 25+ 429 responses within 5 minutes, it is automatically blacklisted.
  • After blacklisting, all API requests from that IP are blocked.

Why you get 429

You’ll receive 429 Too Many Requests when you exceed the concurrent request limits:
  • ≤ 20 inputs per request: up to 1500 concurrent requests
  • > 20 inputs per request: up to 100 concurrent requests

What you should do (customer side)

  1. Stop sending new requests immediately when you receive a 429 (don’t ignore it).
  2. Wait before retrying:
    • If Retry-After header exists, wait that many seconds
    • Otherwise use exponential backoff: 2s, 4s, 8s, 16s, 32s
  3. Reduce concurrency if you keep seeing 429s (you’re over the limit).
If you see 10+ 429s within 5 minutes, reduce your request rate right away to avoid hitting the 25/5min blacklist threshold.

Examples

If you are already blacklisted contact Bright Data Support and include the blocked IP address(es) to request whitelisting.