How to paginate Business Search results

Paginate Business Search results with offset and limit: keep the query fixed, add limit to offset, stop on a short page and deduplicate on bright_id.

This guide collects every record that matches one Business Search query by repeating the same search with a growing offset until a page comes back short or empty.

Prerequisites

• A Bright Data API key with Business Search access, stored in the BRIGHTDATA_API_KEY environment variable as in Step 1 of the quickstart
• A query whose first page already returns the records you want. Tune the conditions with Business Search query syntax before you paginate

Step 1: Send the first page

Send the search with offset set to 0 and limit set to the page size. limit defaults to 10 and its maximum depends on the mode:

Mode	Maximum limit per page
ludicrous	100
instant	100
smart	10

How far a search can page also depends on the mode:

• Ludicrous accepts offset values up to 10,000, so one search can page through up to 10,100 records.
• Instant and Smart expose only the first 100 matches of a search. offset plus limit cannot exceed 100, so paginate inside that window or run the same conditions as a structured query in Ludicrous mode.

This request asks for the first 100 matches of a Ludicrous company search. The highlighted lines set the page:

cURL

curl --request POST "https://api.brightdata.com/search/company" \
  --header "Authorization: Bearer $BRIGHTDATA_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "source": "linkedin_company",
    "mode": "ludicrous",
    "query": {
      "and": [
        { "text": { "industry": "software" } },
        { "equals": { "headquarters_country_code": "US" } },
        { "range": { "employees_in_linkedin": { ">=": 50, "<=": 500 } } }
      ]
    },
    "offset": 0,
    "limit": 100,
    "view": "summary"
  }'

The response echoes the pagination in meta.offset and meta.limit and returns the page in documents. The full meta schema is on the search companies API reference.

Step 2: Request the next page

Add limit to offset and send the body again with every other property unchanged. The category, source, mode, query and view must stay the same, because changing any of them starts a different search.

After a full first page, this request asks for records 101 to 200 of the same search. Only the highlighted offset changes:

cURL

curl --request POST "https://api.brightdata.com/search/company" \
  --header "Authorization: Bearer $BRIGHTDATA_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "source": "linkedin_company",
    "mode": "ludicrous",
    "query": {
      "and": [
        { "text": { "industry": "software" } },
        { "equals": { "headquarters_country_code": "US" } },
        { "range": { "employees_in_linkedin": { ">=": 50, "<=": 500 } } }
      ]
    },
    "offset": 100,
    "limit": 100,
    "view": "summary"
  }'

Repeat this step until the stop condition in step 3 is met.

Do not drive the loop with meta.matched. meta.matched is a lower bound on the matches the search reached, not a total, and it changes between requests. Stepping outside the mode’s window does not return an empty page either: in Instant and Smart, a request with offset plus limit above 100 returns HTTP 500 with error_code backend_search_failed, and in Ludicrous an offset above 10,000 returns HTTP 400 with error_code invalid_query.

Step 3: Stop on a short or empty page

Stop when documents comes back empty or shorter than limit. That page is the last page of the result set.

A Ludicrous request past the last match returns HTTP 200 with an empty documents array, so an empty page is the normal end of a search, not an error. If an Instant or Smart request returns HTTP 500 with error_code backend_search_failed after earlier pages succeeded, the loop has stepped past the 100-record window of those modes. Other causes of each status are listed in Business Search error codes.

An empty documents array on the first page means the search matched nothing. It is not proof that no such company or person exists.

Step 4: Deduplicate the collected records

Each page is a new search against the current index, not a slice of a fixed snapshot, so a record can move between pages while you paginate. Collect every page in one pass and deduplicate on bright_id plus source. Do not re-fetch a single page later and expect the same records.

Frequently asked questions

Can I use meta.matched as the number of results?

No. meta.matched reports how many matches the search reached. Treat it as a lower bound rather than an exhaustive count, particularly for text and natural-language searches, and expect it to grow as offset grows. Do not use it to size a dataset or to tell users that exactly that many companies or people satisfy their intent.

How do I collect more than 100 records from an Instant or Smart search?

Run the same conditions as a structured query in Ludicrous mode. Instant and Smart expose only the first 100 matches of a search, while Ludicrous pages up to offset 10,000. Business Search query syntax lists the operators and searchable fields for the structured form.

What does meta.coverage_percent mean for a paginated set?

meta.coverage_percent reports the share of the index that answered the request. A value below 100 means the result set is partial, and a results page built on that response should say so. A value of 100 does not mean every relevant record was returned or every field was populated.

Why did the same offset return different records?

Each page is a fresh search, and the index refreshes daily, so results and their order can change between requests. A record on page 1 of one run can appear on page 2 of the next. If your job needs a stable set, collect all pages in one pass and deduplicate as in step 4.