Set up Bright Data with Smolagents

Connect Smolagents to Bright Data (1000+ pre-built scrapers) so your agent can crawl sites, run web searches and access data from LinkedIn and Amazon.

Building an AI startup?

You might be eligible for our Startup Program. Get fully funded access to the infrastructure you’re reading about right now (up to $20K value).

Learn more

Why Use Bright Data With Smolagents?

The Bright Data tools for Smolagents provide the following capabilities:

Web Search

• search_tool
  Search Google and get structured search results. This tool can search the web and return relevant results for your queries.

from smolagents import Tool

web_search = Tool.from_space(
    "BrightData/brightdata-search-tool",
    name="search_tool",
    description="search the web"
)

Extract

• extract_tool
  Scrape webpages and extract content as Markdown format. This tool can bypass CAPTCHA and bot detection to reliably extract data from any website.

from smolagents import Tool

extract = Tool.from_space(
    "BrightData/brightdata-scraper-tool",
    name="extract_tool",
    description="extract data from the web as markdown without getting blocked"
)

Structured Data Feeds

• data_feeds_tool
  Retrieve structured data from various platforms including LinkedIn, Amazon, Instagram, Facebook, X (Twitter), Zillow, and more.

from smolagents import Tool

data_feeds = Tool.from_space(
    "BrightData/brightdata-dataset-tool",
    name="data_feeds_tool",
    description="extract structured data from the web"
)

Supported platforms include:

• LinkedIn (profiles and companies)
• Amazon (products and reviews)
• Instagram (profiles, posts, reels, comments)
• Facebook (posts, marketplace listings, company reviews)
• X/Twitter (posts)
• Zillow (property listings)
• Booking.com (hotel listings)
• YouTube (videos)
• And many more

For more information, visit the Bright Data documentation.

How to Integrate Bright Data With Smolagents?

1. Obtain Your Bright Data API Key

   • Log in to your Bright Data dashboard.
   • Go to Account Settings.
   • Generate an API key if you haven’t already done so.
2. Installation

   Install the required packages. Important: As of December 9, 2025, these specific versions are mandatory to align with Smolagents:

   pip install smolagents
   pip install --upgrade --force-reinstall "gradio_client<2.0.0" "gradio<6.0.0"
   
3. Configure API Key

   Set your Bright Data API key as an environment variable:

   export BRIGHTDATA_API_KEY="your-api-key"
   

   Or set it in your Python code:

   import os
   os.environ["BRIGHTDATA_API_KEY"] = "your-api-key"
   
4. Obtain Your Hugging Face API Token

   When using InferenceClientModel, you need a Hugging Face token for authentication:

   • Visit Hugging Face Settings - Tokens
   • Create a new token with “Make calls to the serverless Inference API” permission
   • Set it as an environment variable:

       export HF_TOKEN="your-hf-token"
   

   Or pass it directly when initializing the model:

       model = InferenceClientModel(
           model_id="Qwen/Qwen3-Next-80B-A3B-Thinking",
           token="your-hf-token"
       )
   

   Note: Free Hugging Face accounts include inference credits. Upgrade to PRO for higher rate limits.

5. Usage

   Here’s a complete example of how to use Bright Data tools with Smolagents:

   from smolagents import CodeAgent, InferenceClientModel, Tool
   
   # Load Bright Data tools from Hugging Face Spaces
   web_search = Tool.from_space(
       "BrightData/brightdata-search-tool",
       name="search_tool",
       description="search the web"
   )
   
   extract = Tool.from_space(
       "BrightData/brightdata-scraper-tool",
       name="extract_tool",
       description="extract data from the web as markdown without getting blocked"
   )
   
   data_feeds = Tool.from_space(
       "BrightData/brightdata-dataset-tool",
       name="data_feeds_tool",
       description="extract structured data from the web"
   )
   
   # Initialize the model
   model = InferenceClientModel(model_id="Qwen/Qwen3-Next-80B-A3B-Thinking")
   
   # Create the agent with Bright Data tools
   agent = CodeAgent(tools=[web_search, extract, data_feeds], model=model)
   
   # Run the agent
   response = agent.run(
       "Improve this prompt, then search the web for it.",
       additional_args={'user_prompt': 'who is elon musk'}
   )
   
   print(response)
   

Example Use Cases

Web Research

Use the search tool to find information across the web:

agent.run("Search for the latest developments in quantum computing")

Data Extraction

Scrape and extract content from websites:

agent.run("Extract the main content from https://example.com/article")

Competitor Analysis

Extract structured data from e-commerce platforms:

agent.run("Get product details and reviews for the top-rated laptops on Amazon")

Social Media Intelligence

Retrieve data from social media platforms:

agent.run("Get the latest posts and engagement metrics from a LinkedIn company page")

Tips for Best Results

• Be specific with your prompts to help the agent understand exactly what data you need
• Combine tools for complex tasks - the agent can use search, extract, and data feeds together

For more advanced configurations and detailed API documentation, visit Bright Data’s documentation.