Overview Description

Instruct ChatGPT to write a robust, production-ready python web scraper utilizing requests, BeautifulSoup, and rate-limiting rules.

Prompt Command

74 words
Act as an expert Python developer. Write a complete, production-ready Python script that scrapes product information (title, price, availability, rating) from a paginated e-commerce website. The script should use requests and BeautifulSoup. It must include robust error handling, retry logic for failed requests using a backoff strategy, custom headers (User-Agent rotation), and strict rate-limiting (e.g., delay between requests) to avoid getting blocked. Please document the code thoroughly and explain how to run it.
Tip: Click the copy button to copy the prompt text.

Execution Sample Output

A representation of what the AI outputs when this prompt is executed:

# Example Scrape Success Output Log
import requests
from bs4 import BeautifulSoup

# Script running results:
# [SUCCESS] Fetching page 1...
# [SUCCESS] Parsing product listing data.
# [INFO] Scraped items count: 25. Delaying 2.0s...
# Output structure:
# {"title": "AI Coding Book", "price": "$29.99", "availability": "In Stock"}

Associated tags

No tags associated.
Similar Prompts

Related Recommendations

Explore additional top-rated prompt parameters for similar platform criteria.