Ecommerce scraper: get product data from Amazon, Walmart, Zalando and Mercado Libre as JSON, with Python
Scrape ecommerce product data from Amazon, Walmart, Zalando and Mercado Libre with one API: price, rating, seller and IDs as JSON, plus a tested Python script.
An ecommerce scraper is a program that reads the pages of online stores and turns them into data: the title, price, rating, review count, seller and product codes of each product. The hard part is rarely one store. It is the second, third and fourth, because every store lays out its pages differently, and a scraper written for one of them is no help with the next.
This guide shows how to scrape several stores with one API and one set of fields. It starts with what an ecommerce scraper reads and why stores are hard to combine, then gives a tested Python script of about 130 lines that searches one keyword in four stores at once (Amazon and Walmart in the US, Zalando in Germany and Mercado Libre in Mexico), reads the top product in each, and saves everything to two CSV files with the same columns for every store. Every number below comes from a real run on 17 November 2026.
What does an ecommerce scraper read?
Two kinds of page, and they answer different questions:
| Page | What you send | What comes back | Use it to |
|---|---|---|---|
| Search results | A keyword and a store | Every product on the results page: title, price, original price, rating, review count, image and link | See what a store shows for a query, find products, compare a category |
| Product page | A product link | One product in full: price, availability, brand, seller, product codes, variants, rating and reviews | Get the detail on a product you already know |
Most jobs need both. A search tells you which products exist and roughly what they cost; a product read tells you who sells one, which codes identify it and how many sizes or colors it comes in.
There are three ways to do the reading:
| Route | What you build | What you maintain |
|---|---|---|
| Your own scraper | An HTTP client, an HTML parser and code that finds each field on each store's pages | Request handling, retries and a parser per store that breaks when the store changes its pages |
| An ecommerce scraper API | Calls to one endpoint that take a keyword or a product link | Only your own code; the provider keeps every store's extraction working |
| An agent tool (MCP) | Nothing; your AI agent calls the tool when you ask in plain language | Nothing; you check the answers the agent gives |
The rest of this guide uses datawebot, which reads search and product pages on Amazon, Walmart, eBay, Zalando, Mercado Libre and the other stores on its scraper page, and returns the same fields from all of them as JSON, over REST for your code or MCP for your agent. The product search API page shows a search in any of those stores.
Why is scraping several stores hard?
Because the same idea is written differently in every store. In one run on 17 November 2026, a search for "crocs classic clog" came back like this from the four stores in the script below:
| Store | Country | Top result | Price | Original price | Rating (reviews) |
|---|---|---|---|---|---|
| amazon.com | US | Crocs Unisex-Adult Classic Clog | USD 29.95 | USD 49.99 | 4.8 (486,064) |
| walmart.com | US | Crocs All Terrain Classic Clog | USD 22.50 | USD 54.99 | 4.4 (1,429) |
| zalando.de | DE | Classic Clog K, heritage red | EUR 39.99 | — | — |
| mercadolibre.com.mx | MX | Sandalias Crocs Classic Clog Con Tecnología Crosslite Unisex | MXN 655.71 | MXN 1,299 | 4.9 (6,182) |
Three currencies and four page layouts, yet every row uses the same fields. That is the point of an ecommerce scraper API: price is always {"amount", "currency", "text"}, original_price is the struck-through price when the store shows one, and rating and review_count are numbers you can sort.
The product codes differ the most, because each store uses its own. The product reads returned:
| Store | Codes in details.identifiers |
|---|---|
| amazon.com | asin B0014C0LSY, parent_asin B0FBS23TFH, mpn 209450-2Y2 |
| walmart.com | walmart_item_id 12873903425, gtin 196265544721, mpn 206340-0WV |
| zalando.de | source_sku CR413G045-G11 |
They come back in one list with a type on each, so your code reads them the same way for every store. When a page shows a GTIN, the barcode number, it is in that list too, and it is how you match one product across stores. Our guides to ASIN lookup and product matching cover that step.
How do you scrape several stores with Python?
The script below takes a keyword, searches it in four stores at the same time, reads the first organic result in each, and saves two files: search.csv with every search result and products.csv with the four product reads. It uses the standard library only and needs Python 3.9 or later.
Get access
Every account starts with 200 free credits, and sign-in is by email. The quickest way is to give your coding agent (Claude Code, Cursor or similar) this line:
set up https://mcp.datawebot.com/setup.md
The agent asks for your email and you approve the sign-in from the email datawebot sends. The key is saved to ~/.datawebot/credentials, readable only by you, and the script reads it from there. On a server or in CI, put the key in an environment variable named DATAWEBOT_API_KEY from your secret store instead, and never commit it.
The script
"""Search one keyword in several online stores, read the top product in each,
and save both to CSV files with the same columns for every store.
Standard library only."""
import csv, json, os, sys, time, urllib.request
from concurrent.futures import ThreadPoolExecutor
API = "https://api.datawebot.com/v1/requests"
STORES = [ # (store, country)
("amazon.com", "US"),
("walmart.com", "US"),
("zalando.de", "DE"),
("mercadolibre.com.mx", "MX"),
]
def api_key():
key = os.environ.get("DATAWEBOT_API_KEY")
if key:
return key
# Written by the agent setup (https://mcp.datawebot.com/setup.md).
with open(os.path.expanduser("~/.datawebot/credentials")) as f:
return json.load(f)["profiles"]["https://mcp.datawebot.com"]["api_key"]
def call(method, url, body=None):
req = urllib.request.Request(url, method=method,
data=json.dumps(body).encode() if body else None,
headers={"Authorization": "Bearer " + api_key(),
"Content-Type": "application/json",
"User-Agent": "ecommerce-scraper-example/1.0"})
with urllib.request.urlopen(req, timeout=120) as r:
return json.loads(r.read())
def ask(body):
"""Send a request; when the answer is "working", poll until it is done."""
res = call("POST", API, body)
for _ in range(60):
if res.get("status") != "working":
return res
time.sleep(res.get("check_after_seconds") or 4)
res = call("GET", f"{API}/{res['request_id']}")
raise RuntimeError("still working after four minutes; try again later")
def money(price):
return (price or {}).get("amount")
def search(store, country, keyword, limit):
res = ask({"capability": "search", "query": keyword, "site": store,
"country": country, "limit": limit})
return [{
"store": store,
"rank": n,
"title": p["title"],
"price": money(p.get("price")),
"original_price": money(p.get("original_price")),
"currency": (p.get("price") or {}).get("currency"),
"rating": p.get("rating"),
"review_count": p.get("review_count"),
"url": p["url"],
"as_of": res["meta"]["as_of"],
} for n, p in enumerate(res.get("products") or [], 1)] # organic results only
def product(store, country, url):
res = ask({"capability": "product", "url": url, "country": country})
if "product" not in res: # failed reads are free; report and move on
print(f"{store}: product read failed ({(res.get('error') or {}).get('code')})")
return None
p = res["product"]
d = p.get("details") or {}
ids = {}
for i in d.get("identifiers") or []:
ids.setdefault(i["type"], []).append(i["value"])
return {
"store": store,
"title": p["title"],
"brand": d.get("brand"),
"price": money(p.get("price")),
"original_price": money(p.get("original_price")),
"currency": (p.get("price") or {}).get("currency"),
"availability": p.get("availability"),
"rating": p.get("rating"),
"review_count": p.get("review_count"),
"seller": (d.get("seller") or {}).get("name"),
"seller_type": (d.get("seller") or {}).get("type"),
"identifiers": " ".join(f"{k}:{v}" for k, vs in ids.items() for v in vs),
"variants": len(d.get("variants") or []),
"url": p["url"],
"as_of": res["meta"]["as_of"],
}
def scrape_store(store, country, keyword, limit):
rows = search(store, country, keyword, limit)
top = product(store, country, rows[0]["url"]) if rows else None
return rows, top
def save(path, rows):
with open(path, "w", newline="", encoding="utf-8") as f:
w = csv.DictWriter(f, fieldnames=list(rows[0]))
w.writeheader()
w.writerows(rows)
def main(keyword, limit="5"):
with ThreadPoolExecutor(len(STORES)) as pool: # all stores at once
results = list(pool.map(lambda s: scrape_store(*s, keyword, int(limit)), STORES))
found = [r for rows, _ in results for r in rows]
tops = [top for _, top in results if top]
save("search.csv", found)
save("products.csv", tops)
print(f"{len(found)} search results saved to search.csv, "
f"{len(tops)} product reads saved to products.csv\n")
for t in tops:
was = f", was {t['original_price']}" if t["original_price"] else ""
print(f"{t['store']}: {t['currency']} {t['price']}{was} {t['title'][:50]}")
facts = [f"rating {t['rating']} from {t['review_count']} reviews" if t["rating"] else "",
f"brand {t['brand']}" if t["brand"] else "",
f"seller {t['seller']} ({t['seller_type']})" if t["seller"] else "",
f"{t['variants']} variants" if t["variants"] > 1 else "",
t["identifiers"]]
print(" " + ", ".join(f for f in facts if f) + "\n")
if __name__ == "__main__":
main(*sys.argv[1:])
Run it with the keyword you want:
python3 ecommerce_scraper.py "crocs classic clog"
A second argument sets how many results to keep from each store; the default is 5.
How it works
STORES lists the stores to search, each with the country it serves: US for amazon.com and walmart.com, DE for zalando.de and MX for mercadolibre.com.mx. To add a store, add a line, such as ("amazon.de", "DE"); nothing else in the script changes.
scrape_store runs one store: a search, then a product read of the first result. main runs all four at once with a thread pool, so the whole run takes about as long as the slowest store rather than the sum of all four. search keeps only organic results; sponsored listings come back in a separate sponsored list, which the script leaves out. product turns the product codes into one type:value column, and skips a store with a message if its read fails, since failed reads are free.
What it printed
This is the real output of the run on 17 November 2026, which started at 01:06 UTC and took about two minutes:
20 search results saved to search.csv, 4 product reads saved to products.csv
amazon.com: USD 29.95, was 49.99 Crocs Unisex-Adult Classic Clog, Clogs for Women a
rating 4.8 from 486064 reviews, brand Crocs, seller HAS BRANDS (marketplace), 100 variants, asin:B0014C0LSY parent_asin:B0FBS23TFH mpn:209450-2Y2
walmart.com: USD 22.5, was 54.99 Crocs All Terrain Classic Clog
rating 4.4 from 1429 reviews, brand Crocs, seller Walmart.com (official), walmart_item_id:12873903425 gtin:196265544721 mpn:206340-0WV
zalando.de: EUR 39.99 CLASSIC CLOG K - Clogs - heritage red
brand Crocs, 9 variants, source_sku:CR413G045-G11
mercadolibre.com.mx: MXN 655.71, was 1299 Sandalias Crocs Classic Clog Con Tecnología Crossl
rating 4.9 from 6182 reviews
search.csv had 20 rows, five per store, with the store, rank, title, price, original price, currency, rating, review count, link and read time. products.csv had the four product reads with brand, availability, seller, seller type, product codes and variant count as well.
A few things stand out in that output. The Amazon listing groups 100 variants, colors and sizes of the clog, under one parent ASIN, and is sold by a marketplace seller, not by Amazon. The Walmart one is sold by Walmart itself. The Zalando read lists nine shoe sizes as variants. And the review counts are exact in a product read: the Amazon search page showed 486,000 reviews, rounded, while the product read returned 486,064.
How do you scrape ecommerce sites from an AI agent?
If you work in Claude, ChatGPT, Cursor or another agent that supports MCP, you do not need the script. After the setup line above, ask in plain language:
Use datawebot to search "crocs classic clog" on amazon.com, walmart.com,
zalando.de and mercadolibre.com.mx. For each store, give me the top three
organic results with price, original price, currency and rating, then read
the first one and tell me who sells it.
The agent calls search_products with site set to each store and get_product for the reads. It gets the same JSON as the REST API, so you can try a question in an agent first and move it to a scheduled script later without changing what you get. The MCP server page lists every tool and how to connect each client.
What should you check in ecommerce data?
The run above shows five things to check, whichever ecommerce scraper you use:
- The top result is not always the product you mean. For "crocs classic clog", Walmart's first result was the All Terrain Classic Clog and Zalando's was the kids' Classic Clog K. Search results are what the store chose to show. Check titles, or better, product codes, before you treat the first result as the product.
- Keep the currency with every price. Four stores gave three currencies. Never compare USD 29.95 with MXN 655.71 without converting, and keep the currency column in every file you build.
- Search order changes. Mercado Libre's first result for the same keyword was a different clog when searched about 15 minutes earlier. Store the rank and the read time with every row, and compare searches made at the same time.
- Keep sponsored and organic results apart. An earlier Walmart search for the same keyword returned 5 sponsored listings alongside the organic ones. The script keeps only organic results; if you want the ads, read the
sponsoredlist on its own. - Know who sells it. One listing was a marketplace seller on Amazon; another was Walmart selling directly. The seller and seller type columns tell them apart. Our price scraper guide goes further into prices and sellers.
How do you choose an ecommerce scraper?
Use this checklist when you compare providers. For a side-by-side of 13 options, see our comparison of ecommerce data APIs.
| Question | Why it matters |
|---|---|
| Does it return the same fields for every store? | Otherwise you write and maintain a mapping per store |
| Does it cover the stores and countries you need? | Coverage is the first thing that rules a scraper out |
| Does it return prices as numbers with their currency? | You cannot sort or convert "MX$655.71" without parsing it |
| Does it read both search and product pages? | Search finds products; product reads give the detail |
| Does it return product codes such as GTIN, ASIN and MPN? | They are how you match one product across stores |
| Are sponsored results kept apart from organic ones? | Ads mixed into results distort any ranking or price comparison |
| Is every answer labelled with the time it was read? | Without it you cannot tell a stale price from a current one |
| Are failed requests free? | You should not pay for a page that did not load |
| Can an AI agent call it, not only your code? | MCP support lets you test questions in an agent before writing code |
What does it cost?
A search costs 5 credits a page and a product read 1 credit. Failed requests are free; a page that loads but has no results is still charged. One run of the script, four searches and four product reads, costs 24 credits, so the 200 free credits that every account starts with cover eight keywords across all four stores. More credits are one-time packs with no subscription, listed on the pricing page.
Each store has its own endpoint page with an example request: Amazon, Walmart, Zalando and Mercado Libre. For the fields one store returns in depth, see our guides to the Amazon scraper, the Walmart scraper, Zalando and the Mercado Libre scraper.
Need this at volume?
If you need many keywords, whole categories or regular runs across stores, send a volume request and we will size it in credits. Our published samples show the result format from other stores.
Frequently asked questions
What is an ecommerce scraper?
A program or API that reads online store pages, search results and product pages, and returns their content as data: title, price, original price, rating, review count, seller, product codes and link. With an ecommerce scraper API you send a keyword or a product link and get JSON back, without writing a parser for each store.
How do I scrape an ecommerce website with Python?
Send a search or a product link to an ecommerce scraper API and write the fields you need to a CSV file. The script in this guide does that for four stores at once in about 130 lines of standard-library Python, for 24 credits per keyword.
Can I scrape several stores with the same code?
Yes. The search and product reads return the same fields for every store, so the script in this guide handles Amazon, Walmart, Zalando and Mercado Libre without any store-specific code. Adding a store is one line in the STORES list.
Which ecommerce sites can I scrape?
datawebot reads search and product pages on Amazon, Walmart, eBay, Zalando, Mercado Libre and the other stores listed on its scraper page, each in the countries it lists there. You set the store with site and the country with country.
How do I compare the same product across stores?
Match on product codes, not names. A GTIN, the barcode number, is the same in every store that shows it; a product read returns it in details.identifiers. Our guide to product matching covers matching when a GTIN is missing.
Does it return sponsored products?
Yes, separately. In a search, sponsored listings come back in their own sponsored list, so the products list holds only organic results.
Is there a free ecommerce scraper?
You can start free: every datawebot account gets 200 free credits with no card, enough for 40 searches or 200 product reads. After that, credits are one-time packs with no subscription.
Sources
- https://www.datawebot.com/llms.txt
- https://mcp.datawebot.com/setup.md
- https://www.datawebot.com/scraper
- https://www.datawebot.com/scraper/amazon
- https://www.datawebot.com/scraper/walmart
- https://www.datawebot.com/scraper/zalando
- https://www.datawebot.com/scraper/mercadolibre
- https://docs.python.org/3/library/csv.html
- https://docs.python.org/3/library/urllib.request.html
- https://docs.python.org/3/library/concurrent.futures.html
- https://modelcontextprotocol.io/specification/2025-06-18/server/tools
Search products in online shops from your agent or code.
Every account starts with 200 free credits, with no subscription: a search costs 5 credits and a product read 1. Failed requests are free.