API quickstart

Create a key

Sign in and open API keys on your dashboard. Name the key, choose when it expires (30, 90 or 365 days) and copy it once; it starts with dwb_. Store it as an application secret and send it as Authorization: Bearer <key>. Never put it in a URL.

The API is at https://api.datawebot.com. Every account starts with 200 trial credits.

Find a blueprint

A blueprint is one store and one operation, such as product details or search. Search by store, domain or operation and copy a blueprint_id from the response. Each blueprint carries its input_schema, record_schema, price and availability.

curl "https://api.datawebot.com/v1/job-blueprints?q=based.com" \
  -H "Authorization: Bearer $DATAWEBOT_API_KEY"

Submit once

One Job is one URL. Choose an Idempotency-Key and save it before you send the request. A successful submission returns HTTP 202 with job_id. If the response is lost, send the same body with the same key to get the same Job back; a different body with that key returns 409 idempotency_conflict.

curl "https://api.datawebot.com/v1/jobs" \
  -H "Authorization: Bearer $DATAWEBOT_API_KEY" \
  -H "Idempotency-Key: first-job-0001" \
  -H "Content-Type: application/json" \
  -d '{"blueprint_id":"s_based_com.product_details","url":"https://based.com/products/<handle>"}'

Wait and retrieve

GET /v1/jobs/{id} until status is COMPLETED, PARTIAL_COMPLETE, FAILED or UNAVAILABLE. Wait at least poll_after_seconds between polls and set an upper bound on the total wait.

Then GET /v1/jobs/{id}/results and follow next_cursor until it is null. Results are kept for seven days.

curl "https://api.datawebot.com/v1/jobs/$JOB_ID/results" \
  -H "Authorization: Bearer $DATAWEBOT_API_KEY"

Handle errors by code

Errors return {"error":{"code","message","retryable"}}. Retry only when retryable is true, after Retry-After when it is sent. insufficient_credits (402) means the balance can't cover the Job's credit hold; ask for fewer pages. Quote the X-Request-Id response header when you contact support.