Polling and webhooks

Today you follow a batch by asking for it. Long polling returns the moment something changes, so a poll loop is short and cheap.

Long poll with /wait

GET /batches/{batchId}/wait?after=N&timeout=S needs batches:read. after is the last event number you have seen, starting at 0. timeout is 0 to 30 seconds and defaults to 30. The call returns as soon as the batch has an event newer than after, has reached a state that is not active, or the timeout passes.

response

{
  "batchId": "...",
  "status": "running",
  "reason": null,
  "counts": { "...": 0 },
  "assets": [ "..." ],
  "credits": { "held": 0, "settled": 0, "total": 0 },
  "resultReady": false,
  "events": [
    { "seq": 4, "type": "...", "unitKey": "...", "payload": {}, "createdAt": "..." }
  ],
  "nextAfter": 4,
  "hasMore": false,
  "timedOut": false
}

It returns everything GET /batches/{batchId} returns, plus the new events.
Pass nextAfter as after on the next call. Do not compute it yourself.
timedOut: true means nothing happened. Ask again with the same nextAfter.
hasMore: true means more than 500 events were waiting. Ask again straight away.
Stop when status is not one of queued, running, packaging, canceling, then read the result. Check resultReady before you request result.json.
Set your HTTP client's timeout above 30 seconds. 60 is a safe choice.

Status and events

Polling routes
Route	Use
`GET /batches/{batchId}`	Status, counts, credits, resultReady and nextAfter. No streamlining flag needed. Poll it every few seconds where /wait is off.
`GET /batches/{batchId}/events?after=N`	The event log, oldest first, up to 500 events a call, with `nextAfter` and `hasMore`. Each event has a `seq`, a `type`, an optional `unitKey`, a `payload` and `createdAt`.
`GET /projects/{slug}/progress`	One call for a whole project. It carries `pollAfterSeconds`, which tells you how soon to ask again and is null when nothing is queued or running, and an ETag: send it back as `If-None-Match` and an unchanged project answers 304 with no body.

Backoff and errors

Read bucket. Reads are limited to 240 a minute per key. A 30 second long poll costs about 2 a minute for one batch; polling every 5 seconds costs 12.
429. Wait the Retry-After header, then repeat. See Plans and limits.
503 result_busy. Storage is busy. Wait the Retry-After header (5 seconds) and repeat the download.
503 auth_unavailable or rate_limiter_unavailable. The server fails closed when it cannot check a credential or a limit. Retry later with growing waits.
Network errors and other 5xx. Retry with a growing wait and a cap, and add a little random jitter so many clients do not retry together. For a submit, reuse the same Idempotency-Key, so a retry cannot start a second batch.
4xx other than 429. Do not retry unchanged. Read the error code on the errors page.