# Streams & cursors

A **stream** is a durable, ordered, append-only collection of JSON records. Streams are immutable except for [erasures](https://capless.sh/docs/erasures.md).

```sh
H="authorization: Bearer $CAPLESS_API_KEY"

curl -s https://api.capless.sh/v1/streams -H "$H" -d '{"name":"events","validator":"entry.user_id != \"\""}'
curl -s https://api.capless.sh/v1/streams -H "$H"                       # list (by name)
curl -s https://api.capless.sh/v1/streams/events -H "$H"                # metadata: cursor (head), entryCount, …
curl -s -X PATCH https://api.capless.sh/v1/streams/events -H "$H" -d '{"validator":null}'
curl -s -X DELETE https://api.capless.sh/v1/streams/events -H "$H"      # asynchronous
```

* Names match `^[A-Za-z0-9][A-Za-z0-9_.\-]{0,127}$`. Use the name or the ID (`str_…`) in paths.
* A **validator** is a [CEL](https://capless.sh/docs/cel.md) expression over `entry` that must return `true` for every inserted record. A change (`PATCH`) applies to batches committed after it and increments `validatorVersion`.
* Deleting releases the name immediately; reads and writes return `410 stream_deleted`; storage is removed in the background (`deleting` → `deleted`), and rows are deleted from every materialization the stream fed.

## Inserting

```sh
curl -s https://api.capless.sh/v1/streams/events/entries -H "$H" \
  -d '{"idempotencyKey":"req-1","entries":[{"user_id":"u1","amount":100},{"user_id":"u2","amount":200}]}'
```

```json
{
  "batchId": "019a…",
  "committedAt": "2026-10-30T12:00:00.123Z",
  "inserted": 2,
  "duplicate": false,
  "startCursor": "cur_…",
  "committedCursor": "cur_…",
  "entries": [{ "id": "ent_…", "cursor": "cur_…" }, { "id": "ent_…", "cursor": "cur_…" }]
}
```

* **Atomic**: validation runs over all entries first; one failure rejects everything (`422 validation_failed`, failing indexes in `details.failures`).
* **Durable**: success is returned only after the batch is in object storage (R2) and appended to the stream's history. Acknowledged data is never lost.
* Batches hold 1–100 000 entries; each entry ≤ 1 MiB and ≤ 32 levels deep; request bodies ≤ 8 MiB (`413 request_too_large`). Split larger loads into several batches.
* `"returnEntryIds": false` omits the per-entry array from large responses.

## Idempotency

`idempotencyKey` (per stream, per operation type, ≤ 256 chars) makes retries safe:

* same key, identical body → the original result with `duplicate: true`;
* same key, different body → `409 idempotency_key_reused`;
* concurrent retries resolve to one commit. Keys are retained for 7 days.

## Batches and positions

A stream's history is a sequence of **batches** in commit order. A batch is either an *entries batch* (1..100 000 records) or an *erasure marker* (one position). Every item has a position `(batchId, offset)`.

* **Batch IDs** are UUIDv7 strings assigned *at commit*, never at upload. Within a stream they are strictly increasing, and lexicographic order equals commit order. The first 48 bits are the commit millisecond; clock regressions and bursts reuse the previous millisecond and increment a sequence, so the embedded time is *approximate but monotonic*.
* There is no per-entry sequence number; ordering batches is sufficient.

## Cursors

Cursors (`cur_…`, constant size) are **boundaries between items**, not items.

```text
origin │ B1[0] │ B1[1] │ B2[0] │ B2[1] │ head
       ↑       ↑               ↑
  cur(B1,0) cur(B1,1)       cur(B2,1)
```

In this two-batch stream, `afterCursor: null` reads from the origin, `cur(B1,1)` sits between `B1[0]` and `B1[1]` (it is `B1[0]`'s `cursor`), and `B2[1]`'s `cursor` is the head.

* `cur(B, k)` sits immediately before item `k` of batch `B`. An entry's `cursor` is the boundary right *after* it.
* The **origin cursor** is the boundary before everything; `afterCursor: null` means origin.
* The **head** (`stream.cursor`) is the boundary after the last committed item; `null` for an empty stream.
* Cursors and entry IDs are stable forever: compaction and physical purges rewrite storage objects, never offsets.

## Visibility: no gaps

Each stream has exactly one sequencer (a Durable Object). The head moves in the same storage transaction that appends a batch, and batch IDs are assigned in that transaction, so a reader can never observe batch *N+1* while batch *N* could still appear: **there are no uncommitted gaps**. A consumer that persists its cursor never misses a record committed before it.

## Throughput

One stream handles on the order of a thousand commits per second. Batching amortizes this (100 000 entries per commit). Shard a very hot logical stream across several streams.