silo-server/docs/architecture/playback-protocol-v3.md

# Playback protocol v3

The wire contract between a Silo client and a Silo server for deciding *how* a
piece of media will play, and for recovering when that decision turns out to be
wrong on the device in front of the user.

> **Status: normative.** This document, the JSON Schemas under
> [`docs/design/schemas/playback-v3/`](../design/schemas/playback-v3/), and the
> golden fixtures under `internal/playback/testdata/protocol_v3/` are the
> contract. Where this document and a client implementation disagree, the client
> is wrong. Where this document and the server implementation disagree, that is a
> bug in one of them — file it.

This is written to be sufficient on its own. A third-party client should be able
to implement playback against a Silo server from this document plus the schemas,
without reading any client repository and without reading the server source.

Paths are repository-relative; assume the repository root is the cwd.

---

## 1. Design invariants

Five properties hold everywhere in the protocol. Most of the surprising details
below follow from one of them.

**The server owns the decision.** A client reports what it can do; the server
decides what will be sent and how. Clients do not pick a bitrate ladder rung, do
not choose a container, do not decide whether to remux, and do not post a
transcode recipe. The `playback_plan` in a response is the whole instruction.

**Plan identity is deterministic.** `plan_id` is a pure function of the request
identity and the plan's own shape (§9). The same decision made twice produces
the same `plan_id`. This is what makes replans idempotent rather than merely
retried.

**Attempt keys are server-owned and opaque.** `plan_attempt_key` identifies "this
plan, on this output, with these local mutations" for loop prevention. It is a
hash the server computes. A client stores it, echoes it back, and never parses,
compares by substring, or recomputes it. Its algorithm and preimage are server
implementation details; §9 specifies only the stability clients may rely on.

**Claims are validated, not assumed.** When the server says a route preserves
Atmos, or that Dolby Vision metadata was removed, that claim was checked against
evidence the client supplied, at the strictness its evidence tier allows (§3).
The server never claims something it did not verify.

**Route events are diagnostics, not control.** A client reports what happened
(`first_frame`, `plan_failed`, `terminal`) so the server can learn; the report
never changes the session. Playback recovery goes through replan (§6), which is a
request with a response, not a fire-and-forget event.

Two consequences worth stating early, because they surprise implementers:

- `POST /playback/start` returns **201 for every outcome**, including a terminal
  refusal to play. A terminal is a decision, not a transport failure. 4xx means
  the request was malformed; it never means "this media cannot play."
- `protocol_version: 3` is carried on every body independently, and each is
  validated separately: the start request, the `client_playback_context` nested
  inside it, the replan request, the route event, the decision response, and the
  plan within it. A start request whose envelope says 3 but whose
  `client_playback_context` says otherwise is rejected. There is no negotiation
  and no fallback — a server that does not speak v3 is not a server this
  contract describes.

---

## 2. Endpoints

All paths are relative to `/api/v1`. Every endpoint requires an authenticated
user. The mutation endpoints additionally require a profile, supplied as the
`X-Profile-Id` header.

Every non-2xx response body is the standard error envelope:

```json
{"error": "<machine_code>", "message": "<human sentence>"}
```

The `error` code is the stable part; the `message` is for logs and is not
contract.

### 2.1 `GET /playback/capability`

Feature detection. Auth required; no profile needed.

| Status | Meaning |
| --- | --- |
| `200` | Capability document |
| `401` `unauthorized` | No authenticated user |

v3 is the server's only playback protocol, so `enabled` is constant `true` and
the document is always the full one:

```json
{
  "enabled": true,
  "protocol_versions": [3],
  "features": ["playback_plan_v3", "neutral_playback_v3_contract_v1", "layout_aware_passthrough", "playback_route_diagnostics",
               "device_quirks_v1", "seek_reanchor_v1", "direct_stream_resume_v1",
               "plan_source_duration_v1"],
  "deliveries": ["original_http", "server_remux_progressive", "server_remux_hls", "server_transcode_hls"],
  "transformations": [{"name": "audio_to_aac", "executor": "server", "recipe_version": "1", "validated_claims": ["audio_decode"]}]
}
```

The eight feature strings above are the full set this server version advertises:

| Feature | What it promises |
| --- | --- |
| `playback_plan_v3` | The three plan endpoints exist and behave as specified here |
| `neutral_playback_v3_contract_v1` | The server mints opaque `plan_attempt_key` values that clients only echo, and exposes `track_change` / `quality_change` as intent replans distinct from failure recovery |
| `layout_aware_passthrough` | Audio passthrough is decided from channel *layouts*, not just channel counts (§3) |
| `playback_route_diagnostics` | `POST /playback/route-events` is accepted |
| `device_quirks_v1` | Plans may carry `applied_quirks` and `runtime_corrections` (§9) |
| `seek_reanchor_v1` | The `seek_reanchor` replan operation is available (§6) |
| `direct_stream_resume_v1` | A direct route may resume mid-file rather than restarting |
| `plan_source_duration_v1` | `source.duration_seconds` is populated when known, so its absence means *unknown* rather than *unsupported* (§5) |

That last one is the reason feature detection is a list and not a version
number: without it, a client cannot tell a server that never sends the runtime
apart from a server that knows this particular file's runtime is genuinely
unknown, and both look like an absent field.

`deliveries` reports the four *server-side* delivery values, not the three
delivery classes a client negotiates in. §4 gives the folding.

`transformations` advertises only what the *installed* FFmpeg was probed for at
startup — a server without a `dovi_rpu` bitstream filter does not list
`server_dv7_to_hdr10`. A client must not assume a transformation exists because
this document names it.

`enabled` survives from the rollout period and is now constant `true`; the
negative shape was deliberately removed before v1 lock because v3 is the only
playback protocol. `reason` remains an optional diagnostic for a future
non-rollout condition, but it never changes the meaning of `enabled`.

### 2.2 `POST /playback/start`

Requests a plan. Auth + `X-Profile-Id` required. Request body cap: **256 KiB**.
Body: `StartRequestV3`
([`start-request.schema.json`](../design/schemas/playback-v3/v3/start-request.schema.json)).

| Status | Code | Meaning |
| --- | --- | --- |
| `201` | — | A decision was made. Body is `DecisionResponseV3`: either `outcome: "playable"` with a `playback_plan`, or `outcome: "adaptation_unavailable"` with a `terminal`. |
| `400` | `bad_request` | Malformed JSON, failed validation (the `message` is the validator's own text), missing `X-Profile-Id`, or `profile_id` disagreeing with the header |
| `401` | `unauthorized` | No authenticated user |
| `404` | `not_found` | `file_id` does not exist, is marked missing, or this profile cannot see it |
| `409` | `playback_attempt_reused` | This `playback_attempt_id` was already used for a *different* request |
| `426` | `client_upgrade_required` | The body does not declare the finalized v3 shape: `protocol_version: 3` plus both capability evidence markers |
| `500` | `internal_error` | Session store failure |

A file the profile is not allowed to see is `404`, not `403` — parental and
library restrictions do not confirm that a hidden item exists.

The `426` is what a pre-v3 or draft-v3 client gets. There is no protocol negotiation and no
fallback: the server decodes v3 or it refuses, and the client is expected to
render an "update required" state rather than retry. It is deliberately not a
`400` — the request may be perfectly well-formed for the protocol it was written
against, and the distinction is what lets a client tell "I sent something wrong"
apart from "I am too old to talk to this server".

Note the layering: a request whose *body* is fine but whose *media* cannot be
played is not an HTTP error. It is a `201` with `outcome:
"adaptation_unavailable"` and a terminal reason from §7.3. HTTP statuses on this
endpoint describe the request; the decision lives in the body.

**Idempotency.** The server stores each `playback_attempt_id` alongside a
SHA-256 digest of the exact request body. Replaying a byte-identical body
replays the original response verbatim. Reusing the id with a different body is
`409 playback_attempt_reused` — the id is a claim about *which* playback attempt
this is, so reusing it for different intent is a client bug the server refuses to
paper over. If the attempt is known but its session has since expired, the
response is a `201` terminal with reason `session_expired` rather than a replay
of a plan that no longer exists.

Playable and terminal decisions are both durable attempts. A terminal start has
no `session_id` or plan identity, but its response and ownership are retained
under `playback_attempt_id` for the same TTL. This makes terminal retries obey
the same replay/conflict rules and gives terminal route events an addressable
authorization record.

A client generates a fresh `playback_attempt_id` per user-initiated playback and
reuses it only to retry a request whose response it did not receive.

**Omission is a request, not a default.** Two start fields mean "you decide" when
absent, and the server answers from stored user state rather than from a
constant:

| Field | Omitted | Present |
| --- | --- | --- |
| `start_position` | The profile's saved resume point for this item, or `0` when there is none, it is already complete, or the file is one part of a multipart item (every part shares the item's resume point, so a part-local seek to it would land somewhere arbitrary). It is required when `progress_persistence` is `client` | Exactly that position. `0` means *start over* |
| `audio_track_id` / `audio_track_index` | The profile's preferred audio track, resolved from the series preference, then the profile's audio-language setting, then the library override | Exactly that track |

`progress_persistence` separates the live session clock from durable resume
ownership. Omission (or `server`) means session progress may update the item's
resume/history normally. `client` keeps heartbeats, route diagnostics, and live
session state intact but suppresses those durable writes because the client
persists its own item-global timeline (for example through `/sync/progress`). A
client choosing that mode must send `start_position` explicitly, including
explicit `0`; the server never substitutes saved resume state for it.

Both are settled *before* planning, not after. This is not an implementation
detail a client can ignore: the plan's timeline is cut at the start position
(§5), and the audio track is part of the plan's identity (§9), so a route chosen
for position zero and then seeked is a different route than the one the server
would have chosen for the resume point. A client that resolves resume state
itself and sends the position explicitly gets identical behaviour — that is the
supported way to override the server's policy.

### 2.3 `POST /playback/{session_id}/replan`

Asks for a different plan for an existing session — after a failure, or because
the user changed a track or the quality. Auth + `X-Profile-Id` required. Request
body cap: 256 KiB. Body: `ReplanRequestV3`
([`replan-request.schema.json`](../design/schemas/playback-v3/v3/replan-request.schema.json)).

| Status | Code | Meaning |
| --- | --- | --- |
| `200` | — | `DecisionResponseV3`, plan or terminal |
| `400` | `bad_request` | Malformed or failed validation |
| `401` | `unauthorized` | No authenticated user, or no `X-Profile-Id` |
| `403` | `forbidden` | The session belongs to another user or profile |
| `404` | `playback_session_not_found` | No such session, or its session has ended |
| `409` | `stale_playback_plan` | `failed_plan_id` is not the session's current plan, `playback_attempt_id` is not the session's attempt, or a newer replacement is already active |
| `409` | `idempotency_key_reused` | This `replan_request_id` was used for a different replan |
| `409` | `replan_in_progress` | A replan for this session holds the lease right now |
| `503` | `replan_capacity_exhausted` | Server-wide concurrent replan limit (8) reached; retryable |
| `500` | `internal_error` | Store outage |

Note that a missing profile is `401` here, where start answers `400` — start
validates the body first and reports the header as one more field problem, while
replan treats identity as a precondition. Neither is retryable, so the
difference does not change client behaviour.

Checks run in this order: auth → body decode and validation → concurrency slot →
session lock → attempt lookup → ownership → attempt match → live session → lease.
A client that sees `503` therefore knows nothing was read or written for its
session, and can retry the identical request unchanged. Note that validation
comes before every session lookup: a malformed body against a session that does
not exist answers `400`, not `404`.

**Leases.** A replan takes a 15-second lease on the session. A second request
carrying the *same* `replan_request_id` while the lease is in flight gets `409
replan_in_progress`; once the original completes, the same id replays its
response verbatim. This is what makes a client's retry-on-timeout safe.

Note the deliberate asymmetry with start: a store outage during replan is `500`,
never `404`. Clients tear playback down on session-not-found, so a transient
store failure must read as retryable rather than as the session having vanished.

### 2.4 `POST /playback/route-events`

Reports what happened on the device. Auth + `X-Profile-Id` required. Request body
cap: **32 KiB**. Body: a single `RouteEventV3`, not a batch
([`route-event.schema.json`](../design/schemas/playback-v3/v3/route-event.schema.json)).

| Status | Code | Meaning |
| --- | --- | --- |
| `202` | — | Accepted. **No response body.** |
| `400` | `bad_request` | Malformed or failed validation |
| `401` | `unauthorized` | No authenticated user or no profile |
| `403` | `forbidden` | The session or attempt belongs to another profile, or the referenced session/attempt does not exist |
| `429` | `event_rate_limited` | 120 events/attempt/minute or 600/user/minute exceeded |
| `500` | `internal_error` | Store outage |

The checks run in that order: auth, then body decode and validation, then the
rate limit, and only then the session-ownership lookup. The
limiter sits in front of the ownership lookup deliberately — it exists to bound
store reads as much as writes, so it has to precede the read that would establish
ownership.

Two consequences for clients. A `429` means "drop this event," never "retry it";
the events are diagnostics and losing one costs nothing. And an unknown session
is `403`, not `404` — the handler does not distinguish "not yours" from "not
there." A store outage during that lookup is `500`, so a `403` genuinely means
the event will never be accepted and should be dropped rather than retried.

A terminal decision returned by `POST /playback/start` has a durable attempt but
no playback session or plan. The client reports it with `event: "terminal"`,
the start request's `playback_attempt_id`, and no `session_id`, `plan_id`,
`plan_attempt_id`, or `plan_attempt_key`. The server authorizes the event through
the persisted attempt ownership and returns `202`.

Event names are the eleven in §7.4. `diagnostics` is a string→string map, capped
at 32 entries, and the server keeps only the keys on its allowlist (§7.5),
truncating each value to 256 characters. Unknown keys are dropped silently; a
client sending them is not an error, it just achieves nothing.

---

## 3. Capability evidence tiers

The hardest problem in this protocol is that clients lie — not maliciously, but
because platform APIs vary in how much they actually know. Android can enumerate
`MediaCodecList` and answer "this exact decoder supports H.264 High@4.1 at 8-bit
up to 1920×1080@60". A browser can only answer `isTypeSupported("video/mp4;
codecs=avc1.640028")` → true. Apple can attest that VideoToolbox handles a codec
family but not enumerate levels.

So a client declares *how it knows*, per media type, and the server applies a
different strictness to each tier. `video_evidence` and `audio_evidence` are
required and are one of:

| Tier | Who reports it | What the server does with it |
| --- | --- | --- |
| `exact` | Android (`MediaCodecList`) | Full strict validation. The server walks `video_decode[]` and requires a hardware entry matching codec, profile, level, bit depth, and every `max_*` bound. Only this tier can earn a validated audio **passthrough** claim. |
| `platform_attested` | Apple (VideoToolbox) | Same walk, but profile and level are **skipped** — the platform attests the codec family rather than enumerating modes. All other bounds still apply. |
| `declared` | Web (`isTypeSupported`) | Flat list match only: `codecs_video` / `codecs_video_hardware` membership. No `video_decode[]` walk. |

Four rules follow from the table and are easy to get wrong:

**A flat claim without backing detail is a refusal, not a pass.** On `exact` and
`platform_attested`, if a codec appears in `codecs_video` but no `video_decode[]`
entry names that codec with `hardware: true`, the source is *not* eligible for a
direct route. The plan is downgraded and carries the decision reason
`evidence_insufficient_for_direct` plus the matching degradation warning, which
distinguishes "your evidence didn't support this" from "your device said no." A
client advertising a strict tier must populate `video_decode[]`; the flat lists
alone earn it nothing. On `declared` the flat lists are the whole mechanism, so
that tier never produces this signal.

Note the precise trigger: the signal fires only when *no* entry named the codec.
If an entry matched the codec but the source exceeded one of its bounds — a
4K file against a `max_height: 1080` decoder — that is a real device limit, and
the plan is downgraded with no evidence warning. The two cases mean different
things and a client should not conflate them in its telemetry.

**An omitted bound means "unconstrained", not "unknown".** Within a
`video_decode[]` entry, an empty `profiles`, `levels`, or `bit_depths` list and a
zero `max_width` / `max_height` / `max_frame_rate` / `max_bitrate_kbps` are each
*skipped*, not failed. An entry that names a codec and nothing else therefore
claims that decoder handles every variant of it. That is a strong claim, and on
`exact` it is the client's job not to make it carelessly: the server will honour
it and hand back a direct route. Report what the platform actually enumerated.

The three list bounds are also not matched the same way, which matters when
populating them:

| Field | Match |
| --- | --- |
| `profiles` | Case-insensitive string equality against the source profile |
| `levels` | **At-least**: any listed level ≥ the source level passes |
| `bit_depths` | Exact integer equality |

So a decoder that tops out at H.264 level 4.1 may report `[41]` and still
validate a level-3.0 stream, while a decoder that handles 8- and 10-bit must
list both — `[10]` alone rejects an 8-bit source. Levels use the integer form
(4.1 → `41`).

**Every validated video route requires complete routing metadata.** Before any
tier logic runs, the server requires video codec, bit depth, width, height,
frame rate, and bitrate. Profile and level are decoder bounds instead: an
`exact` entry that supplies either bound cannot validate a source whose matching
probe value is absent or unknown, while an omitted bound keeps the explicit
“unconstrained” meaning above. This permits server adaptation of sources such as
VP9 whose probe reports an unknown level without allowing that sentinel to
satisfy a concrete client limit. A source missing the routing fields is
ineligible for any route and this case is *not* reported as
`evidence_insufficient_for_direct` — the client's evidence was never the
problem.

**Passthrough requires `exact` audio evidence.** A validated passthrough claim
(bitstreaming E-AC-3/TrueHD to a receiver) additionally requires the
`layout_aware_passthrough` feature in `client_features`, the codec listed in
`audio_passthrough.passthrough_codecs`, and a matching
`audio_passthrough.entries[]` whose `channel_counts` and `layouts` cover the
source. Only a client that can enumerate real sink layouts — the Android audio
HAL — can supply that. `platform_attested` and `declared` audio evidence still
qualify for ordinary decode/copy routes; they simply cannot earn
`claims.audio.passthrough = true`.

**HDR is decided against the output, not the decoder.** `output.hdr_details` (the
display or receiver actually attached) takes precedence over
`client_capabilities.hdr_details` (what the device could do in principle). A
source whose dynamic range is recorded as `hdr_unknown` — legacy rows that only
stored a file-level HDR boolean — is treated as HDR10 when the output supports
HDR10, and the plan carries the `hdr_range_assumed_hdr10` degradation warning.
Refusing to play those outright would be worse than an assumption the client is
told about.

---

## 4. Deliveries

A delivery is *how bytes reach the player*. The server works in four values; the
client negotiates in three classes.

| Server `delivery` | Client class | What it is |
| --- | --- | --- |
| `original_http` | `original_http` | The source file, byte-for-byte, over HTTP with range support |
| `server_remux_progressive` | `progressive` | Repackaged into a new container, streamed as one chunked response |
| `server_remux_hls` | `hls` | Repackaged into HLS segments; codecs untouched |
| `server_transcode_hls` | `hls` | Re-encoded and segmented |

`client_playback_context.deliveries` is keyed by **class**, because a client's
answer to "can you play HLS" does not differ between a remux and a transcode —
the same player component handles both. The server folds its four values into
three when checking eligibility, and reports the specific one it chose in the
plan.

Each `deliveries` entry describes one class:

| Field | Meaning |
| --- | --- |
| `enabled` | The client is *willing* to use this class right now (user setting, network policy) |
| `supported_on_device` | The client is *able* to — the platform has a player for it at all |
| `failure_reason` | Optional free text explaining a `false` above; diagnostics only |
| `containers`, `video_codecs`, `audio_decode_codecs` | Flat lowercase name lists |
| `audio_passthrough_codecs` | Bitstream-out candidates; only ever honoured under the `exact` tier (§3) |
| `max_channels` | Optional ceiling applied to audio routing |
| `hdr_details` | Optional per-class HDR support, overriding the device-level value |
| `subtitles` | Six booleans: `embedded_text`, `sidecar_text`, `ass_styling`, `embedded_bitmap`, `sidecar_bitmap`, `font_attachments` |
| `features` | Class-scoped feature strings |
| `auth_header_refresh` | The client can re-fetch stream auth headers without restarting playback |
| `validated_claims` | Claims the client asserts it has verified for this class |
| `transformations` | Client-executed transformations offered for this class (§11) |

Both booleans must be true for the class to be eligible; they are separate
because "the user turned HLS off" and "this device has no HLS player" call for
different degradation warnings and different diagnostics. A class the client
omits entirely is unavailable — the server will not guess.

`stream.header_refresh` tells the client what to do when the stream URL's auth
expires: `none` means the URL is stable for the session, `session` means
re-request headers from `header_refresh_url` rather than restarting playback.

---

## 5. The timeline model

The single most common client bug in v2 was assuming the player's zero and the
source's zero are the same instant. In v3 they usually are not, and the plan says
so explicitly.

`timeline` carries four numbers that a client must keep distinct:

- `source_start_seconds` — where in the *media* this plan begins.
- `stream_origin_seconds` — the source position that the *transport's* byte-zero
  corresponds to.
- `player_start_seconds` — where the client should seek the player after load.
- `timeline_offset_seconds` — what to add to a player position to get a source
  position.

Three shapes exist:

**Direct and transcoded routes** hand the player a complete timeline.
`stream_origin` and `timeline_offset` are `0`, `player_start` is the requested
position, `can_seek_anywhere` is true when the runtime is known, and
`seek_restoration` is `player_position` — the client seeks locally.

**Copy remux over HLS** is served from FFmpeg's live, still-growing playlist,
which starts at the requested position. So `player_start` is `0`,
`stream_origin` and `timeline_offset` both equal the seek position,
`seek_window_start_seconds` is that position, and **`seek_window_end_seconds` is
deliberately absent**. An open end marks the window as incomplete; combined with
`can_seek_anywhere: false` it routes every seek back through the server as a
reanchor (§6), which is correct because the playlist has no bytes for positions
FFmpeg has not reached yet. `seek_restoration` is `source_position`.

**Progressive remux** is a freshly generated chunked response with no byte-range
support, so it behaves the same way: player-zero is stream-zero is the seek
position, the window is open-ended, and seeks go through the server.

### `source.duration_seconds`

The media's full runtime, and nothing else. Specifically:

- It is **not** `total − source_start`. It does not shrink because the plan
  starts mid-file.
- It is **not** adjusted by `timeline_offset_seconds`.
- It is **omitted** rather than sent as `null` when the server does not know it,
  because a client coercing `null` to a numeric default would read it as zero.
- A client **must not** substitute the playback engine's reported duration for
  it. On an HLS copy remux the engine reports the length produced so far, not the
  runtime — using it makes the scrubber grow while the user watches.

---

## 6. Replan

Replan is the only way a plan changes. It covers both "that didn't work" and
"the user asked for something else," and the distinction matters to the server.

`operation` is one of five:

| Operation | Meaning | Requires |
| --- | --- | --- |
| `failure_recovery` | The plan failed on the device | `failure.classification` |
| `seek_failure_recovery` | A seek failed | `failure.classification` |
| `seek_reanchor` | Move a server-anchored timeline to a new position | — |
| `track_change` | The user picked a different audio or subtitle track | — |
| `quality_change` | The user picked a rung from `available_qualities` | non-empty `quality_preference` |

The asymmetry is intentional. A seek reanchor is a timeline operation, not a
failed recipe — a classification is still accepted from older callers but never
selects seek semantics. A track or quality change is not a failure at all, so
demanding a classification would force clients to invent one. But
`quality_change` *must* name the rung it wants: an empty `quality_preference`
normalizes to `auto`, which is a different user intent than the menu selection
the operation models, so the server rejects it rather than silently doing
something else.

**User-intent operations behave differently from failure recovery.**
`track_change` and `quality_change` replace what were separate v2 endpoints (an
audio PATCH and a client-posted transcode start). Nothing failed, so the previous
route stays eligible: neither the attempted-key history nor the failed-plan
exclusion applies. A client may therefore be handed back a plan it has already
tried — that is correct here, and a client must not treat a repeated
`plan_attempt_key` as a loop.

When such an operation actually changes something — the request's tracks or
quality differ from what the session currently has — the server also tries to
return to the *requested* edition rather than staying on whatever alternate
version a previous fallback landed on, since a user switching tracks may well
want the original file back. That is a preference, not a guarantee: if the
requested edition no longer resolves or fails its preflight, the healthy active
alternate is kept. Track identities are remapped only when the edition really
changes, because remapping within one file would degrade an exact selection to a
best-match lookup and could silently move a listener off a commentary track.

Omitting `quality_preference` on any replan preserves the session's current
preference; sending it replaces that preference. A track change therefore does
not silently reset `original` or a pinned rung, and failure recovery does not
discard the viewer's requested quality unless the client explicitly asks it to.
Clients should still send the current preference when they know it, so their
intent remains explicit in diagnostics.

For failure recovery, `attempted_plan_keys` is the loop guard. The client sends
back every `plan_attempt_key` it has already tried for this attempt (up to 16);
the server will not hand back a plan whose key is in that list. `attempt_count`
(1–8) bounds the whole recovery chain. Together they mean a device that fails
every route reaches a terminal instead of cycling forever.

Failure, seek, and quality replans may omit unchanged track identities. The
server overlays only identities present in those requests and preserves the
durable selected subtitle otherwise. Only `operation: "track_change"` gives an
omitted `selected_tracks.subtitle` the explicit meaning "subtitles off". A
fallback to another media version must remap the selected subtitle; if no
equivalent exists, it returns terminal reason `subtitle_unavailable_in_version`
instead of silently continuing with subtitles off.

`local_mutations` (up to 8 entries, 64 chars each) reports client-side
adjustments — a transport reopen, a PCM decode fallback — that change the
effective route without changing the plan. They feed the attempt key, so a plan
retried after a local mutation is a *different* attempt and is not blocked by the
loop guard.

A seek-scoped recovery refuses to accept new capability or device evidence: a
seek is not an authority boundary for replacing the client's declared abilities
mid-session.

---

## 7. Registries

### 7.1 Decision reasons

Why the server picked this route. Informational; clients may log or display but
must not branch on an unrecognized value.

`validated_original_playback`, `container_normalization`, `audio_adaptation`,
`hls_audio_adaptation`, `hls_packaging_required`, `subtitle_burn_in_required`,
`client_dv7_to_dv81`, `client_dv7_to_hdr10`, `evidence_insufficient_for_direct`,
and the quality reasons `quality_original`, `quality_auto_source`,
`quality_fixed_rung`, `quality_device_limit`, `quality_bandwidth_limit`,
`quality_metered_limit`, `quality_bandwidth_cap`.

### 7.2 Degradation warnings

The plan will play, but something the user might notice was given up.

| Code | Meaning |
| --- | --- |
| `hdr_range_assumed_hdr10` | Source range unknown; treated as HDR10 |
| `dolby_vision_removed` | DV metadata stripped |
| `dolby_vision_strip_unsupported_by_source` | DV could not be stripped |
| `dolby_vision_enhancement_layer_discarded` | FEL/MEL dropped, base layer kept |
| `audio_converted` | Audio re-encoded rather than copied |
| `subtitle_burn_in` | Subtitles rendered into the video |
| `quality_reduction_unavailable` | Requested rung could not be produced |
| `quality_preference_normalized` | Unknown `quality_preference` normalized to `auto` |
| `bandwidth_cap_applied` | `bandwidth_cap_kbps` limited the selection |
| `evidence_insufficient_for_direct` | Evidence tier blocked a direct route |

### 7.3 Terminal reasons

Playback will not proceed. `terminal.retryable` says whether trying again could
help. Delivered inside a `201` (start) or `200` (replan), never a 4xx.

*Planner:* `adaptation_exhausted`, `adaptation_unavailable`,
`client_hls_unsupported`, `conversion_tool_unavailable`,
`hdr_transcode_unsupported`, `no_alternate_version`,
`source_metadata_incomplete`, `source_unavailable`,
`audio_conversion_unsupported`, `video_conversion_unsupported`,
`dv_conversion_unsupported`, `transcoding_disabled`,
`subtitle_conversion_unsupported`.

*Subtitle policy:* `subtitle_burn_in_source_unsupported`,
`subtitle_codec_unsupported`, `subtitle_track_invalid`,
`subtitle_track_unavailable`, `subtitle_unavailable_in_version`.

*Transport and session:* `internal_error`, `session_expired`,
`subtitle_artifact_unavailable`, `capacity_unavailable`,
`audio_transcoding_disabled`,
`transcode_start_failed`, `transcode_node_unavailable`,
`transcode_node_capability_unavailable`, `track_unavailable`,
`invalid_seek_position`, `invalid_replan`, `seek_reanchor_route_changed`,
`seek_reanchor_recipe_unavailable`,
`seek_reanchor_intent_mismatch`, `seek_failure_recovery_intent_mismatch`,
`policy_denied`.

### 7.4 Route event names

`plan_selected`, `plan_invalidated`, `plan_failed`, `first_frame`, `terminal`,
`stopped`, `runtime_correction_applied`, `runtime_correction_succeeded`,
`runtime_correction_failed`, `seek_reanchor_requested`, `seek_reanchored`.

### 7.5 Diagnostics allowlist

Route-event `diagnostics` keys the server retains. Everything else is dropped;
every value is truncated to 256 characters.

`decoder_name`, `decoder_init_ms`, `first_frame_ms`, `device_model`,
`requested_quality`, `effective_quality`, `pcm_recovery`, `retry_outcome`,
`replan_request_id`, `video_mime`, `video_codecs`, `video_width`, `video_height`,
`color_transfer`, `color_range`, `error_code`, `error_code_name`, `error_cause`,
`transformation_name`, `transformation_version`, `transformation_stage`,
`input_dv_profile`, `output_dv_profile`, `rpu_converted_count`,
`rpu_failed_count`, `el_nal_dropped_count`, `sample_count`,
`transform_buffer_peak_bytes`, `requested_media_file_id`,
`effective_media_file_id`, `audio_output_mode`, `audio_mime`, `audio_channels`,
`audio_decoder_name`, `correction_id`, `correction_stage`, `network_transport`,
`network_metered`, `network_validated`, `bandwidth_estimate_kbps`,
`link_downstream_kbps`, `target_source_position_seconds`, `reason`.

---

## 8. Track identity and the subtitle ordinal space

Every track is addressed as `file:{media_file_id}:{kind}:{ordinal}` — for
example `file:42:audio:0`. When a client sends both an id and an index they must
agree; the server rejects a disagreeing pair rather than picking one.

Subtitles occupy a single **combined ordinal space** spanning three sources, in
three dense consecutive ranges:

1. **External** sidecar files, in catalog order — ordinals `0 … E-1`
2. **Embedded** container streams, in container stream order — `E … E+M-1`
3. **Downloaded** subtitles, in `created_at` order — `E+M … E+M+D-1`

The space is dense and gap-free, and this is load-bearing. A track that has no
sidecar representation the stream handler can serve — a DVD or DVB bitmap stream
— **keeps its ordinal** and is published with `delivery: "burn_in_only"` and no
`url`. Omitting it would leave a hole, and any client deriving the
downloaded-track base by counting published URLs would then undercount and
address the wrong track. (That was a real bug; this rule is the fix.)

`playback_plan.subtitle.inventory` is the authoritative list. A client selects a
track by echoing an entry's `track_id` or `combined_index`. It must never derive
an ordinal by counting tracks, summing array lengths, or taking `max(index)+1`.

Each entry carries `source` (`external` | `embedded` | `downloaded`), `delivery`
(`sidecar` | `burn_in_only`), the `forced` / `default` / `hearing_impaired`
flags, a `url` when deliverable, and a `font_bundle_url` for embedded ASS tracks
with attachments. `default` reflects the source container's own default flag, so
only embedded and external tracks can carry it — a downloaded subtitle is never
`default`. `url` is present only on `sidecar` tracks, and only once a session
exists to scope it to — but it does not depend on the current selection: a start
or replan that resolves to `subtitle.mode: "off"` still publishes every sidecar
entry with its fetchable `url`, so a client can build its full subtitle menu
without first asking for a plan it does not want.

`subtitle.mode` is `off`, `render` (client draws the sidecar), `convert` (server
transcodes it to a client-renderable format first — always to WebVTT, served as
`text/vtt` at a `.vtt` URL), or `burn_in` (rendered into the video, which forces
a transcode).

The sidecar URL suffix is part of the representation contract, not decoration.
An embedded `hdmv_pgs_subtitle`/PGS sidecar is lossless binary PGS at a `.sup`
URL with `application/octet-stream`; cached full-track responses support `HEAD`
and byte ranges. Text conversion is always WebVTT at `.vtt`, while lossless
ASS/SSA uses `.ass`. A suffix that does not match the selected track or a valid
conversion is rejected with `415` rather than returning bytes of a different
type under the requested extension.

---

## 9. Plan identity

The server mints both identifiers. **Clients treat both as opaque, case-sensitive
tokens and never implement either identity algorithm.** Their wire prefixes and
lengths are validation syntax, not a derivation recipe.

`plan_id` identifies the server's playback decision. It is stable when the same
attempt produces the same source, delivery, recipe, tracks, subtitle mode,
transformations, applied quirks, and recipe revision. A change to any of those
inputs produces a different identity.

`plan_attempt_key` is the replan loop guard for a plan as attempted on one output
route with a set of client-reported local mutations. The server canonicalizes
order-insensitive inputs internally. The client:

1. stores the exact key from `playback_plan.plan_attempt_key`;
2. echoes it unchanged as `plan_attempt_key` when reporting that plan;
3. adds the unchanged token to `attempted_plan_keys` after the plan fails; and
4. never case-folds, parses, truncates, hashes, or synthesizes a replacement.

`internal/playback/testdata/protocol_v3/attempt_keys.json` contains opaque
cross-message vectors: a server-emitted token, the exact replan echo, and the
loop-rejection result. The generator computes the server token internally but
does not publish its preimage.

---

## 10. Quality

`playback_plan.available_qualities` is the menu. The client renders it and, on
selection, sends a `quality_change` replan with the entry's `label`. It does not
compute rungs.

The source rung is always present, labelled `original`, with
`preserves_source: true`. Transcode rungs are added only below the source's own
height, and only when HLS is available to the client, transcoding is enabled,
4K transcoding is permitted for a 4K source, and the source is not HDR. Ladder
bitrates:

| Rung | kbps |
| --- | --- |
| 2160p | 20000 |
| 1080p | 6000 |
| 720p | 2000 |
| 480p | 1500 |

Registry availability is deliberately *not* consulted when building the menu: a
capability check there could trigger lazy node fetches that a source-preserving
start must never pay for. A rung whose toolchain turns out to be missing degrades
to a retryable terminal at replan time instead.

Audio-only sources publish a single `original` rung — quality rungs are a video
concept.

`quality_preference` accepts `auto`, `original` (aliases `source`, `max`), and
`2160p` / `1080p` / `720p` / `480p` with the obvious aliases (`4k`, `uhd`, `fhd`,
`hd`, `sd`). An unrecognized value normalizes to `auto` and the response carries
the `quality_preference_normalized` warning rather than an error.

---

## 11. Transformations

A transformation is a named, versioned media operation with claims attached.

| Name | Executor | Recipe version | Promises | Claims |
| --- | --- | --- | --- | --- |
| `audio_to_aac` | `server` | `1` | — | `audio_decode` |
| `video_to_h264` | `server` | `2` | `sdr` output | `h264_decode` |
| `server_dv7_to_hdr10` | `server` | `1` | `hdr10` output | `dolby_vision_metadata_removed`, `hdr10_base_layer_preserved`, `enhancement_layer_discarded` |

They are advertised only if the installed FFmpeg actually has the required
capability, probed once at startup:

| Transformation | Probe |
| --- | --- |
| `server_dv7_to_hdr10` | `ffmpeg -bsfs` contains `dovi_rpu` |
| `audio_to_aac` | `ffmpeg -encoders` contains an `aac` encoder |
| `video_to_h264` | `ffmpeg -encoders` contains any of `libx264`, `h264_qsv`, `h264_vaapi`, `h264_nvenc`, `h264_videotoolbox` |

`GET /playback/capability` reports the *local* probe only. A deployment with
pooled transcode nodes may still plan an HLS route using a transformation those
nodes advertise but the local FFmpeg lacks, so the capability list is a floor,
not a ceiling — one more reason a client must not precompute routes from it.

An unavailable transformation is not silently skipped at plan time: it produces
its own terminal reason (`dv_conversion_unsupported`,
`audio_conversion_unsupported`, `video_conversion_unsupported`) so the client
learns which conversion was missing rather than seeing a generic refusal.

A client may advertise its *own* transformations in a delivery's
`transformations[]` with `executor: "client"` — Dolby Vision profile 7 → 8.1
conversion, for instance. The server accepts a client executor only when that
delivery is both `enabled` and `supported_on_device` **and** the request's
top-level `client_features` includes `client_video_transformations_v1`.
Duplicate `executor:name:recipe_version` triples are rejected. Client
transformations participate in plan identity exactly like server ones, so a
client that changes its transform version invalidates its prior attempt keys —
which is the intent.

---

## 12. Conformance

Three artifacts, in decreasing order of authority:

1. **`internal/playback/testdata/protocol_v3/`** — golden fixtures generated by
   `cmd/playbackfixtures` from the live server types. `make playback-fixtures`
   regenerates them; `make verify-playback-fixtures` fails CI if they are stale.
   Android and Apple CI vendor these and compare against them as **opaque
   expected output**. The direction of authority is inverted from where this
   protocol started: the server defines the contract and clients prove
   conformance, not the reverse. `conformance_matrix.json` covers the release
   train's evidence tiers, delivery fallback chain, replan operations and
   idempotency, quality ladder, audio-only route, HDR/Dolby Vision decisions,
   audio adaptation and exact-layout passthrough, text/bitmap subtitle policy,
   failure recovery, restart replay, capacity cleanup, output change, route
   event limits, opaque loop guard, and legacy-upgrade response in one
   generated cross-client corpus.
2. **`docs/design/schemas/playback-v3/`** — JSON Schemas for every wire body,
   with valid and invalid fixtures. Every bound mirrors a server validator, so a
   body these schemas reject is a body the server rejects.
   `internal/playback/contract` enforces that the schemas, the Go types, and the
   golden fixtures agree.
3. **This document** — the reasoning behind the above, and the normative source
   for anything the schemas cannot express (idempotency semantics, the timeline
   model, evidence-tier strictness, ordinal density).

A client that decodes the golden fixtures, echoes attempt keys byte-for-byte,
and round-trips a replan without computing an identity is conforming.