docs(prds): leave apply-status liveness open instead of deciding it

RULES.md says not to turn a requested outcome into extra product features and
to keep inferences open until the maintainer or repository evidence settles
them. The first draft broke that: a reviewer found that a record replaced in
place lets a process that applied a version and then died keep reading as
current, and I answered it by adding a periodic re-stamp, a recurring write in
three processes that neither the tracking issue nor runtime-control-plane.md
asks for.

The hazard is real, so it stays in the document as an open question with three
options and a recommendation, for Sam to settle. The re-stamp and the instance
identity come out of the design.

Also the sentence pass RULES.md requires before approval, which had not been
done: the staleness rule was stated in three sections, and rationale stood
where a requirement belongs.

Relates to archie-core-pskb.
This commit is contained in:
sam committed 2026-09-21 15:33:14 +10:00
1 parent 2bfb75763d
commit 285474a552
1 file changed
+52 -53
+52 -53
View File
@@ -7,51 +7,39 @@
## Outcome
An operator who edits a control-plane resource can see which processes are
running that version, which are still on an older one, which rejected it, and
which have stopped saying anything at all.
running that version, which are still on an older one, and which rejected it.
`docs/prds/runtime-control-plane.md` requires the first three ("Each process
reports the version it is using or why it failed. The UI shows both") and does
not say where the report goes. This decides that, and adds the fourth, because
without it a process that dies after a successful report keeps asserting
health.
`docs/prds/runtime-control-plane.md` requires this and does not say where the
report goes. This decides that.
## Where the report goes
Apply status is a State Store contract surface. `ControlPlaneService` is the
surface for editing a resource, and a status report is not an edit. The
matching precedent is the config snapshot pair: a process publishes what it
knows, the UI process reads it, and both RPCs are administrative, so a
task-scoped grant reaches neither. That denial needs no new rule;
`authorizesTaskScopedCall` names three methods and denies everything else.
precedent is the config snapshot pair: a process publishes what it knows, the
UI process reads it, and both RPCs are administrative, so a task-scoped grant
reaches neither. That denial needs no new rule: `authorizesTaskScopedCall`
names three methods and denies everything else.
`docs/prds/state-store-contract.md` gains the pair in its inventory, and its
RPC count at two places moves from 45 to 47.
The read returns every record as a list, including an empty one. It is not a
singleton read, so the `found=false` convention for absent single rows does not
apply to it.
The read returns all records as a list, including an empty one. The
`found=false` convention for absent single rows does not apply.
## What is reported
One record per process instance and resource kind, replaced in place. It holds:
- the process name and the instance identity it generated at startup;
- the resource kind;
- the version the process applied, and the error if it could not;
- the time it last reported.
One record per process and resource kind, replaced in place. It holds the
process name, the resource kind, the version applied, the error if the process
could not apply it, and the time it reported.
A process reports at the point it applies a resource, so the record says what
happened rather than what was intended. Nothing probes.
Each reporting process re-stamps its records on a fixed interval. The report
time is therefore a liveness signal, not just a write timestamp: a record that
has stopped being re-stamped belongs to a process that has stopped running.
Process names come from one shared constant naming `archied`, `archie-gateway`
and `archie-messaging`. A deployment runs at most one instance of each. A
producer that invents its own name would be indistinguishable from a process
that is down, so writer and reader are bound to that constant by test.
and `archie-messaging`. A deployment runs at most one of each. Writer and
reader are bound to that constant by test, because a drifted name is
indistinguishable from a process that never reported.
## Which processes report
@@ -64,49 +52,60 @@ that is down, so writer and reader are bound to that constant by test.
## What can be reported, and what cannot
A process reaches its apply point only after the State Store has answered it.
When the State Store is unreachable, the process exits or refuses to start, and
writes no record at all. An unreachable store is therefore never reported as a
failure by the process it affected. What a record can carry is the other
failure: the store answered, and the value it returned would not validate.
When the State Store is unreachable, the process exits or refuses to start and
writes no record. An unreachable store is never reported as a failure by the
process it affected. A record carries the other failure: the store answered,
and the value would not validate.
The UI distinguishes three absences, and never collapses them:
- no process applies this kind, so there is no record and none is expected;
- a process has records for other kinds but none for this one;
- a process has a record that has gone stale, meaning it is no longer running.
Absence has two meanings and the UI never collapses them: no process applies
this kind, or a process that applies other kinds has not reported this one.
## Relationship to the reload status
Apply status covers control-plane resource kinds. `ReloadStatus` covers the
file document, and stays the only report for a SIGHUP that failed to resolve or
validate the file. The two are not merged and neither is derived from the
other.
file document and stays the only report for a SIGHUP that failed to resolve or
validate it. Neither is derived from the other.
Writing an apply-status record is best-effort everywhere, and on the SIGHUP
path specifically it gets its own bounded context, separate from the budget the
reload spends reading the control plane. A failed report is logged and never
turned into a failed reload: the configuration it describes has already been
applied, and reporting it badly must not raise a banner saying the reload
failed.
Writing a record is best-effort. On the SIGHUP path it takes its own bounded
context, separate from the budget the reload spends reading the control plane,
and a failed write is logged rather than turned into a failed reload.
## What the UI shows
The settings page shows, per resource, the stored version beside each process's
applied version, error, and how recently it reported. A process on an older
version of a restart-required resource is pending a restart. A process with an
error is failed, with the error beside it. A process whose record has gone
stale is unknown, never current.
applied version, error, and report time. A process on an older version of a
restart-required resource is pending a restart. A process with an error is
failed, with the error beside it.
## Open question: a record outlives the process that wrote it
Records are replaced in place, so a process that applies a version and then
stops leaves a record asserting that version indefinitely. The settings page
would show a dead process as current. `docs/prds/status-health-surface.md`
faced the same choice and stamped the start of a poll pass so a wedged poller
reads as stale rather than healthy.
This is not settled here, because the fix is a mechanism the tracking issue
does not ask for. Three options:
1. Each process re-stamps its records on an interval, and a record that stops
being re-stamped reads as unknown. Costs a recurring write per process.
2. Records carry the instance identity generated at process start, and the UI
reads a changed identity as a restart. Detects restarts, not deaths.
3. Accept it. The page answers "what did each process apply" and not "is it
still running", and says so.
Recommended: 1, because the question an operator brings to this page is whether
their change is live now.
## Verification
- A process that applies a resource writes a record carrying that version and
its instance identity.
- A process that applies a resource writes a record carrying that version.
- A process whose apply fails validation writes a record carrying the error and
no new version.
- A reload that fails leaves the previously applied version recorded.
- A record that stops being re-stamped renders as unknown rather than current.
- A record written under a process name outside the shared constant fails the
test that binds writer to reader.
test binding writer to reader.
- A task-scoped credential is refused by both RPCs.
- A failed report does not fail the reload that produced it.