docs(prds): leave apply-status liveness open instead of deciding it
RULES.md says not to turn a requested outcome into extra product features and to keep inferences open until the maintainer or repository evidence settles them. The first draft broke that: a reviewer found that a record replaced in place lets a process that applied a version and then died keep reading as current, and I answered it by adding a periodic re-stamp, a recurring write in three processes that neither the tracking issue nor runtime-control-plane.md asks for. The hazard is real, so it stays in the document as an open question with three options and a recommendation, for Sam to settle. The re-stamp and the instance identity come out of the design. Also the sentence pass RULES.md requires before approval, which had not been done: the staleness rule was stated in three sections, and rationale stood where a requirement belongs. Relates to archie-core-pskb.
This commit is contained in:
1 parent
2bfb75763d
commit
285474a552
1 file changed
+52
-53
@@ -7,51 +7,39 @@
|
||||
## Outcome
|
||||
|
||||
An operator who edits a control-plane resource can see which processes are
|
||||
running that version, which are still on an older one, which rejected it, and
|
||||
which have stopped saying anything at all.
|
||||
running that version, which are still on an older one, and which rejected it.
|
||||
|
||||
`docs/prds/runtime-control-plane.md` requires the first three ("Each process
|
||||
reports the version it is using or why it failed. The UI shows both") and does
|
||||
not say where the report goes. This decides that, and adds the fourth, because
|
||||
without it a process that dies after a successful report keeps asserting
|
||||
health.
|
||||
`docs/prds/runtime-control-plane.md` requires this and does not say where the
|
||||
report goes. This decides that.
|
||||
|
||||
## Where the report goes
|
||||
|
||||
Apply status is a State Store contract surface. `ControlPlaneService` is the
|
||||
surface for editing a resource, and a status report is not an edit. The
|
||||
matching precedent is the config snapshot pair: a process publishes what it
|
||||
knows, the UI process reads it, and both RPCs are administrative, so a
|
||||
task-scoped grant reaches neither. That denial needs no new rule;
|
||||
`authorizesTaskScopedCall` names three methods and denies everything else.
|
||||
precedent is the config snapshot pair: a process publishes what it knows, the
|
||||
UI process reads it, and both RPCs are administrative, so a task-scoped grant
|
||||
reaches neither. That denial needs no new rule: `authorizesTaskScopedCall`
|
||||
names three methods and denies everything else.
|
||||
|
||||
`docs/prds/state-store-contract.md` gains the pair in its inventory, and its
|
||||
RPC count at two places moves from 45 to 47.
|
||||
|
||||
The read returns every record as a list, including an empty one. It is not a
|
||||
singleton read, so the `found=false` convention for absent single rows does not
|
||||
apply to it.
|
||||
The read returns all records as a list, including an empty one. The
|
||||
`found=false` convention for absent single rows does not apply.
|
||||
|
||||
## What is reported
|
||||
|
||||
One record per process instance and resource kind, replaced in place. It holds:
|
||||
|
||||
- the process name and the instance identity it generated at startup;
|
||||
- the resource kind;
|
||||
- the version the process applied, and the error if it could not;
|
||||
- the time it last reported.
|
||||
One record per process and resource kind, replaced in place. It holds the
|
||||
process name, the resource kind, the version applied, the error if the process
|
||||
could not apply it, and the time it reported.
|
||||
|
||||
A process reports at the point it applies a resource, so the record says what
|
||||
happened rather than what was intended. Nothing probes.
|
||||
|
||||
Each reporting process re-stamps its records on a fixed interval. The report
|
||||
time is therefore a liveness signal, not just a write timestamp: a record that
|
||||
has stopped being re-stamped belongs to a process that has stopped running.
|
||||
|
||||
Process names come from one shared constant naming `archied`, `archie-gateway`
|
||||
and `archie-messaging`. A deployment runs at most one instance of each. A
|
||||
producer that invents its own name would be indistinguishable from a process
|
||||
that is down, so writer and reader are bound to that constant by test.
|
||||
and `archie-messaging`. A deployment runs at most one of each. Writer and
|
||||
reader are bound to that constant by test, because a drifted name is
|
||||
indistinguishable from a process that never reported.
|
||||
|
||||
## Which processes report
|
||||
|
||||
@@ -64,49 +52,60 @@ that is down, so writer and reader are bound to that constant by test.
|
||||
## What can be reported, and what cannot
|
||||
|
||||
A process reaches its apply point only after the State Store has answered it.
|
||||
When the State Store is unreachable, the process exits or refuses to start, and
|
||||
writes no record at all. An unreachable store is therefore never reported as a
|
||||
failure by the process it affected. What a record can carry is the other
|
||||
failure: the store answered, and the value it returned would not validate.
|
||||
When the State Store is unreachable, the process exits or refuses to start and
|
||||
writes no record. An unreachable store is never reported as a failure by the
|
||||
process it affected. A record carries the other failure: the store answered,
|
||||
and the value would not validate.
|
||||
|
||||
The UI distinguishes three absences, and never collapses them:
|
||||
|
||||
- no process applies this kind, so there is no record and none is expected;
|
||||
- a process has records for other kinds but none for this one;
|
||||
- a process has a record that has gone stale, meaning it is no longer running.
|
||||
Absence has two meanings and the UI never collapses them: no process applies
|
||||
this kind, or a process that applies other kinds has not reported this one.
|
||||
|
||||
## Relationship to the reload status
|
||||
|
||||
Apply status covers control-plane resource kinds. `ReloadStatus` covers the
|
||||
file document, and stays the only report for a SIGHUP that failed to resolve or
|
||||
validate the file. The two are not merged and neither is derived from the
|
||||
other.
|
||||
file document and stays the only report for a SIGHUP that failed to resolve or
|
||||
validate it. Neither is derived from the other.
|
||||
|
||||
Writing an apply-status record is best-effort everywhere, and on the SIGHUP
|
||||
path specifically it gets its own bounded context, separate from the budget the
|
||||
reload spends reading the control plane. A failed report is logged and never
|
||||
turned into a failed reload: the configuration it describes has already been
|
||||
applied, and reporting it badly must not raise a banner saying the reload
|
||||
failed.
|
||||
Writing a record is best-effort. On the SIGHUP path it takes its own bounded
|
||||
context, separate from the budget the reload spends reading the control plane,
|
||||
and a failed write is logged rather than turned into a failed reload.
|
||||
|
||||
## What the UI shows
|
||||
|
||||
The settings page shows, per resource, the stored version beside each process's
|
||||
applied version, error, and how recently it reported. A process on an older
|
||||
version of a restart-required resource is pending a restart. A process with an
|
||||
error is failed, with the error beside it. A process whose record has gone
|
||||
stale is unknown, never current.
|
||||
applied version, error, and report time. A process on an older version of a
|
||||
restart-required resource is pending a restart. A process with an error is
|
||||
failed, with the error beside it.
|
||||
|
||||
## Open question: a record outlives the process that wrote it
|
||||
|
||||
Records are replaced in place, so a process that applies a version and then
|
||||
stops leaves a record asserting that version indefinitely. The settings page
|
||||
would show a dead process as current. `docs/prds/status-health-surface.md`
|
||||
faced the same choice and stamped the start of a poll pass so a wedged poller
|
||||
reads as stale rather than healthy.
|
||||
|
||||
This is not settled here, because the fix is a mechanism the tracking issue
|
||||
does not ask for. Three options:
|
||||
|
||||
1. Each process re-stamps its records on an interval, and a record that stops
|
||||
being re-stamped reads as unknown. Costs a recurring write per process.
|
||||
2. Records carry the instance identity generated at process start, and the UI
|
||||
reads a changed identity as a restart. Detects restarts, not deaths.
|
||||
3. Accept it. The page answers "what did each process apply" and not "is it
|
||||
still running", and says so.
|
||||
|
||||
Recommended: 1, because the question an operator brings to this page is whether
|
||||
their change is live now.
|
||||
|
||||
## Verification
|
||||
|
||||
- A process that applies a resource writes a record carrying that version and
|
||||
its instance identity.
|
||||
- A process that applies a resource writes a record carrying that version.
|
||||
- A process whose apply fails validation writes a record carrying the error and
|
||||
no new version.
|
||||
- A reload that fails leaves the previously applied version recorded.
|
||||
- A record that stops being re-stamped renders as unknown rather than current.
|
||||
- A record written under a process name outside the shared constant fails the
|
||||
test that binds writer to reader.
|
||||
test binding writer to reader.
|
||||
- A task-scoped credential is refused by both RPCs.
|
||||
- A failed report does not fail the reload that produced it.
|
||||
|
||||
|
||||
Reference in new issue
Block a user