feat(runs): the investigate prompt names the labels and the figure (archie #2005)

An alert about a filesystem carries instance and mountpoint, and those are the
labels the forecast query needs, so the prompt names both and gives the query
that answers what an operator asks: the seconds until the disk is full, from
predict_linear and deriv over the same window. A negative figure is the
filesystem not filling, which the prompt says to report as that rather than as
a time.
This commit is contained in:
sam committed 2026-11-16 00:13:13 +11:00
1 parent deb3865d75
commit a3d32e92e5
1 file changed
+11 -6
+11 -6
View File
@@ -13,12 +13,17 @@ The event is a claim, not a fact. Check it against what you measure, and if
they disagree (an alert says a disk is full and it is at 40%), say so first:
that disagreement is a finding in itself.
A trend is a query, not an estimate. When something is filling up, get the
time until it's full from the metrics server rather than working it out from
raw points: Prometheus answers `predict_linear` through `query`, so with node
exporter's filesystem metrics a figure comes from a query such as
`predict_linear(node_filesystem_avail_bytes{instance="host:9100",mountpoint="/"}[6h], 24*3600) < 0`.
Report that figure and the query it came from.
A trend is a query, not an estimate. An event about a filesystem carries
`instance` and `mountpoint`, and those labels go into the metric's own
selector: ask the metrics server for the forecast rather than working one out
from raw points. Prometheus answers `predict_linear` through `query`, and with
node exporter's filesystem metrics the seconds until the disk is full are
-predict_linear(node_filesystem_avail_bytes{instance="<instance>",mountpoint="<mountpoint>"}[6h], 0)
/ deriv(node_filesystem_avail_bytes{instance="<instance>",mountpoint="<mountpoint>"}[6h])
Report the figure that query returns and how long it is. A negative figure
means the filesystem isn't filling: say that instead of giving a time.
When you're done, reply with your findings in plain text: