Files
archie/internal/logging/reader.go
T
sam 2ef962912a feat(tasks): attribute events, changes and config to the attempt that produced them
The events table had no attempt column, so two attempts of the same task were
indistinguishable: a retried task's stage events interleaved into one timeline and
nothing could say which run produced a given result.

- events gains an `attempt` column through the existing presence-gated
  migrateTasks loop rather than a version bump, so user_version stays 3 and an
  existing archie.db is migrated in place. A legacy-database test builds the
  pre-change schema and proves both that existing rows survive unattributed
  (attempt = 0) and that a new value round-trips.
- Event.attempt = 11 and ReadTaskLogRequest.stage = 9 are field additions to
  existing messages: no new RPC, no new table, and no change to any canonical error
  message that crosses the state-store wire.
- Every task event the workflow engine emits now carries the attempt. Producers
  that have no attempt yet (task_queued, binding_dispatch_failure) stay
  unattributed rather than guessing one.
- Changed files are captured as a durable changes_captured event at commit and push
  time, reusing the existing three-dot diff primitive: paths from FilePatches(),
  counts from Stats(), base_sha recorded as the merge base actually diffed rather
  than the base tip, a 200-file cap with totals kept complete. Consumption goes
  through an unexported optional interface so workflow.Trees -- the yaegi capability
  surface for interpreted stage plugins -- is not widened. A capture is also taken
  once the pull request exists, so the capture can link it.
- The per-attempt configuration is captured as a durable config_captured event
  carrying the non-secret config.TaskConfig, instead of a new table plus two new
  state-store RPCs: events are already this system's per-attempt provenance stream,
  so a table would have given one feature two provenance mechanisms. Redaction is
  proven against canary secrets. Measured payload: 539 bytes typical, 2333 bytes at
  20 models.
- Captures carry a schema marker and are refused unless their key set matches the
  declared payload shape, so drift surfaces as "recorded but could not be read"
  rather than as a capture of zeroes.
- Three read endpoints serve the attempt rail, the changed-file captures and a raw
  debug view, mapping to UI-owned response types rather than dumping store types.
- The task-log read gains a stage filter, bound to the stage logger for that
  stage's own lines; agent and tool output records no stage.

A task-scoped grant may already insert events on its own task, so this provenance
is exactly as trustworthy as the rest of that task's event stream. That limit is
documented rather than papered over.
2026-09-18 18:32:59 +10:00

620 lines
23 KiB
Go

package logging
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"slices"
"strings"
"time"
)
// Entry is one decoded log line.
//
// Reading lives here rather than in the dashboard because this package owns
// the on-disk format. If the handler shape or field names change, the reader
// changes with them in the same file, and no consumer has to know slog wrote
// JSON with a "msg" key.
type Entry struct {
ID int64 `json:"id,omitempty"`
Time time.Time `json:"time"`
Level string `json:"level"`
Message string `json:"msg"`
// Fields carries every remaining key so structured context survives the
// round trip -- component, task id, error and anything else a call site
// attached.
Fields map[string]any `json:"fields,omitempty"`
}
// Query filters a read. The zero value returns the most recent DefaultTailLines
// entries unfiltered.
type Query struct {
// Levels restricts results to these levels (case-insensitive). Empty
// means all levels.
Levels []string
// Component matches the "component" field exactly. Empty means any.
Component string
// Stage matches the entry's own "stage" field exactly (case-insensitive).
// Empty means any.
//
// Only entries a stage tagged carry that field. Agent and tool output is
// logged by the runtime that produced it and carries no stage, so a stage
// filter NARROWS the file rather than covering it: the caller is obliged to
// say that, because an honest-looking filter over a partial field is worse
// than no filter. An entry with no stage never matches.
Stage string
// Contains matches the message or any field value, case-insensitively.
Contains string
// Limit caps returned entries. Zero selects DefaultTailLines; values
// above MaxTailLines are clamped.
Limit int
// Since, when set, excludes entries whose Time is strictly before it.
// Until, when set, excludes entries whose Time is strictly after it.
// Both bounds are inclusive. The zero time disables its bound.
Since time.Time
Until time.Time
}
// Read bounds. A log file is unbounded input, so a request must not be able to
// pull an arbitrary amount of it into memory.
const (
DefaultTailLines = 200
MaxTailLines = 2000
// maxScanBytes caps how much of the tail is examined. Beyond this the
// result is reported as truncated rather than silently partial.
maxScanBytes = 8 << 20
)
// Result is a page of log entries, newest last.
type Result struct {
Entries []Entry `json:"entries"`
// Truncated reports that the scan hit maxScanBytes before satisfying the
// query, so older matching entries exist that were not examined.
Truncated bool `json:"truncated"`
// File is the path read, for the UI to show where this came from.
File string `json:"file"`
// Found reports whether the file existed. False with a nil error is not a
// failure: a caller asking for a log that has no file yet, or one whose
// attempt produced no output, must be told that rather than shown an empty
// page indistinguishable from a log that is genuinely empty. This is also
// what separates "there is no log for this attempt" from "this process
// cannot read logs at all" -- two conditions that a reader has to report
// differently because only the second is a configuration matter.
Found bool `json:"found"`
}
// TaskLogPage is one task attempt's log as a task-log reader returns it: the
// page Tail selected, whether the attempt has a file at all, and the distinct
// components present in the file's tail so a filter is built from what is
// actually there rather than a hardcoded list that drifts.
type TaskLogPage struct {
Result
Components []string `json:"components"`
}
// ErrTaskLogsUnavailable reports that this process cannot read task logs at
// all -- no reader is configured here. It is deliberately not the same answer
// as "this attempt has no log" (Result.Found = false): a caller that conflates
// them tells an operator task logging is switched off when it is not, which is
// the bug this error exists to prevent.
var ErrTaskLogsUnavailable = errors.New("logging: task log reader unavailable")
// taskLogChunkBytes bounds one write a TaskLogContent makes, so a reader that
// pushes into a bounded transport emits chunks that transport can carry.
// io.Copy would otherwise hand it reads of its own choosing.
const taskLogChunkBytes = 256 << 10
// TaskLog reads one task attempt's persisted log from this registry. A nil
// registry (task logging not configured for this process) reports
// ErrTaskLogsUnavailable, never an empty page: a process that cannot read logs
// must say so rather than implicate the attempt. Safe to call on a nil
// receiver, like every other method here.
func (r *TaskRegistry) TaskLog(_ context.Context, taskID int64, attempt int, q Query) (TaskLogPage, error) {
if r == nil {
return TaskLogPage{}, ErrTaskLogsUnavailable
}
res, err := Tail(TaskLogPath(r.baseDir, taskID, attempt), q)
if err != nil {
return TaskLogPage{}, err
}
page := TaskLogPage{Result: res, Components: []string{}}
// The filter list is a convenience: failing to build it must not cost the
// caller their entries. Components only reads the tail this page already
// read, so its error is the same "treat as empty" case the daemon-wide
// handler has always applied -- it is dropped deliberately, not missed.
if res.Found {
if components, err := Components(res.File); err == nil {
page.Components = components
}
}
return page, nil
}
// TaskLogContent writes one task attempt's log to w exactly as it is on disk,
// for a caller that wants the file rather than a decoded page. found is false,
// with a nil error, when the attempt has no log file -- a normal state for an
// attempt that produced no output, and not an error. A nil registry reports
// ErrTaskLogsUnavailable, matching TaskLog.
//
// The content is streamed rather than returned as one slice because a log file
// is unbounded input (it rotates at DefaultMaxSizeMB), and a caller moving it
// across a transport with a message-size limit needs it in pieces.
func (r *TaskRegistry) TaskLogContent(_ context.Context, taskID int64, attempt int, w io.Writer) (bool, error) {
if r == nil {
return false, ErrTaskLogsUnavailable
}
f, err := os.Open(TaskLogPath(r.baseDir, taskID, attempt))
if err != nil {
if os.IsNotExist(err) {
return false, nil
}
return false, fmt.Errorf("logging: open task log: %w", err)
}
defer func() { _ = f.Close() }()
if _, err := io.CopyBuffer(w, f, make([]byte, taskLogChunkBytes)); err != nil {
return false, fmt.Errorf("logging: read task log: %w", err)
}
return true, nil
}
// PageResult is what a single forward-paged call returns.
//
// Cursor is the byte offset within path at which the next page should
// resume -- pass it back as Page's cursor argument. A Cursor equal to
// or past the file's current size means there is nothing more to read.
// MoreAvailable is true iff the scan saw at least one matching entry it
// did not return because Limit was already satisfied; it does NOT imply
// the file has been read to the end (combine with Truncated).
type PageResult struct {
Entries []Entry `json:"entries"`
Truncated bool `json:"truncated"`
MoreAvailable bool `json:"more_available"`
Cursor int64 `json:"cursor"`
File string `json:"file"`
}
// logWindow is the region of a log file one read examines: the
// tail-most maxScanBytes, held in memory so offset arithmetic is
// against a contiguous buffer rather than a live file position.
type logWindow struct {
// data is the window's bytes; data[0] is at file offset start.
data []byte
// start is the file offset data begins at (0 when the whole file fits).
start int64
// fileSize is the file's size when the window was taken.
fileSize int64
// truncated reports that data is a strict suffix of the file, so
// older bytes exist that this window does not contain.
truncated bool
}
// readWindow opens path and reads its scan window into memory.
// found is false (with a nil error) for an absent file or empty path:
// file logging is optional, so neither is an error condition.
func readWindow(path string) (w logWindow, found bool, err error) {
if strings.TrimSpace(path) == "" {
return logWindow{}, false, nil
}
f, err := os.Open(path)
if err != nil {
if os.IsNotExist(err) {
return logWindow{}, false, nil
}
return logWindow{}, false, fmt.Errorf("logging: open %s: %w", path, err)
}
defer func() { _ = f.Close() }()
info, err := f.Stat()
if err != nil {
return logWindow{}, false, fmt.Errorf("logging: stat %s: %w", path, err)
}
w = logWindow{fileSize: info.Size()}
size := info.Size()
if size > maxScanBytes {
w.start = size - maxScanBytes
size = maxScanBytes
w.truncated = true
}
w.data = make([]byte, size)
// ReadAt, never Read: io.ReaderAt guarantees it either fills the
// buffer or returns a non-nil error, whereas Read may legally return
// fewer bytes with a nil error. A silent short read here would leave
// the tail of the buffer as zero bytes, which decode rejects -- so
// the NEWEST entries would vanish from every result while the call
// still reported success. That is the worst possible failure for a
// log reader, and it is the one Read invites.
//
// Guarded on len>0 because a zero-length ReadAt at end-of-file is
// permitted to return io.EOF, and an empty log is a valid empty
// result rather than an error.
if len(w.data) > 0 {
if _, err := f.ReadAt(w.data, w.start); err != nil {
return logWindow{}, false, fmt.Errorf("logging: read %s: %w", path, err)
}
}
return w, true, nil
}
// readLines walks every JSONL line in path within the scan window
// (tail-most maxScanBytes), invoking step with each decoded entry that
// matches q and the file offset just past the line's terminating '\n'.
// It returns whether the scan hit maxScanBytes before EOF (so older
// matching entries exist that were not examined) and the file offset
// the caller should use as its next-page Cursor.
//
// cursor=0 begins at the scan window's start (same as Tail); a cursor
// past the scan window's start resumes partway through. A cursor at or
// past EOF returns ok=false without invoking step -- either the caller
// has read everything, or the file shrank under rotation.
//
// SCAN WINDOW IS A HARD CEILING, and it bounds pagination too. For a
// file larger than maxScanBytes the oldest bytes are never examined by
// ANY call, no matter how the caller pages: every call re-derives the
// window from the CURRENT file size, and the cursor is clamped up into
// it. Paging therefore walks the window exhaustively and stops; it does
// not walk backwards into history. truncated=true is the signal that
// this happened, and it is the caller's job to surface that rather than
// implying the log was read whole. Rotated generations
// (<path>.1, .2, ...) are separate files this function never opens.
//
// Implementation: the window is read into memory in one Read, then
// walked by exact slice offsets. Offsets into a single contiguous
// buffer are trivially correct, which an incremental reader over the
// file is not: bufio.Reader buffers ahead, so its underlying file
// position runs up to one buffer past its logical position and cannot
// be used to derive a cursor.
func readLines(path string, q Query, cursor int64, step func(e Entry, endOff int64) bool) (truncated bool, cursorOut int64, ok bool, err error) {
w, found, err := readWindow(path)
if err != nil || !found {
return false, 0, false, err
}
truncated = w.truncated
windowStart, data := w.start, w.data
// Clamp the caller's cursor to the scan window. A cursor below
// windowStart cannot read older history than Tail would have shown;
// we clamp upward so the returned cursor stays inside the window.
if cursor < windowStart {
cursor = windowStart
}
// A cursor at or past EOF means the caller has read everything we
// can serve. Return ok=false so the caller stops looping; the
// preserved cursor lets it detect overshoot (file shrank under
// rotation) versus exact-fit.
if cursor >= w.fileSize {
return truncated, cursor, false, nil
}
// The caller's cursor is a file offset. Convert to an offset within
// `data` (which begins at windowStart). The clamp above guarantees
// cursor >= windowStart, so this cannot go negative; min() only
// guards against a caller-supplied cursor inside the window but past
// its end, which the EOF check above does not catch when the file is
// larger than maxScanBytes.
cursorInWindow := min(cursor-windowStart, int64(len(data)))
// When the caller resumes at the very start of the scan window, the
// first line may have been truncated by the size-cap seek. Drop
// bytes up to and including the first '\\n' so the caller never sees
// a partial first entry. If there is no '\\n' at all, the file (or
// the visible window) is one giant unterminated line: report it as
// truncated (already true) and stop -- there is nothing for the
// caller to page through.
if cursor == windowStart && truncated {
idx := bytes.IndexByte(data, '\n')
if idx < 0 {
return truncated, w.fileSize, false, nil
}
cursorInWindow = int64(idx + 1)
}
return truncated, walkWindow(data, windowStart, cursorInWindow, q, step), true, nil
}
// walkWindow invokes step for each entry in data at or after the
// in-buffer offset from that decodes and matches q, and returns the FILE
// offset at which the next page should resume.
//
// step receives the file offset just past the line it was given, and
// returns false to stop the walk; walkWindow then returns the cursor
// exactly there. A line that fails to decode is skipped rather than
// ending the walk -- a log may carry subprocess output that does not
// share our format.
//
// NOTE: a `false` return from step does NOT mean "no more matches". It
// means "you have enough for this page; stop here so you can return to
// the caller." The next call, started from the cursor walkWindow
// returns, will revisit the file offset step saw and continue walking.
// The caller is responsible for distinguishing the two outcomes via
// the limit it passed.
//
// The returned cursor is the FILE offset of the first byte after the
// last line walked, regardless of whether step stopped the walk early
// or the buffer was exhausted. Cursor values are in the same coordinate
// system across calls (file offset), so passing one straight back into
// the next Page call resumes the walk cleanly.
func walkWindow(data []byte, windowStart, from int64, q Query, step func(e Entry, endOff int64) bool) int64 {
pos := from
for pos < int64(len(data)) {
// lineEnd indexes the line's terminating '\n'; when the final
// line is unterminated (daemon killed mid-write) it indexes the
// end of the buffer and endOff stops there.
nl := bytes.IndexByte(data[pos:], '\n')
lineEnd := int64(len(data))
endOff := windowStart + int64(len(data))
if nl >= 0 {
lineEnd = pos + int64(nl)
endOff = windowStart + lineEnd + 1 // past the '\n'
}
raw := data[pos:lineEnd] // excludes the '\n'
if len(raw) > 0 && raw[len(raw)-1] == '\r' {
raw = raw[:len(raw)-1] // CRLF
}
if entry, decoded := decode(raw); decoded && q.matches(entry) && !step(entry, endOff) {
return endOff
}
if nl < 0 {
return windowStart + int64(len(data))
}
pos = lineEnd + 1
}
return windowStart + pos
}
// Tail returns the most recent entries in path matching q.
//
// Entries are returned oldest-first so the UI can append newer ones from the
// live stream without reordering. A missing file is not an error: file logging
// is optional, and an empty result with a clear File value lets the caller say
// so rather than showing a failure.
//
// Tail preserves the legacy ring-buffer semantics: when more than Limit
// matching entries exist within the scan window, the OLDEST matches are
// dropped and Truncated is set. Page is the forward-paged alternative --
// callers that want every entry reachable should use Page, not Tail.
func Tail(path string, q Query) (Result, error) {
res := Result{Entries: []Entry{}, File: path}
limit := q.Limit
if limit <= 0 {
limit = DefaultTailLines
}
if limit > MaxTailLines {
limit = MaxTailLines
}
// found is read before readLines so an absent file is reported as such
// rather than only as an empty entry list -- see Result.Found. It is a
// Stat rather than a second readWindow: readLines reads the whole scan
// window, and doing that twice per Tail call would double the cost of
// every log view for one boolean.
if strings.TrimSpace(path) != "" {
if _, err := os.Stat(path); err == nil {
res.Found = true
} else if !os.IsNotExist(err) {
return Result{}, fmt.Errorf("logging: stat %s: %w", path, err)
}
}
matches := make([]Entry, 0, limit)
truncated, _, ok, err := readLines(path, q, 0, func(e Entry, _ int64) bool {
if len(matches) == limit {
matches = append(matches[:0], matches[1:]...)
res.Truncated = true
}
matches = append(matches, e)
return true
})
if err != nil {
return Result{}, err
}
if !ok {
// A "nothing to read" early return (file > maxScanBytes with no
// terminator in the window) is still Truncated: the scan cap
// fired. Preserve that signal even though we have no entries to
// surface -- the caller must know that older matching entries
// exist beyond what we examined.
res.Truncated = truncated
return res, nil
}
res.Truncated = res.Truncated || truncated
res.Entries = matches
return res, nil
}
// Page walks path forward from cursor, returning up to Limit matching
// entries (clamped to MaxTailLines). A cursor of 0 starts at the
// beginning of the scanned window, exactly as Tail does.
//
// The returned Cursor is the byte offset at which the next call should
// resume; pass it back unchanged. When the file has been read to EOF
// within maxScanBytes, Cursor equals the file's size and MoreAvailable
// is false. Truncated is true when the scan cap fired before EOF,
// meaning older matching entries exist that were never examined -- the
// caller must decide whether to walk the rotation history or surface
// the gap.
//
// A cursor that points past the file's current size returns an empty
// page with MoreAvailable=false: rotation may have shrunk the file, and
// the next page is whatever the file now contains.
//
// Cursor=0 with a file larger than maxScanBytes begins at the first
// complete line AFTER the size-cap seek, exactly as Tail does --
// identical output to Tail under the same query when Limit is large
// enough to hold everything.
func Page(path string, q Query, cursor int64) (PageResult, error) {
res := PageResult{Entries: []Entry{}, File: path}
limit := q.Limit
if limit <= 0 {
limit = DefaultTailLines
}
if limit > MaxTailLines {
limit = MaxTailLines
}
type lineAt struct {
entry Entry
endOff int64
}
matches := make([]lineAt, 0, limit)
truncated, endOff, ok, err := readLines(path, q, cursor, func(e Entry, end int64) bool {
if len(matches) < limit {
matches = append(matches, lineAt{entry: e, endOff: end})
return true
}
// Once we have limit matches, any further match means more is
// available; we keep walking only to maintain endOff accuracy.
res.MoreAvailable = true
return true
})
if err != nil {
return PageResult{}, err
}
if !ok {
// Preserved endOff carries the cursor the caller asked for
// (when ok=false because cursor > EOF, so the next call can
// detect overshoot). The scan-cap signal is also preserved.
res.Truncated = truncated
res.Cursor = endOff
return res, nil
}
res.Truncated = truncated
res.Entries = make([]Entry, len(matches))
for i, m := range matches {
res.Entries[i] = m.entry
}
// The cursor must resume AFTER the last entry this page returned, so
// the next call does not re-walk the same entries. When the walk
// stopped because the limit was reached (MoreAvailable), that resume
// point is the end of the last matching line. When the walk ran the
// whole buffer to completion (no more matches), the cursor is the
// walk's own end offset -- which can sit past trailing lines that
// failed the filter or failed to decode, and must, otherwise the next
// call would re-walk those trailing bytes forever. The two values
// differ only in the MoreAvailable=false case, which is exactly the
// case where the doc comment promises "Cursor equals the file's size".
if res.MoreAvailable {
res.Cursor = matches[len(matches)-1].endOff
} else {
res.Cursor = endOff
}
return res, nil
}
// decode parses one slog JSON line, lifting the known keys out and keeping the
// rest as structured fields. A line that is not JSON is skipped: a log file may
// contain output from a subprocess that does not share our format.
func decode(line []byte) (Entry, bool) {
var raw map[string]any
if err := json.Unmarshal(line, &raw); err != nil {
return Entry{}, false
}
entry := Entry{Fields: map[string]any{}}
for k, v := range raw {
switch k {
case "time":
if s, ok := v.(string); ok {
entry.Time, _ = time.Parse(time.RFC3339Nano, s)
}
case "level":
entry.Level, _ = v.(string)
case "msg":
entry.Message, _ = v.(string)
default:
entry.Fields[k] = v
}
}
if len(entry.Fields) == 0 {
entry.Fields = nil
}
return entry, true
}
// withinTimeBounds reports whether t satisfies q's Since/Until bounds.
// Both bounds are inclusive, and an unset (zero) bound is no bound.
//
// An entry whose timestamp did not parse -- missing, non-string, or
// malformed "time", all of which leave decode's Entry.Time zero rather
// than dropping the record -- cannot be placed in a range, so ANY
// time-bounded query excludes it. That check is explicit rather than
// implied, because the two comparisons are not symmetric about the zero
// time on their own: zero.Before(since) is true so Since would drop such
// entries, while zero.After(until) is false so Until would silently keep
// them, and "errors before noon" would come back carrying records with
// no known time at all.
func (q Query) withinTimeBounds(t time.Time) bool {
if q.Since.IsZero() && q.Until.IsZero() {
return true
}
if t.IsZero() {
return false
}
if !q.Since.IsZero() && t.Before(q.Since) {
return false
}
return q.Until.IsZero() || !t.After(q.Until)
}
func (q Query) matches(e Entry) bool {
if len(q.Levels) > 0 && !slices.ContainsFunc(q.Levels, func(l string) bool {
return strings.EqualFold(l, e.Level)
}) {
return false
}
if q.Component != "" {
got, _ := e.Fields["component"].(string)
if !strings.EqualFold(got, q.Component) {
return false
}
}
if q.Stage != "" {
got, _ := e.Fields["stage"].(string)
if !strings.EqualFold(got, q.Stage) {
return false
}
}
if !q.withinTimeBounds(e.Time) {
return false
}
if q.Contains != "" {
needle := strings.ToLower(q.Contains)
if strings.Contains(strings.ToLower(e.Message), needle) {
return true
}
for _, v := range e.Fields {
if strings.Contains(strings.ToLower(fmt.Sprint(v)), needle) {
return true
}
}
return false
}
return true
}
// Components returns the distinct "component" values present in the tail, so
// the UI can offer a filter built from what is actually there rather than a
// hardcoded list that drifts.
func Components(path string) ([]string, error) {
res, err := Tail(path, Query{Limit: MaxTailLines})
if err != nil {
return nil, err
}
seen := map[string]struct{}{}
out := []string{}
for _, e := range res.Entries {
if c, ok := e.Fields["component"].(string); ok && c != "" {
if _, dup := seen[c]; !dup {
seen[c] = struct{}{}
out = append(out, c)
}
}
}
slices.Sort(out)
return out, nil
}