docs(prds): complete the image gen/edit PRD across all five open beads

The contract-only doc (#497) explicitly deferred hosted provider (#498),
local GPU provider (#499), /image routing (#500), and channel/web delivery
(#501) to sibling beads with no design recorded. Fill in all four plus a
call-site inventory:

- Hosted: decide ai-sdk for generate, direct HTTP for edit (ai-sdk has no
  edit support) -- the decision the original doc explicitly punted on.
- Local: ComfyUI as the baseline recipe (no prior art in this repo to
  reconcile), submit/poll shaped like minimax.Client.GenerateAndWait.
- /image: Router.handleImage beside handleModel, per-request mechanism
  selection (session-scoped preference has no generic home in Router
  today, confirmed by reading its fields -- not invented here).
- Delivery: mostly already built. generate_video's MultimodalResult ->
  MediaEvent -> channel pipeline already handles "image" generically;
  Telegram already calls SendPhoto. Documented per-channel evidence
  instead of assuming a gap that isn't there.
This commit is contained in:
sam committed 2026-09-19 18:13:55 +10:00
1 parent d591d02f0b
commit 559e27dded
1 file changed
+222 -34
+222 -34
View File
@@ -1,11 +1,16 @@
# Image capability contract and configuration
Epic: archie-core-1786748942120-1-6636e629 ("Image generation capability:
hosted API baseline plus local GPU workflows").
Scope: archie-core-1786748942147-2-47cd5bd1 — the contract and configuration
only. Hosted provider implementation (gpt-image-2), local GPU backend,
`/image` routing, and delivery are separate, dependent beads and are not
built here.
Epic: archie-core-1786748942120-1-6636e629 / GitHub #496 ("Image generation
capability: hosted API baseline plus local GPU workflows").
The contract and configuration (#497) shipped first and is settled — see
"Package: `internal/domain/image`" below. This document now covers the full
feature set: hosted provider (#498), local GPU provider (#499), `/image`
routing and clarification (#500), delivery through channels and web (#501).
Documentation (#502) and the cross-cutting test matrix (#503) are execution
work against the design below, not open design questions, so they are not
given their own sections — each preceding section states its own test
obligations inline.
## Problem
@@ -233,37 +238,220 @@ whole point of the contract being provider-neutral.
- No provider implementation exists yet, so no live-call tests — those
belong to the hosted/local provider beads.
## Non-goals (deferred to sibling beads)
- Implementing any real `Provider` (hosted or local).
- `/image` clarification/routing flow and session-scoped mechanism
selection.
- Delivery through channel abstractions.
- Wiring `Registry` into `bootstrap.go` (nothing to wire until a real
provider exists to register).
## Note for the hosted-provider bead (archie-core-1786748942169-3-ae7cfaea)
## Hosted provider (#498)
`github.com/samcharles93/ai-sdk` (already a repo dependency) has its own
provider-agnostic image facility: `image.Provider` (`Name`,
`GenerateImage(ctx, GenerateImageRequest) (GenerateImageResponse, error)`),
`image.Client`/`core.GenerateImage`, and `registry.Registry.RegisterImage`/
`.Image(name)` — the same shape chat model registration already uses. Three
backends implement it today (azure, togetherai, xai); OpenAI does not yet.
`image.Client`, and `registry.Registry.RegisterImage`/`.Image(name)` — the
same shape chat model registration already uses. Three backends implement it
today (azure, togetherai, xai); OpenAI does not yet.
The hosted-provider bead should adapt this instead of writing a raw HTTP
client, the same way `curatorLLMRunner` (internal/app/archied/main.go)
adapts `runtime.Runtime` for chat rather than the domain layer talking to
ai-sdk directly — `internal/infrastructure/image.<name>Provider` wraps an
`ai-sdk/image.Provider` (or a new `ai-sdk/provider/openai` image
implementation, upstream, if one is added) behind this package's `Provider`
interface.
`internal/infrastructure/image.<name>Provider` wraps an `ai-sdk/image.Provider`
behind this package's `Provider` interface, the same way `curatorLLMRunner`
(`internal/app/archied/main.go`) adapts `runtime.Runtime` for chat rather
than the domain layer talking to ai-sdk directly.
One real gap: ai-sdk's `image.GenerateImageRequest` has no input-image or
mask fields — text-to-image only. `EditRequest`/`Capability.Edit` in this
contract has nothing to adapt to yet. The hosted-provider bead needs either
an upstream ai-sdk change (adding edit support to the `image` package) or a
direct, non-ai-sdk client for the edit endpoint specifically, with
generation still going through ai-sdk. Decide and record which at that
bead's own design-doc step; this doc takes no position since editing was
out of scope here.
**Decision: generation goes through ai-sdk, edit does not.** ai-sdk's
`image.GenerateImageRequest` is text-to-image only — no input-image or mask
fields — so there is nothing to adapt `EditRequest`/`Capability.Edit` onto.
Blocking the entire hosted provider on an upstream ai-sdk change is not
this epic's call to make on someone else's project's timeline. Instead:
- `Generate` wraps `ai-sdk/image.Client.GenerateImage` against whichever
backend `ImageHostedProvider.Class` names (azure/togetherai/xai today).
- `Edit` is a direct HTTP client hitting the OpenAI-compatible
`/images/edits` multipart endpoint, built the same way `minimax.Client`
(`internal/tools/minimax/minimax.go`) is a direct client for an API
ai-sdk doesn't cover — not a second abstraction layer, one `http.Client`
call with request/response mapping into this contract's `EditRequest`/
`Result`.
- A provider that only wraps ai-sdk (no edit client configured) declares
`Capability{Generate: true, Edit: false}` truthfully rather than
returning `ErrUnsupported` at call time for something `Capability`
already said it couldn't do.
- If ai-sdk gains edit support upstream later, the direct client is deleted
and `Edit` moves onto it — recorded here so that's a known, not a
surprise, follow-up.
**Provider error mapping** (every ai-sdk/HTTP error translated at the
adapter boundary, never leaked as-is):
| ai-sdk / HTTP condition | contract error |
|---|---|
| `image.ErrNoProvider`, `image.ErrAuthFailed`, missing/invalid credential, backend unreachable | `ErrUnavailable` |
| `image.ErrEditNotSupported`, request needs a `Capability` the provider didn't declare | `ErrUnsupported` |
| input image over configured byte limit (checked before the call, per "Timeouts, size limits, cancellation" above) | `ErrInputTooLarge` |
| `ctx.Err() == context.DeadlineExceeded` | `ErrTimeout` |
| `ctx.Err() == context.Canceled` | `ErrCanceled` |
| `image.ErrRateLimited`, `image.ErrContentFiltered`, any other 4xx/5xx | wrapped verbatim under `ErrUnavailable` with the provider's own message as detail — a rate limit or content filter is "unavailable for this request", not a new sentinel the caller needs to branch on differently |
**Testing**: `httptest.Server` fixtures for the edit client (request
multipart shape, response decoding, 4xx/5xx mapping); a fake `ai-sdk/image.Provider`
for the generate path so the adapter's translation is tested without a live
backend, matching #498's "no live API key needed for tests" criterion.
## Local GPU provider (#499)
**Decision: ComfyUI is the baseline recipe**, not a name this doc picked
arbitrarily — it is the de facto scriptable HTTP API for Stable-Diffusion-class
local generation, already shaped like the async submit/poll pattern
`minimax.Client.GenerateAndWait` established in this codebase (submit a
workflow, poll `/history/{id}` until the output node has files, download).
No other local backend is referenced anywhere in this repo or its docs, so
this decision has no competing prior art to reconcile — it is a fresh
choice, recorded here so the implementing bead does not re-litigate it.
```go
// internal/infrastructure/image/comfyui
type Config struct {
Enabled bool
BaseURL string // e.g. http://127.0.0.1:8188
Workflow string // path to a saved ComfyUI workflow JSON template
// (prompt/size/seed substituted at call time)
Timeout time.Duration
PollInterval time.Duration
MaxWait time.Duration
}
```
- `Generate`/`Edit` load `Workflow`, substitute the templated fields (prompt,
size, seed, and for `Edit` the input image as a base64 node input — ComfyUI
supports img2img/inpaint workflows natively, so `Capability.Edit`/`Mask`
can both be true for a correctly authored workflow, unlike the hosted
provider), POST to `/prompt`, then poll `/history/{prompt_id}` the same
shape MiniMax's poll loop already proves out in this codebase.
- `Capability.Sizes` is empty (ComfyUI workflows declare their own
resolution in the template, not a fixed enum) — the contract already
allows an empty `Sizes` to mean "provider does not constrain size".
- Backend-down (`BaseURL` unreachable) and workflow-missing are both
`ErrUnavailable`, not a startup failure — same "degrade the capability,
never break text chat" rule the contract already states.
- Per #499's own acceptance criteria, at least one recipe must be exercised
end-to-end on real hardware or a reproducible harness, and no model
weights are ever committed or fetched by Archie itself — the operator
points `Workflow`/`BaseURL` at their own running ComfyUI instance and
already-downloaded checkpoint, exactly like `[providers.ollama]` points at
an already-running Ollama rather than Archie managing the model.
**Testing**: a fake HTTP server standing in for ComfyUI's `/prompt` and
`/history` endpoints (submit, poll, malformed history, timeout, job
not-found); a real-hardware smoke test documented as a manual `task`
target per #503, not part of `task check` — the same reasoning `task vuln`
and `task deadcode` are excluded from the gate.
## `/image` routing and clarification flow (#500)
**Call site: `internal/gateway/gateway.go`'s `Router`**, following the
`/model` shape exactly — `/model`, `/spawn`, `/approve` etc. are already
"commands handled without the LLM" (`Router.handleModel`, dispatched via
`isLocalCommand`/`parseCmd` against `messaging.LocalCommandSpecs()`).
`/image` is a new entry in `internal/domain/messaging/commands.go`'s
`localCommandSpecs`, and a new `Router.handleImage(ctx, arg)` beside
`handleModel`, gated on a new `Router.Images *image.Registry` field (nil =
"/image not configured", the same pattern `Models`/`Tasks`/`Controller`
already use — not a new gating convention).
**Decision: mechanism selection is per-request, not session-persisted.**
The epic allows "a remembered preference only where existing session/config
conventions support it" — and checking `Router`'s actual fields
(`ModelPersist`, `Identity`, the rest) shows there is no generic per-session
key/value store the neutral command path can lean on today; `/model`'s own
`--session`/`--global` scoping lives entirely in
`internal/channels/telegram/model.go`, a Telegram-specific enrichment, not
something `Router` provides generically. Inventing a new generic
session-preference store just for `/image` would be exactly the kind of
scaffolding CLAUDE.md's Architectural Simplicity rule warns against for one
caller. So: every `/image` invocation with no explicit mechanism produces
the clarification prompt; an explicit `--hosted`/`--local` flag (mirroring
`parseModelRest`'s `--provider` flag shape) skips it for that one call.
Revisit if a second command wants session-scoped preference too — that's
the point at which shared infra earns its cost.
**Flow**, matching `handleModel`'s reply-string shape (no separate state
machine object, since the whole exchange is "ask once, use the answer
inline in the next message" — the model-selector's own worked example):
1. `/image <prompt>` with no `--hosted`/`--local` → reply lists the
registered classes (from `Registry.ByClass`) and asks the user to repeat
the command with `--hosted` or `--local`. No provider is called.
2. `/image --hosted <prompt>` or `--local` → resolves to
`Registry.ByClass(class)`, picks `ImageConfig.Default` if it names a
provider of that class, else the sole registered provider of that class,
else (more than one, no default) asks which named provider.
3. Edit requests (`/image --edit <prompt>` with an attached image on the
inbound message) validate against `Capability.Edit`/`MaxInputs`/`Mask`
before calling — an edit request to a generate-only provider is a
clarification reply ("this provider doesn't support editing"), not a
provider call that returns `ErrUnsupported`.
4. A successful call returns a `Result`; delivery is the next section's
concern, not this one's — `handleImage` returns the same
`(string, error)` shape `handleModel` does for the text portion, and a
separate mechanism (below) carries the actual image bytes.
**Testing**: channel-neutral state-machine tests for `handleImage` against
a fake `*image.Registry` (mirrors `#500`'s own acceptance criterion); Telegram
adapter tests only for wiring, not for re-testing the state machine.
## Delivery through channels and web (#501)
This is the smallest section because most of the delivery path **already
exists and is already proven** — by `generate_video`
(`internal/tools/minimax`), which ships today. Do not rebuild it.
The existing pipeline: a result carrying media returns
`tools.MultimodalResult{IsMultimodal: true, URLs: []tools.MediaRef{{Type: "image", ...}}}`
→ `multimodalMediaRefs` (`internal/app/archied/telegram_setup.go`) decodes
it → `TurnStream.Media(gateway.MediaEvent)` → each channel's own
`Media`/`MediaAttachment` handling. `MediaRef.Type` already documents
`"image"` as a valid value alongside `"video"`; nothing in the pipeline is
video-specific.
**Per-channel state, checked directly rather than assumed:**
| channel | image delivery | evidence |
|---|---|---|
| Telegram | **already works** | `internal/channels/telegram/media.go`'s `telegramMediaSender.dispatch` already branches on `att.Type` and calls `bot.SendPhoto` for `"image"` (`SendVideo` for `"video"` is the same function) |
| Dashboard (webui) | **tracked gap, not new** | `internal/webui/api_chat.go:355`'s `chatStreamSink.Media` has no inline rendering path yet and degrades to a link/error string — already tracked as archie-core-1786748942243-6-f109697e. This epic does not need to fix it, but #501's acceptance criterion ("unsupported paths report a clear limitation") is already met by the existing degrade-to-text behavior; closing the dashboard gap is that bead's job, not this one's |
| Email | **no attachment sender exists** | `grep MediaAttachment internal/channels/email` returns nothing. Out of scope for this epic per #496's non-goals (no channel-abstraction rework) — email delivery of a generated image degrades the same way the dashboard does today, which is acceptable under #501's "or unsupported paths report a clear limitation" clause |
| Webhook | not applicable | webhook is an inbound intake channel (`internal/channels/webhook`), not an outbound chat reply target — no `/image` command can originate there in the first place |
**The only new work `internal/tools/minimax` didn't already need**: the
autonomous-tool call site. `/image` (previous section) calls a provider
directly and does not go through the model's tool-calling loop at all.
Separately, exposing `generate_image`/`edit_image` as an ordinary tool
(so an agent task can generate a diagram or illustration mid-task, not only
a human typing `/image`) is a thin wrapper — `internal/tools/image/tool.go`,
same shape as `internal/tools/minimax/tool.go`, calling
`image.Registry.Get(name).Generate/Edit` and returning the identical
`MultimodalResult` shape. Wired in `bootstrap.go` next to
`registerMinimaxTool`, gated the same way (registered only when a provider
resolved a working credential/backend, never advertised broken). This
reuses the entire delivery pipeline above with zero new plumbing — the tool
call site is what's new, not the delivery.
**Testing**: `multimodalMediaRefs` already has coverage for the shape;
add a table case for `Type: "image"` if one doesn't already exist. The tool
wrapper gets the same fake-provider test shape as the routing flow above.
No new delivery-path tests are needed — that pipeline's tests belong to
whichever change originally proved it out.
## Call site inventory
Every file a full implementation touches, so no bead re-derives this by
grep:
| concern | file | change |
|---|---|---|
| domain contract | `internal/domain/image/{contract,registry}.go` | done (#497) |
| config schema + validation | `internal/infrastructure/configuration/{types,validate}.go` | done (#497); extended here only if ComfyUI's `Workflow` field needs a new validation rule (path exists, non-empty when `Local.Enabled`) |
| hosted provider | `internal/infrastructure/image/openai/*.go` (or per `Class`) | new (#498) |
| local provider | `internal/infrastructure/image/comfyui/*.go` | new (#499) |
| `/image` command spec | `internal/domain/messaging/commands.go` | new entry in `localCommandSpecs` |
| `/image` routing | `internal/gateway/gateway.go` (`Router.Images`, `handleImage`) | new (#500) |
| autonomous tool | `internal/tools/image/tool.go` | new (#501) |
| tool registration | `internal/app/archied/bootstrap.go` (`registerImageTool`, beside `registerMinimaxTool`) | new (#501) |
| `Registry` wiring | `internal/app/archied/bootstrap.go` (construct `image.Registry`, register configured providers, hand to `Router.Images`) | new — this is the "nothing to wire until a real provider exists" step the original contract doc deferred |
| Telegram delivery | `internal/channels/telegram/media.go` | none — already handles `"image"` |
| Dashboard delivery | `internal/webui/api_chat.go` | none in this epic — tracked separately (archie-core-1786748942243-6-f109697e) |
| docs | `docs/` user-facing guide (#502) | new, against this design |