docs(prds): complete the image gen/edit PRD across all five open beads
The contract-only doc (#497) explicitly deferred hosted provider (#498), local GPU provider (#499), /image routing (#500), and channel/web delivery (#501) to sibling beads with no design recorded. Fill in all four plus a call-site inventory: - Hosted: decide ai-sdk for generate, direct HTTP for edit (ai-sdk has no edit support) -- the decision the original doc explicitly punted on. - Local: ComfyUI as the baseline recipe (no prior art in this repo to reconcile), submit/poll shaped like minimax.Client.GenerateAndWait. - /image: Router.handleImage beside handleModel, per-request mechanism selection (session-scoped preference has no generic home in Router today, confirmed by reading its fields -- not invented here). - Delivery: mostly already built. generate_video's MultimodalResult -> MediaEvent -> channel pipeline already handles "image" generically; Telegram already calls SendPhoto. Documented per-channel evidence instead of assuming a gap that isn't there.
This commit is contained in:
1 parent
d591d02f0b
commit
559e27dded
1 file changed
+222
-34
@@ -1,11 +1,16 @@
|
||||
# Image capability contract and configuration
|
||||
|
||||
Epic: archie-core-1786748942120-1-6636e629 ("Image generation capability:
|
||||
hosted API baseline plus local GPU workflows").
|
||||
Scope: archie-core-1786748942147-2-47cd5bd1 — the contract and configuration
|
||||
only. Hosted provider implementation (gpt-image-2), local GPU backend,
|
||||
`/image` routing, and delivery are separate, dependent beads and are not
|
||||
built here.
|
||||
Epic: archie-core-1786748942120-1-6636e629 / GitHub #496 ("Image generation
|
||||
capability: hosted API baseline plus local GPU workflows").
|
||||
|
||||
The contract and configuration (#497) shipped first and is settled — see
|
||||
"Package: `internal/domain/image`" below. This document now covers the full
|
||||
feature set: hosted provider (#498), local GPU provider (#499), `/image`
|
||||
routing and clarification (#500), delivery through channels and web (#501).
|
||||
Documentation (#502) and the cross-cutting test matrix (#503) are execution
|
||||
work against the design below, not open design questions, so they are not
|
||||
given their own sections — each preceding section states its own test
|
||||
obligations inline.
|
||||
|
||||
## Problem
|
||||
|
||||
@@ -233,37 +238,220 @@ whole point of the contract being provider-neutral.
|
||||
- No provider implementation exists yet, so no live-call tests — those
|
||||
belong to the hosted/local provider beads.
|
||||
|
||||
## Non-goals (deferred to sibling beads)
|
||||
|
||||
- Implementing any real `Provider` (hosted or local).
|
||||
- `/image` clarification/routing flow and session-scoped mechanism
|
||||
selection.
|
||||
- Delivery through channel abstractions.
|
||||
- Wiring `Registry` into `bootstrap.go` (nothing to wire until a real
|
||||
provider exists to register).
|
||||
|
||||
## Note for the hosted-provider bead (archie-core-1786748942169-3-ae7cfaea)
|
||||
## Hosted provider (#498)
|
||||
|
||||
`github.com/samcharles93/ai-sdk` (already a repo dependency) has its own
|
||||
provider-agnostic image facility: `image.Provider` (`Name`,
|
||||
`GenerateImage(ctx, GenerateImageRequest) (GenerateImageResponse, error)`),
|
||||
`image.Client`/`core.GenerateImage`, and `registry.Registry.RegisterImage`/
|
||||
`.Image(name)` — the same shape chat model registration already uses. Three
|
||||
backends implement it today (azure, togetherai, xai); OpenAI does not yet.
|
||||
`image.Client`, and `registry.Registry.RegisterImage`/`.Image(name)` — the
|
||||
same shape chat model registration already uses. Three backends implement it
|
||||
today (azure, togetherai, xai); OpenAI does not yet.
|
||||
|
||||
The hosted-provider bead should adapt this instead of writing a raw HTTP
|
||||
client, the same way `curatorLLMRunner` (internal/app/archied/main.go)
|
||||
adapts `runtime.Runtime` for chat rather than the domain layer talking to
|
||||
ai-sdk directly — `internal/infrastructure/image.<name>Provider` wraps an
|
||||
`ai-sdk/image.Provider` (or a new `ai-sdk/provider/openai` image
|
||||
implementation, upstream, if one is added) behind this package's `Provider`
|
||||
interface.
|
||||
`internal/infrastructure/image.<name>Provider` wraps an `ai-sdk/image.Provider`
|
||||
behind this package's `Provider` interface, the same way `curatorLLMRunner`
|
||||
(`internal/app/archied/main.go`) adapts `runtime.Runtime` for chat rather
|
||||
than the domain layer talking to ai-sdk directly.
|
||||
|
||||
One real gap: ai-sdk's `image.GenerateImageRequest` has no input-image or
|
||||
mask fields — text-to-image only. `EditRequest`/`Capability.Edit` in this
|
||||
contract has nothing to adapt to yet. The hosted-provider bead needs either
|
||||
an upstream ai-sdk change (adding edit support to the `image` package) or a
|
||||
direct, non-ai-sdk client for the edit endpoint specifically, with
|
||||
generation still going through ai-sdk. Decide and record which at that
|
||||
bead's own design-doc step; this doc takes no position since editing was
|
||||
out of scope here.
|
||||
**Decision: generation goes through ai-sdk, edit does not.** ai-sdk's
|
||||
`image.GenerateImageRequest` is text-to-image only — no input-image or mask
|
||||
fields — so there is nothing to adapt `EditRequest`/`Capability.Edit` onto.
|
||||
Blocking the entire hosted provider on an upstream ai-sdk change is not
|
||||
this epic's call to make on someone else's project's timeline. Instead:
|
||||
|
||||
- `Generate` wraps `ai-sdk/image.Client.GenerateImage` against whichever
|
||||
backend `ImageHostedProvider.Class` names (azure/togetherai/xai today).
|
||||
- `Edit` is a direct HTTP client hitting the OpenAI-compatible
|
||||
`/images/edits` multipart endpoint, built the same way `minimax.Client`
|
||||
(`internal/tools/minimax/minimax.go`) is a direct client for an API
|
||||
ai-sdk doesn't cover — not a second abstraction layer, one `http.Client`
|
||||
call with request/response mapping into this contract's `EditRequest`/
|
||||
`Result`.
|
||||
- A provider that only wraps ai-sdk (no edit client configured) declares
|
||||
`Capability{Generate: true, Edit: false}` truthfully rather than
|
||||
returning `ErrUnsupported` at call time for something `Capability`
|
||||
already said it couldn't do.
|
||||
- If ai-sdk gains edit support upstream later, the direct client is deleted
|
||||
and `Edit` moves onto it — recorded here so that's a known, not a
|
||||
surprise, follow-up.
|
||||
|
||||
**Provider error mapping** (every ai-sdk/HTTP error translated at the
|
||||
adapter boundary, never leaked as-is):
|
||||
|
||||
| ai-sdk / HTTP condition | contract error |
|
||||
|---|---|
|
||||
| `image.ErrNoProvider`, `image.ErrAuthFailed`, missing/invalid credential, backend unreachable | `ErrUnavailable` |
|
||||
| `image.ErrEditNotSupported`, request needs a `Capability` the provider didn't declare | `ErrUnsupported` |
|
||||
| input image over configured byte limit (checked before the call, per "Timeouts, size limits, cancellation" above) | `ErrInputTooLarge` |
|
||||
| `ctx.Err() == context.DeadlineExceeded` | `ErrTimeout` |
|
||||
| `ctx.Err() == context.Canceled` | `ErrCanceled` |
|
||||
| `image.ErrRateLimited`, `image.ErrContentFiltered`, any other 4xx/5xx | wrapped verbatim under `ErrUnavailable` with the provider's own message as detail — a rate limit or content filter is "unavailable for this request", not a new sentinel the caller needs to branch on differently |
|
||||
|
||||
**Testing**: `httptest.Server` fixtures for the edit client (request
|
||||
multipart shape, response decoding, 4xx/5xx mapping); a fake `ai-sdk/image.Provider`
|
||||
for the generate path so the adapter's translation is tested without a live
|
||||
backend, matching #498's "no live API key needed for tests" criterion.
|
||||
|
||||
## Local GPU provider (#499)
|
||||
|
||||
**Decision: ComfyUI is the baseline recipe**, not a name this doc picked
|
||||
arbitrarily — it is the de facto scriptable HTTP API for Stable-Diffusion-class
|
||||
local generation, already shaped like the async submit/poll pattern
|
||||
`minimax.Client.GenerateAndWait` established in this codebase (submit a
|
||||
workflow, poll `/history/{id}` until the output node has files, download).
|
||||
No other local backend is referenced anywhere in this repo or its docs, so
|
||||
this decision has no competing prior art to reconcile — it is a fresh
|
||||
choice, recorded here so the implementing bead does not re-litigate it.
|
||||
|
||||
```go
|
||||
// internal/infrastructure/image/comfyui
|
||||
type Config struct {
|
||||
Enabled bool
|
||||
BaseURL string // e.g. http://127.0.0.1:8188
|
||||
Workflow string // path to a saved ComfyUI workflow JSON template
|
||||
// (prompt/size/seed substituted at call time)
|
||||
Timeout time.Duration
|
||||
PollInterval time.Duration
|
||||
MaxWait time.Duration
|
||||
}
|
||||
```
|
||||
|
||||
- `Generate`/`Edit` load `Workflow`, substitute the templated fields (prompt,
|
||||
size, seed, and for `Edit` the input image as a base64 node input — ComfyUI
|
||||
supports img2img/inpaint workflows natively, so `Capability.Edit`/`Mask`
|
||||
can both be true for a correctly authored workflow, unlike the hosted
|
||||
provider), POST to `/prompt`, then poll `/history/{prompt_id}` the same
|
||||
shape MiniMax's poll loop already proves out in this codebase.
|
||||
- `Capability.Sizes` is empty (ComfyUI workflows declare their own
|
||||
resolution in the template, not a fixed enum) — the contract already
|
||||
allows an empty `Sizes` to mean "provider does not constrain size".
|
||||
- Backend-down (`BaseURL` unreachable) and workflow-missing are both
|
||||
`ErrUnavailable`, not a startup failure — same "degrade the capability,
|
||||
never break text chat" rule the contract already states.
|
||||
- Per #499's own acceptance criteria, at least one recipe must be exercised
|
||||
end-to-end on real hardware or a reproducible harness, and no model
|
||||
weights are ever committed or fetched by Archie itself — the operator
|
||||
points `Workflow`/`BaseURL` at their own running ComfyUI instance and
|
||||
already-downloaded checkpoint, exactly like `[providers.ollama]` points at
|
||||
an already-running Ollama rather than Archie managing the model.
|
||||
|
||||
**Testing**: a fake HTTP server standing in for ComfyUI's `/prompt` and
|
||||
`/history` endpoints (submit, poll, malformed history, timeout, job
|
||||
not-found); a real-hardware smoke test documented as a manual `task`
|
||||
target per #503, not part of `task check` — the same reasoning `task vuln`
|
||||
and `task deadcode` are excluded from the gate.
|
||||
|
||||
## `/image` routing and clarification flow (#500)
|
||||
|
||||
**Call site: `internal/gateway/gateway.go`'s `Router`**, following the
|
||||
`/model` shape exactly — `/model`, `/spawn`, `/approve` etc. are already
|
||||
"commands handled without the LLM" (`Router.handleModel`, dispatched via
|
||||
`isLocalCommand`/`parseCmd` against `messaging.LocalCommandSpecs()`).
|
||||
`/image` is a new entry in `internal/domain/messaging/commands.go`'s
|
||||
`localCommandSpecs`, and a new `Router.handleImage(ctx, arg)` beside
|
||||
`handleModel`, gated on a new `Router.Images *image.Registry` field (nil =
|
||||
"/image not configured", the same pattern `Models`/`Tasks`/`Controller`
|
||||
already use — not a new gating convention).
|
||||
|
||||
**Decision: mechanism selection is per-request, not session-persisted.**
|
||||
The epic allows "a remembered preference only where existing session/config
|
||||
conventions support it" — and checking `Router`'s actual fields
|
||||
(`ModelPersist`, `Identity`, the rest) shows there is no generic per-session
|
||||
key/value store the neutral command path can lean on today; `/model`'s own
|
||||
`--session`/`--global` scoping lives entirely in
|
||||
`internal/channels/telegram/model.go`, a Telegram-specific enrichment, not
|
||||
something `Router` provides generically. Inventing a new generic
|
||||
session-preference store just for `/image` would be exactly the kind of
|
||||
scaffolding CLAUDE.md's Architectural Simplicity rule warns against for one
|
||||
caller. So: every `/image` invocation with no explicit mechanism produces
|
||||
the clarification prompt; an explicit `--hosted`/`--local` flag (mirroring
|
||||
`parseModelRest`'s `--provider` flag shape) skips it for that one call.
|
||||
Revisit if a second command wants session-scoped preference too — that's
|
||||
the point at which shared infra earns its cost.
|
||||
|
||||
**Flow**, matching `handleModel`'s reply-string shape (no separate state
|
||||
machine object, since the whole exchange is "ask once, use the answer
|
||||
inline in the next message" — the model-selector's own worked example):
|
||||
|
||||
1. `/image <prompt>` with no `--hosted`/`--local` → reply lists the
|
||||
registered classes (from `Registry.ByClass`) and asks the user to repeat
|
||||
the command with `--hosted` or `--local`. No provider is called.
|
||||
2. `/image --hosted <prompt>` or `--local` → resolves to
|
||||
`Registry.ByClass(class)`, picks `ImageConfig.Default` if it names a
|
||||
provider of that class, else the sole registered provider of that class,
|
||||
else (more than one, no default) asks which named provider.
|
||||
3. Edit requests (`/image --edit <prompt>` with an attached image on the
|
||||
inbound message) validate against `Capability.Edit`/`MaxInputs`/`Mask`
|
||||
before calling — an edit request to a generate-only provider is a
|
||||
clarification reply ("this provider doesn't support editing"), not a
|
||||
provider call that returns `ErrUnsupported`.
|
||||
4. A successful call returns a `Result`; delivery is the next section's
|
||||
concern, not this one's — `handleImage` returns the same
|
||||
`(string, error)` shape `handleModel` does for the text portion, and a
|
||||
separate mechanism (below) carries the actual image bytes.
|
||||
|
||||
**Testing**: channel-neutral state-machine tests for `handleImage` against
|
||||
a fake `*image.Registry` (mirrors `#500`'s own acceptance criterion); Telegram
|
||||
adapter tests only for wiring, not for re-testing the state machine.
|
||||
|
||||
## Delivery through channels and web (#501)
|
||||
|
||||
This is the smallest section because most of the delivery path **already
|
||||
exists and is already proven** — by `generate_video`
|
||||
(`internal/tools/minimax`), which ships today. Do not rebuild it.
|
||||
|
||||
The existing pipeline: a result carrying media returns
|
||||
`tools.MultimodalResult{IsMultimodal: true, URLs: []tools.MediaRef{{Type: "image", ...}}}`
|
||||
→ `multimodalMediaRefs` (`internal/app/archied/telegram_setup.go`) decodes
|
||||
it → `TurnStream.Media(gateway.MediaEvent)` → each channel's own
|
||||
`Media`/`MediaAttachment` handling. `MediaRef.Type` already documents
|
||||
`"image"` as a valid value alongside `"video"`; nothing in the pipeline is
|
||||
video-specific.
|
||||
|
||||
**Per-channel state, checked directly rather than assumed:**
|
||||
|
||||
| channel | image delivery | evidence |
|
||||
|---|---|---|
|
||||
| Telegram | **already works** | `internal/channels/telegram/media.go`'s `telegramMediaSender.dispatch` already branches on `att.Type` and calls `bot.SendPhoto` for `"image"` (`SendVideo` for `"video"` is the same function) |
|
||||
| Dashboard (webui) | **tracked gap, not new** | `internal/webui/api_chat.go:355`'s `chatStreamSink.Media` has no inline rendering path yet and degrades to a link/error string — already tracked as archie-core-1786748942243-6-f109697e. This epic does not need to fix it, but #501's acceptance criterion ("unsupported paths report a clear limitation") is already met by the existing degrade-to-text behavior; closing the dashboard gap is that bead's job, not this one's |
|
||||
| Email | **no attachment sender exists** | `grep MediaAttachment internal/channels/email` returns nothing. Out of scope for this epic per #496's non-goals (no channel-abstraction rework) — email delivery of a generated image degrades the same way the dashboard does today, which is acceptable under #501's "or unsupported paths report a clear limitation" clause |
|
||||
| Webhook | not applicable | webhook is an inbound intake channel (`internal/channels/webhook`), not an outbound chat reply target — no `/image` command can originate there in the first place |
|
||||
|
||||
**The only new work `internal/tools/minimax` didn't already need**: the
|
||||
autonomous-tool call site. `/image` (previous section) calls a provider
|
||||
directly and does not go through the model's tool-calling loop at all.
|
||||
Separately, exposing `generate_image`/`edit_image` as an ordinary tool
|
||||
(so an agent task can generate a diagram or illustration mid-task, not only
|
||||
a human typing `/image`) is a thin wrapper — `internal/tools/image/tool.go`,
|
||||
same shape as `internal/tools/minimax/tool.go`, calling
|
||||
`image.Registry.Get(name).Generate/Edit` and returning the identical
|
||||
`MultimodalResult` shape. Wired in `bootstrap.go` next to
|
||||
`registerMinimaxTool`, gated the same way (registered only when a provider
|
||||
resolved a working credential/backend, never advertised broken). This
|
||||
reuses the entire delivery pipeline above with zero new plumbing — the tool
|
||||
call site is what's new, not the delivery.
|
||||
|
||||
**Testing**: `multimodalMediaRefs` already has coverage for the shape;
|
||||
add a table case for `Type: "image"` if one doesn't already exist. The tool
|
||||
wrapper gets the same fake-provider test shape as the routing flow above.
|
||||
No new delivery-path tests are needed — that pipeline's tests belong to
|
||||
whichever change originally proved it out.
|
||||
|
||||
## Call site inventory
|
||||
|
||||
Every file a full implementation touches, so no bead re-derives this by
|
||||
grep:
|
||||
|
||||
| concern | file | change |
|
||||
|---|---|---|
|
||||
| domain contract | `internal/domain/image/{contract,registry}.go` | done (#497) |
|
||||
| config schema + validation | `internal/infrastructure/configuration/{types,validate}.go` | done (#497); extended here only if ComfyUI's `Workflow` field needs a new validation rule (path exists, non-empty when `Local.Enabled`) |
|
||||
| hosted provider | `internal/infrastructure/image/openai/*.go` (or per `Class`) | new (#498) |
|
||||
| local provider | `internal/infrastructure/image/comfyui/*.go` | new (#499) |
|
||||
| `/image` command spec | `internal/domain/messaging/commands.go` | new entry in `localCommandSpecs` |
|
||||
| `/image` routing | `internal/gateway/gateway.go` (`Router.Images`, `handleImage`) | new (#500) |
|
||||
| autonomous tool | `internal/tools/image/tool.go` | new (#501) |
|
||||
| tool registration | `internal/app/archied/bootstrap.go` (`registerImageTool`, beside `registerMinimaxTool`) | new (#501) |
|
||||
| `Registry` wiring | `internal/app/archied/bootstrap.go` (construct `image.Registry`, register configured providers, hand to `Router.Images`) | new — this is the "nothing to wire until a real provider exists" step the original contract doc deferred |
|
||||
| Telegram delivery | `internal/channels/telegram/media.go` | none — already handles `"image"` |
|
||||
| Dashboard delivery | `internal/webui/api_chat.go` | none in this epic — tracked separately (archie-core-1786748942243-6-f109697e) |
|
||||
| docs | `docs/` user-facing guide (#502) | new, against this design |
|
||||
Reference in new issue
Block a user