MCP npm servers re-fetched from npm on every task container start #1535

Closed
opened 2026-10-27 13:40:47 +00:00 by sam · 0 comments
Owner

Problem

d4bb914b/#64 fixed the daemon-side case: cmd/archied/main.go's
configuredMCPProvider now points npx-launched MCP servers at a persistent
NPM_CONFIG_CACHE under the daemon's own work_dir, so a daemon restart
reuses a warm cache instead of re-resolving/re-downloading from the npm
registry.

internal/app/agentworker/mcp_providers.go builds the identical
StdioTransportConfig for archie-agent's own per-task MCP servers, but was
deliberately left untouched -- it's not a separate code path, the exact
same cfg.Tools.MCPServers list (including any npx entries) is threaded
through to every per-task NATS agent worker (bootstrap.go -> NATSRunner.MCPServers
-> taskrun.Request -> task_execution.go's startMCPProviders). Since each
task runs in a fresh ephemeral container, an npx-based MCP server
configured for task use re-resolves/re-downloads on every task, not
just every daemon restart -- more frequent, and thus a worse amplifier of
a registry outage than the bug already fixed.

Why this needs its own issue

Unlike the daemon case, a bare NPM_CONFIG_CACHE= pointed at a host
directory is meaningless inside an ephemeral container unless a
persistent Docker volume is actually bind-mounted at that path. No such
volume currently exists for this purpose -- internal/container/pool.go's
mounts are caller-provided, and nothing wires an npm-cache volume in.
That's real new infrastructure (a shared, persistent volume across
per-task containers), not a parameter-threading fix like d4bb914b was.

Acceptance criteria

  • Task containers reuse a persistent npm cache across tasks, the same
    way the daemon now does across restarts.
  • The cache volume/mount is wired through internal/container/pool.go's
    Acquire (or equivalent) without weakening per-task filesystem isolation
    for anything other than the npm cache path itself.
  • Startup time for a task's npx-based MCP server no longer scales with
    package size once the cache is warm.

Close reason

Fixed: MCP npm servers for per-task containers now reuse a persistent npm cache. Added mcp.NpmCacheEnv (shared with the daemon-side fix), an always-on archie-cache-mcp-npm Docker volume mounted at /data/cache/mcp-npm on every task container regardless of ecosystem, and wired it into internal/app/agentworker/mcp_providers.go's stdio transport construction. Adversarial review found zero surviving findings. [Closure restored 2026-08-24: originally closed 2026-08-22, lost when the beads DB was rebuilt (.beads.backup-pre-recovery).]


Imported from beads archie-core-qk8m · GitHub #532 · type bug · priority P2 · created 2026-08-18 · closed 2026-08-24

## Problem d4bb914b/#64 fixed the daemon-side case: cmd/archied/main.go's configuredMCPProvider now points npx-launched MCP servers at a persistent NPM_CONFIG_CACHE under the daemon's own work_dir, so a daemon restart reuses a warm cache instead of re-resolving/re-downloading from the npm registry. internal/app/agentworker/mcp_providers.go builds the identical StdioTransportConfig for archie-agent's own per-task MCP servers, but was deliberately left untouched -- it's not a separate code path, the exact same cfg.Tools.MCPServers list (including any npx entries) is threaded through to every per-task NATS agent worker (bootstrap.go -> NATSRunner.MCPServers -> taskrun.Request -> task_execution.go's startMCPProviders). Since each task runs in a fresh ephemeral container, an npx-based MCP server configured for task use re-resolves/re-downloads on every task, not just every daemon restart -- more frequent, and thus a worse amplifier of a registry outage than the bug already fixed. ## Why this needs its own issue Unlike the daemon case, a bare NPM_CONFIG_CACHE=<path> pointed at a host directory is meaningless inside an ephemeral container unless a persistent Docker volume is actually bind-mounted at that path. No such volume currently exists for this purpose -- internal/container/pool.go's mounts are caller-provided, and nothing wires an npm-cache volume in. That's real new infrastructure (a shared, persistent volume across per-task containers), not a parameter-threading fix like d4bb914b was. ## Acceptance criteria - Task containers reuse a persistent npm cache across tasks, the same way the daemon now does across restarts. - The cache volume/mount is wired through internal/container/pool.go's Acquire (or equivalent) without weakening per-task filesystem isolation for anything other than the npm cache path itself. - Startup time for a task's npx-based MCP server no longer scales with package size once the cache is warm. ### Close reason Fixed: MCP npm servers for per-task containers now reuse a persistent npm cache. Added mcp.NpmCacheEnv (shared with the daemon-side fix), an always-on archie-cache-mcp-npm Docker volume mounted at /data/cache/mcp-npm on every task container regardless of ecosystem, and wired it into internal/app/agentworker/mcp_providers.go's stdio transport construction. Adversarial review found zero surviving findings. [Closure restored 2026-08-24: originally closed 2026-08-22, lost when the beads DB was rebuilt (.beads.backup-pre-recovery).] --- _Imported from beads `archie-core-qk8m` · GitHub #532 · type bug · priority P2 · created 2026-08-18 · closed 2026-08-24_
sam added the mediumbug labels 2026-10-27 13:40:47 +00:00
sam self-assigned this 2026-10-27 13:40:47 +00:00
sam closed this issue 2026-10-27 13:40:47 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1535