refactor(indexing): delete the workspace codesearch indexer
The index built google/codesearch over the chat workspace on every daemon start, for the process lifetime, through a child-process helper and a SQLite metadata DB -- and grep consulted it only for searches confined to the workspace root, which the analysed sessions showed was a small fraction of searches. The compute was continuous; the benefit was not. Removes internal/indexing, the workspace-codesearch helper subcommand, the [indexing] config section, and the GrepIndex accelerator seam from the grep tool, which returns to its direct walk. google/codesearch leaves go.mod with it.
This commit is contained in:
25 files changed
+139
-1463
No files matched your search
+9
-1
@@ -350,6 +350,10 @@ tasks:
|
||||
archie-gateway and archie-ui dial it -- none owns
|
||||
the DB in-process
|
||||
(docs/prds/state-store-contract.md §12 step 7)
|
||||
* state admin read-only PocketBase surface over archie.db on
|
||||
127.0.0.1:8686 (view collections only; the gRPC
|
||||
contract stays the only writer). Served by the
|
||||
archie-state-store process, not a sixth one.
|
||||
* archie-ui serves the dashboard on 127.0.0.1:8484 from the
|
||||
embedded dist (deployments/dev.toml). After the
|
||||
UI cutover (archie-core-8cda.5.4) archied runs no
|
||||
@@ -380,8 +384,12 @@ tasks:
|
||||
|
||||
dev:state-store:
|
||||
desc: archie-state-store only, no live-reload (prefer `task dev`)
|
||||
# -admin-addr serves the read-only operator dashboard over the same
|
||||
# archie.db this process already owns. It is a flag rather than a config
|
||||
# key for the same reason -ready-addr is: the address is a property of
|
||||
# how this process was launched, not of the deployment's config document.
|
||||
cmds:
|
||||
- go run ./cmd/archie-state-store -config-overlay deployments/dev.toml
|
||||
- go run ./cmd/archie-state-store -config-overlay deployments/dev.toml -admin-addr 127.0.0.1:8686
|
||||
|
||||
dev:gateway:
|
||||
desc: archie-gateway only, no live-reload (prefer `task dev`)
|
||||
|
||||
@@ -15,11 +15,5 @@ func main() {
|
||||
if archied.IsSetupArgs(args) {
|
||||
os.Exit(archied.RunSetup(args[1:], os.Stdin, os.Stdout, os.Stderr))
|
||||
}
|
||||
// The codesearch helper is dispatched before the daemon's flags are
|
||||
// parsed: it is this same binary re-invoked as a short-lived child by
|
||||
// internal/indexing, and it must not touch config, the store or NATS.
|
||||
if archied.IsCodesearchHelperArgs(args) {
|
||||
os.Exit(archied.RunCodesearchHelper(args[1:], os.Stdout))
|
||||
}
|
||||
os.Exit(archied.Run())
|
||||
}
|
||||
@@ -227,13 +227,6 @@ class = "ollama"
|
||||
# work_dir = "/path/to/workspace"
|
||||
# parallel_tool_calls = false
|
||||
|
||||
# Workspace code-search index. Paths default under work_dir when unset.
|
||||
[indexing]
|
||||
# Code-search index files (default: <work_dir>/indexes).
|
||||
# index_dir = "/var/lib/archie/indexes"
|
||||
# Per-index lifecycle metadata DB (default: <work_dir>/workspace-indexes.db).
|
||||
# db_path = "/var/lib/archie/workspace-indexes.db"
|
||||
|
||||
# archied's own liveness surface, always on. GET /healthz and /health answer
|
||||
# 200 once the daemon is serving (503 while it is still booting), and
|
||||
# /health/detailed reports the readiness probes. This is what the update
|
||||
|
||||
@@ -124,3 +124,21 @@ indicates dual-store ownership (`docs/prds/state-store-contract.md` §12 step 7/
|
||||
The readiness probe is the single-writer check the runbook has against a
|
||||
degraded store: a `503` from `/health/detailed` means `archied` should be
|
||||
started only after the store is healthy.
|
||||
|
||||
### Inspecting the store
|
||||
|
||||
`-admin-addr` starts a read-only operator dashboard on the same process, over
|
||||
the file it already owns:
|
||||
|
||||
```bash
|
||||
archie-state-store -admin-addr 127.0.0.1:8686 # dashboard on http://127.0.0.1:8686/_/
|
||||
```
|
||||
|
||||
It co-tenants on `<db_path>-tasks.sqlite` rather than opening a second
|
||||
database, so it does not disturb the single-writer rule above: it runs inside
|
||||
the process that holds the ownership claim, and every archie table is exposed
|
||||
as a read-only view collection. The routes that could write past those views
|
||||
(`POST /api/sql`, `/api/backups`) are refused, so the gRPC contract stays the
|
||||
only writer and `archie-state-store backup` stays the only snapshot path.
|
||||
|
||||
Leave the flag off and no dashboard listener is started.
|
||||
@@ -58,7 +58,7 @@ survive as a shared package, configuration domain, or renamed global model.
|
||||
Its loading behaviour moves to `internal/infrastructure/configuration`. Existing
|
||||
types and methods are reassigned by behaviour. Repository execution policy,
|
||||
dispatch policy, retry policy, ecosystem defaults, tool policy, memory settings,
|
||||
indexing settings, provider settings, and channel settings move to their
|
||||
provider settings, and channel settings move to their
|
||||
respective owners.
|
||||
|
||||
## Repository settings
|
||||
@@ -163,8 +163,7 @@ existing structure combines different owners.
|
||||
| webhook target | `Notify` | Notification adapter instance settings. |
|
||||
| web listen address | `Web` | Web transport settings. |
|
||||
| `listen` under `[services.state]` and `[services.gateway]` | `ServiceConnection` | The bind address of the process that owns the service. Read only by that process, and defaulted to the address its clients dial so the pair cannot drift. |
|
||||
| index directory and database path | `Indexing` | Index infrastructure settings, derived from storage settings unless explicitly overridden. |
|
||||
| forge, dispatch, models, providers, containers, memory, tools, indexing and repositories | `IdentityConfig` | Existing composite duplicates unrelated settings. Replace it with associations from an `IdentityID` to independently owned configured instances. |
|
||||
| forge, dispatch, models, providers, containers, memory, tools and repositories | `IdentityConfig` | Existing composite duplicates unrelated settings. Replace it with associations from an `IdentityID` to independently owned configured instances. |
|
||||
| owner, name, base, gate, protection, ecosystem, preflight, storage, retry and concurrency | `Repo` | Split across work intake, workspace, workflow execution, policy, gate, and scheduling as described in Repository settings. |
|
||||
| `TaskConfig`, `TaskForge` | worker snapshot | Replace with the minimum immutable, versioned WorkflowExecution input assembled by the application. It is not a second configuration model. |
|
||||
| `Extra` and arbitrary maps | root/provider config | Allowed only at an extension boundary whose owning plugin/provider supplies schema and validation. Core configuration never consumes an untyped catch-all. |
|
||||
@@ -187,7 +186,6 @@ decision; exact setting names are implementation details.
|
||||
| NATS worker consumer | stream `ARCHIE_TASKS`, consumer `archie-agent`, deduplication `2m`, poll `5s`, acknowledgement wait `30m`, deliveries `3`, inactive threshold `24h`, queue group and retry delay `1s` | Worker transport settings and delivery/retry policy. Shared stream defaults must have one owner rather than two copies. |
|
||||
| Worker readiness | readiness timeout `20s`, retry delay `250ms` | Worker lifecycle and retry policy. |
|
||||
| Execution/RPC | agent execution `30m`, task-run RPC `60s`, worktree RPC `15m` | Per-operation timeout policy. |
|
||||
| Indexing | build timeout `10m`, metadata timeout `5s` | Index operation policy. |
|
||||
| Plugin/tool cleanup | plugin rollback `10s`, tool-provider cleanup `10s`, MCP-provider cleanup `10s` | Lifecycle shutdown/rollback policy owned by each engine, with a shared policy vocabulary where appropriate. |
|
||||
| Memory read path | per-scope record limit `20`, rendered `<memory>` block byte cap `8192` | Memory read policy owned by the engine family (`internal/domain/memory`). |
|
||||
| Built-in memory store | per-file size limit `100KiB`, query result limit `100` | Split into memory resource policy, input policy, and presentation settings. |
|
||||
|
||||
@@ -9,27 +9,36 @@ require (
|
||||
github.com/ProtonMail/go-crypto v1.4.1 // indirect
|
||||
github.com/antithesishq/antithesis-sdk-go v0.8.0 // indirect
|
||||
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/cloudflare/circl v1.6.5 // indirect
|
||||
github.com/containerd/errdefs v1.0.0 // indirect
|
||||
github.com/containerd/errdefs/pkg v0.3.0 // indirect
|
||||
github.com/davidmz/go-pageant v1.0.2 // indirect
|
||||
github.com/disintegration/imaging v1.6.2 // indirect
|
||||
github.com/distribution/reference v0.6.0 // indirect
|
||||
github.com/docker/go-connections v0.8.1 // indirect
|
||||
github.com/docker/go-units v0.5.0 // indirect
|
||||
github.com/domodwyer/mailyak/v3 v3.6.2 // indirect
|
||||
github.com/dustin/go-humanize v1.1.0 // indirect
|
||||
github.com/emirpasic/gods v1.18.1 // indirect
|
||||
github.com/fatih/color v1.19.0 // indirect
|
||||
github.com/felixge/httpsnoop v1.1.0 // indirect
|
||||
github.com/fsnotify/fsnotify v1.10.1 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.15 // indirect
|
||||
github.com/ganigeorgiev/fexpr v0.6.0 // indirect
|
||||
github.com/go-fed/httpsig v1.1.0 // indirect
|
||||
github.com/go-git/gcfg/v2 v2.0.2 // indirect
|
||||
github.com/go-logr/logr v1.4.4 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
|
||||
github.com/google/go-querystring v1.2.0 // indirect
|
||||
github.com/google/go-tpm v0.9.8 // indirect
|
||||
github.com/hashicorp/go-version v1.9.0 // indirect
|
||||
github.com/kevinburke/ssh_config v1.6.0 // indirect
|
||||
github.com/klauspost/compress v1.20.0 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.15 // indirect
|
||||
github.com/mattn/go-isatty v0.0.24 // indirect
|
||||
github.com/minio/highwayhash v1.0.4 // indirect
|
||||
github.com/moby/docker-image-spec v1.3.1 // indirect
|
||||
@@ -41,8 +50,12 @@ require (
|
||||
github.com/opencontainers/image-spec v1.1.1 // indirect
|
||||
github.com/pjbgf/sha1cd v0.6.0 // indirect
|
||||
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
|
||||
github.com/pocketbase/ozzo-validation/v4 v4.3.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
github.com/sergi/go-diff v1.4.0 // indirect
|
||||
github.com/spf13/cast v1.10.0 // indirect
|
||||
github.com/spf13/cobra v1.10.2 // indirect
|
||||
github.com/spf13/pflag v1.0.10 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.71.0 // indirect
|
||||
go.opentelemetry.io/otel v1.46.0 // indirect
|
||||
@@ -51,6 +64,7 @@ require (
|
||||
go.yaml.in/yaml/v3 v3.0.5 // indirect
|
||||
golang.org/x/crypto v0.57.0 // indirect
|
||||
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba // indirect
|
||||
golang.org/x/image v0.46.0 // indirect
|
||||
golang.org/x/sync v0.23.0 // indirect
|
||||
golang.org/x/sys v0.48.0 // indirect
|
||||
golang.org/x/text v0.42.0 // indirect
|
||||
@@ -71,13 +85,14 @@ require (
|
||||
github.com/go-git/go-git/v6 v6.0.0-alpha.5
|
||||
github.com/go-jose/go-jose/v4 v4.1.5
|
||||
github.com/go-telegram/bot v1.27.0
|
||||
github.com/google/codesearch v1.3.0-rc.1
|
||||
github.com/google/go-github/v78 v78.0.0
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/moby/moby/api v1.56.0
|
||||
github.com/moby/moby/client v0.6.0
|
||||
github.com/nats-io/nats-server/v2 v2.15.0
|
||||
github.com/nats-io/nats.go v1.54.0
|
||||
github.com/pocketbase/dbx v1.12.0
|
||||
github.com/pocketbase/pocketbase v0.40.4
|
||||
github.com/samcharles93/ai-sdk v0.1.39
|
||||
github.com/traefik/yaegi v0.16.1
|
||||
golang.org/x/net v0.59.0
|
||||
|
||||
@@ -25,6 +25,9 @@ github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYW
|
||||
github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
|
||||
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
|
||||
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
|
||||
github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:oGkLhpf+kjZl6xBf758TQhh5XrAeiJv/7FRz/2spLIg=
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
|
||||
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
|
||||
github.com/bwesterb/go-ristretto v1.2.4/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
@@ -38,17 +41,22 @@ github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmC
|
||||
github.com/containerd/typeurl/v2 v2.2.0/go.mod h1:8XOOxnyatxSWuG8OfsZXVnAF4iZfedjS/8UHSPJnX4g=
|
||||
github.com/coreos/go-oidc/v3 v3.21.0 h1:wZo4Q9Pum8dYEj0eMUPrqR+kvuGkeUplbLpNCkBqoWM=
|
||||
github.com/coreos/go-oidc/v3 v3.21.0/go.mod h1:DYCf24+ncYi+XkIH97GY1+dqoRlbaSI26KVTCI9SrY4=
|
||||
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
|
||||
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davidmz/go-pageant v1.0.2 h1:bPblRCh5jGU+Uptpz6LgMZGD5hJoOt7otgT454WvHn0=
|
||||
github.com/davidmz/go-pageant v1.0.2/go.mod h1:P2EDDnMqIwG5Rrp05dTRITj9z2zpGcD9efWSkTNKLIE=
|
||||
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
|
||||
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
|
||||
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
|
||||
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
|
||||
github.com/docker/go-connections v0.8.1 h1:JibmG5hULs5qXSr/cp/w3Pw5fZuStt4MOHMUExb29/M=
|
||||
github.com/docker/go-connections v0.8.1/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
|
||||
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
|
||||
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
|
||||
github.com/domodwyer/mailyak/v3 v3.6.2 h1:x3tGMsyFhTCaxp6ycgR0FE/bu5QiNp+hetUuCOBXMn8=
|
||||
github.com/domodwyer/mailyak/v3 v3.6.2/go.mod h1:lOm/u9CyCVWHeaAmHIdF4RiKVxKUT/H5XX10lIKAL6c=
|
||||
github.com/dustin/go-humanize v1.1.0 h1:dbKTrvD0klcbBV/h4AWJdMuZogJACoMlvWIWZ5b2xWg=
|
||||
github.com/dustin/go-humanize v1.1.0/go.mod h1:hc1CvRkJMsgxqjmjMQF3QNRAZBwY8AXBAzKYoSX9sFI=
|
||||
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
|
||||
@@ -57,8 +65,16 @@ github.com/envoyproxy/go-control-plane v0.14.0/go.mod h1:NcS5X47pLl/hfqxU70yPwL9
|
||||
github.com/envoyproxy/go-control-plane/envoy v1.39.0/go.mod h1:5e4ylfTZO723MEEFsCpSW4ZEBWR8mwkEyXfwJBTCZ9c=
|
||||
github.com/envoyproxy/go-control-plane/ratelimit v0.1.0/go.mod h1:Wk+tMFAFbCXaJPzVVHnPgRKdUdwW/KdbRt94AzgRee4=
|
||||
github.com/envoyproxy/protoc-gen-validate v1.3.3/go.mod h1:TsndJ/ngyIdQRhMcVVGDDHINPLWB7C82oDArY51KfB0=
|
||||
github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
|
||||
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
|
||||
github.com/felixge/httpsnoop v1.1.0 h1:3YtUj32ZZkqZtt3sZZsClsymw/QDuVfpNhoA31zeORc=
|
||||
github.com/felixge/httpsnoop v1.1.0/go.mod h1:Zqxgdd+1Rkcz8euOqdr7lqgCRJztwr5hp9vDSi5UZCE=
|
||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
|
||||
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
|
||||
github.com/ganigeorgiev/fexpr v0.6.0 h1:Fza3O/QMBKEudUvxV862qe6GjxM60GJjjKytdp+VQus=
|
||||
github.com/ganigeorgiev/fexpr v0.6.0/go.mod h1:RyGiGqmeXhEQ6+mlGdnUleLHgtzzu/VGO2WtJkF5drE=
|
||||
github.com/gliderlabs/ssh v0.3.8 h1:a4YXD1V7xMF9g5nTkdfnja3Sxy1PVDCj1Zg4Wb8vY6c=
|
||||
github.com/gliderlabs/ssh v0.3.8/go.mod h1:xYoytBv1sV0aL3CavoDuJIQNURXkkfPA/wxQ1pL1fAU=
|
||||
github.com/go-fed/httpsig v1.1.0 h1:9M+hb0jkEICD8/cAiNqEB66R87tTINszBRTjwjQzWcI=
|
||||
@@ -81,14 +97,16 @@ github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ4
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow=
|
||||
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
|
||||
github.com/go-telegram/bot v1.27.0 h1:PG9yBDzM8O26oUerE9SZwWlHon4RpH8WcNslZ7CEZmE=
|
||||
github.com/go-telegram/bot v1.27.0/go.mod h1:i2TRs7fXWIeaceF3z7KzsMt/he0TwkVC680mvdTFYeM=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/golang/glog v1.2.5/go.mod h1:6AhwSGph0fcJtXVM/PEHPqZlFeoLxhs7/t5UDAwmO+w=
|
||||
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/codesearch v1.3.0-rc.1 h1:vSFGuF5wS3sjimuLzqPG+PGjWQ9VvEtPFbxdZD7YuuQ=
|
||||
github.com/google/codesearch v1.3.0-rc.1/go.mod h1:hKdpx7wOsvk2WLT/D6NLPxFeLT8jKCehZu21JpA2rIY=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
@@ -110,6 +128,7 @@ github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaX
|
||||
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
||||
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
|
||||
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
|
||||
github.com/kevinburke/ssh_config v1.6.0 h1:J1FBfmuVosPHf5GRdltRLhPJtJpTlMdKTBjRgTaQBFY=
|
||||
github.com/kevinburke/ssh_config v1.6.0/go.mod h1:q2RIzfka+BXARoNexmF9gkxEX7DmvbW9P4hIVx2Kg4M=
|
||||
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
|
||||
@@ -123,6 +142,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
|
||||
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
|
||||
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
|
||||
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
|
||||
@@ -156,17 +177,31 @@ github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmd
|
||||
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
|
||||
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10/go.mod h1:t/avpk3KcrXxUnYOhZhMXJlSEyie6gQbtLq5NM3loB8=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/pocketbase/dbx v1.12.0 h1:/oLErM+A0b4xI0PWTGPqSDVjzix48PqI/bng2l0PzoA=
|
||||
github.com/pocketbase/dbx v1.12.0/go.mod h1:xXRCIAKTHMgUCyCKZm55pUOdvFziJjQfXaWKhu2vhMs=
|
||||
github.com/pocketbase/ozzo-validation/v4 v4.3.0 h1:uKBDVma7bZqgR2a6AwE+k9hkuDFfiZMpBHQdZ1z3iQs=
|
||||
github.com/pocketbase/ozzo-validation/v4 v4.3.0/go.mod h1:6XNjSTw/Jb2F8LOkKO3oyzIWExbrGiYoS4uVxVwz90g=
|
||||
github.com/pocketbase/pocketbase v0.40.4 h1:0SvSUreR3NhUMCs9LchE59oEG53efZ3cKiMGyAGBN9U=
|
||||
github.com/pocketbase/pocketbase v0.40.4/go.mod h1:2mU+80FLiY1fb13WZRg8Xx/lKg4nTjgKVJMigyRH6k0=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
||||
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||
github.com/russross/blackfriday v1.6.0/go.mod h1:ti0ldHuxg49ri4ksnFxlkCfN+hvslNlmVHqNRXXJNAY=
|
||||
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
|
||||
github.com/samcharles93/ai-sdk v0.1.39 h1:gbNPYeA27MifReCruBiqMgb4fFSjCDi1Zv57a9bo0Vo=
|
||||
github.com/samcharles93/ai-sdk v0.1.39/go.mod h1:u4zrQ9/Q58L5mNEut4bNQ1ks+RCKq71LN5EDz2N/0Zk=
|
||||
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY=
|
||||
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
|
||||
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
|
||||
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
|
||||
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
|
||||
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
|
||||
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
|
||||
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
|
||||
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
github.com/spiffe/go-spiffe/v2 v2.8.1/go.mod h1:47Q0Q9/AqGha8QLHp+kxpH4Wca7X7EnOtlIJy3mxZ3U=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
|
||||
@@ -189,6 +224,7 @@ go.opentelemetry.io/otel/sdk/metric v1.46.0 h1:0piZ26EG4RBfebb2jhDH6ERCYHoVWduc3
|
||||
go.opentelemetry.io/otel/sdk/metric v1.46.0/go.mod h1:I1PbKrdVc8Qu8HYVDNtqVIwLwjNrhsV/uFuxfwg8mO4=
|
||||
go.opentelemetry.io/otel/trace v1.46.0 h1:OULy7ccdJnZtJ0UDYFOIGaCmiWzJ8Vi2G/Rsu60qs1c=
|
||||
go.opentelemetry.io/otel/trace v1.46.0/go.mod h1:J7GAXweO77XSFkB/rmAqk9D6ihszhFjLU+d9WuUxDLI=
|
||||
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
|
||||
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
|
||||
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
@@ -198,9 +234,13 @@ golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
|
||||
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
|
||||
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba h1:Ck8QetSgk912qxWLMCKxd0in+aiyBQyDSMae6e/xmpU=
|
||||
golang.org/x/exp v0.0.0-20260908205506-85c1c2202aba/go.mod h1:50RgIsmK7OwqzTTeqcSXQW8SswW0o8fRcDxmqGluJ8E=
|
||||
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
|
||||
golang.org/x/image v0.46.0 h1:b1+oYj0Jbp6K5MDT4i4/eZpYlk3V8SJhhDKh6LBHAyQ=
|
||||
golang.org/x/image v0.46.0/go.mod h1:3B3W05VGVQyuXucLINLjXKrqISASfi4Xj+iCVkLMwew=
|
||||
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
|
||||
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
|
||||
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
|
||||
@@ -221,6 +261,7 @@ golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9sn
|
||||
golang.org/x/term v0.46.0 h1:3+OXuTbaKDgwk8jTi3aSLHRlmWqHEUDUtxnbFigO4YE=
|
||||
golang.org/x/term v0.46.0/go.mod h1:+K02xbkittuwc0Am4abfA3Fc+XRGXkvBXNO88NCXPoc=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
|
||||
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
|
||||
@@ -234,6 +275,7 @@ golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated/go.mod h1:RVAQXBGN
|
||||
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
|
||||
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
|
||||
google.golang.org/api v0.293.0/go.mod h1:6n5tjEB1gzwniZTepZ0g5u+wM7Bof5GeULCx/zh8ZE0=
|
||||
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260918162117-cecb64721679 h1:FEp7JNE32DTAwbnI/ixagnmj7Xm1eTONofGEUXFjZ4w=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260918162117-cecb64721679/go.mod h1:52bV8FLAQ9Qmcqaq9ECLmuEHZthk+6OPV45aKBBrsNw=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260921155816-b14227669459 h1:GS9OIt/j7c8bvBjYNgnKQysVfmV7e4jM0H8ZK95G4t8=
|
||||
|
||||
@@ -45,7 +45,6 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/forge"
|
||||
forgewebhook "github.com/samcharles93/archie-core/internal/forge/webhook"
|
||||
"github.com/samcharles93/archie-core/internal/gateway"
|
||||
"github.com/samcharles93/archie-core/internal/indexing"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/configuration"
|
||||
infraembedding "github.com/samcharles93/archie-core/internal/infrastructure/embedding"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/eventbus/nats"
|
||||
@@ -65,7 +64,6 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/skill"
|
||||
"github.com/samcharles93/archie-core/internal/storage"
|
||||
"github.com/samcharles93/archie-core/internal/tools"
|
||||
toolsbuiltin "github.com/samcharles93/archie-core/internal/tools/builtin"
|
||||
"github.com/samcharles93/archie-core/internal/tools/minimax"
|
||||
toolprovider "github.com/samcharles93/archie-core/internal/tools/provider"
|
||||
builtintoolprovider "github.com/samcharles93/archie-core/internal/tools/provider/builtin"
|
||||
@@ -1115,37 +1113,6 @@ func (b *boot) setupGuardrails() {
|
||||
)
|
||||
}
|
||||
|
||||
// workspaceIndex builds the optional codesearch index that narrows grep's
|
||||
// candidate files. It returns a slice so a failure is simply "no index"
|
||||
// rather than a typed-nil reaching the provider.
|
||||
//
|
||||
// Every failure degrades instead of propagating: grep falls back to its
|
||||
// direct walk, which is what it did before this was wired at all. An
|
||||
// accelerator that can refuse to build must never stop the daemon starting.
|
||||
//
|
||||
// The context is detached from cancellation but keeps its values, because
|
||||
// NewManager starts an asynchronous build whose lifetime is the process
|
||||
// while the gateway path's boot context is cancelled once startup returns.
|
||||
// Inheriting cancellation would kill the build mid-flight and leave grep
|
||||
// permanently unindexed with nothing to show why; using Background instead
|
||||
// would silently drop any values the caller's context carries.
|
||||
func (b *boot) workspaceIndex(ctx context.Context, workspace string) []toolsbuiltin.GrepIndex {
|
||||
cfg, log := b.cfg, b.log
|
||||
idxCfg := cfg.Indexing.WithDefaults(cfg.WorkDir)
|
||||
manager, err := indexing.NewManager(
|
||||
context.WithoutCancel(ctx),
|
||||
indexing.Config{IndexDir: idxCfg.IndexDir, DBPath: idxCfg.DBPath, Log: log},
|
||||
workspace,
|
||||
)
|
||||
if err != nil {
|
||||
log.Warn("workspace index unavailable; grep will search unindexed",
|
||||
"workspace", workspace, "index_dir", idxCfg.IndexDir, "error", err)
|
||||
return nil
|
||||
}
|
||||
log.Info("workspace index enabled", "workspace", workspace, "index_dir", idxCfg.IndexDir)
|
||||
return []toolsbuiltin.GrepIndex{manager}
|
||||
}
|
||||
|
||||
// registerTools registers the tool providers with a lifecycle: workspace
|
||||
// file/shell tools and optional MCP servers. Memory tools
|
||||
// (memory_create/update/delete/list) are not registered here -- they are
|
||||
@@ -1159,8 +1126,7 @@ func (b *boot) registerTools(ctx context.Context) error {
|
||||
// deliberate choice rather than a default.
|
||||
if workspace := cfg.Chat.Workspace; workspace != "" {
|
||||
unrestricted := cfg.Chat.UnrestrictedFilesystem
|
||||
index := b.workspaceIndex(ctx, workspace)
|
||||
if err := b.providerRegistry.Register(builtintoolprovider.New(workspace, unrestricted, index...)); err != nil {
|
||||
if err := b.providerRegistry.Register(builtintoolprovider.New(workspace, unrestricted)); err != nil {
|
||||
log.Error("workspace tool provider registration failed", "err", err)
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -1,121 +0,0 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/indexing"
|
||||
)
|
||||
|
||||
// flagErrorOutput receives helper diagnostics. It is a variable so tests can
|
||||
// capture them, and it is deliberately not stdout: the parent parses stdout
|
||||
// as JSON, so a stray usage message there would corrupt the result.
|
||||
var flagErrorOutput io.Writer = os.Stderr
|
||||
|
||||
// codesearchHelperCommand is the argv[1] token that switches archied out of
|
||||
// daemon mode and into the codesearch helper.
|
||||
//
|
||||
// indexing.Manager deliberately runs the google/codesearch library in a child
|
||||
// process: upstream calls fatal exits on a corrupt index and leaks mmapped
|
||||
// regions, neither of which may be allowed to take down a long-lived daemon.
|
||||
// The child it spawns is this binary, so this name is a contract with
|
||||
// internal/indexing -- renaming either side silently disables indexed search
|
||||
// and falls the agent back to unindexed grep.
|
||||
const codesearchHelperCommand = "workspace-codesearch"
|
||||
|
||||
// IsCodesearchHelperArgs reports whether args (os.Args[1:]) selects the
|
||||
// helper rather than the daemon.
|
||||
func IsCodesearchHelperArgs(args []string) bool {
|
||||
return len(args) > 0 && args[0] == codesearchHelperCommand
|
||||
}
|
||||
|
||||
// RunCodesearchHelper executes one helper subcommand and returns a process
|
||||
// exit code. Output is the machine-readable result the parent decodes; all
|
||||
// diagnostics go to stderr so they cannot corrupt it.
|
||||
func RunCodesearchHelper(args []string, out io.Writer) int {
|
||||
if len(args) == 0 {
|
||||
helperErrorf("%s: expected a subcommand (build|candidates)\n", codesearchHelperCommand)
|
||||
return 2
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
switch args[0] {
|
||||
case "build":
|
||||
fs := newHelperFlagSet("build")
|
||||
root := fs.String("root", "", "workspace root to index")
|
||||
index := fs.String("index", "", "path of the index sidecar to write")
|
||||
if err := fs.Parse(args[1:]); err != nil {
|
||||
return 2
|
||||
}
|
||||
if *root == "" || *index == "" {
|
||||
helperErrorf("build: --root and --index are required\n")
|
||||
return 2
|
||||
}
|
||||
if err := indexing.BuildIndex(ctx, *root, *index); err != nil {
|
||||
helperErrorf("build: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
|
||||
case "candidates":
|
||||
return runCandidates(args[1:], out)
|
||||
|
||||
default:
|
||||
helperErrorf("%s: unknown subcommand %q\n", codesearchHelperCommand, args[0])
|
||||
return 2
|
||||
}
|
||||
}
|
||||
|
||||
func newHelperFlagSet(name string) *flag.FlagSet {
|
||||
fs := flag.NewFlagSet(name, flag.ContinueOnError)
|
||||
fs.SetOutput(flagErrorOutput)
|
||||
return fs
|
||||
}
|
||||
|
||||
// runCandidates handles the "candidates" subcommand, returning candidate
|
||||
// file paths as a JSON array on stdout.
|
||||
func runCandidates(args []string, out io.Writer) int {
|
||||
fs := newHelperFlagSet("candidates")
|
||||
index := fs.String("index", "", "path of the index sidecar to query")
|
||||
pattern := fs.String("pattern", "", "search pattern")
|
||||
literal := fs.Bool("literal", false, "treat the pattern as a literal string")
|
||||
caseSensitive := fs.Bool("case-sensitive", false, "match case sensitively")
|
||||
if err := fs.Parse(args); err != nil {
|
||||
return 2
|
||||
}
|
||||
if *index == "" || *pattern == "" {
|
||||
helperErrorf("candidates: --index and --pattern are required\n")
|
||||
return 2
|
||||
}
|
||||
files, err := indexing.IndexCandidates(*index, *pattern, *literal, *caseSensitive)
|
||||
if err != nil {
|
||||
helperErrorf("candidates: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
// The parent decodes a JSON array and treats any decode failure as
|
||||
// "no index", so a nil slice must still marshal as [] rather than null.
|
||||
if files == nil {
|
||||
files = []string{}
|
||||
}
|
||||
encoded, err := json.Marshal(files)
|
||||
if err != nil {
|
||||
helperErrorf("candidates: encode result: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
if _, err := out.Write(encoded); err != nil {
|
||||
helperErrorf("candidates: write result: %v\n", err)
|
||||
return 1
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// helperErrorf writes a diagnostic to stderr. The write result is ignored
|
||||
// deliberately: if stderr is gone there is nowhere left to report it, and the
|
||||
// exit code already carries the outcome the parent acts on.
|
||||
func helperErrorf(format string, args ...any) {
|
||||
_, _ = fmt.Fprintf(flagErrorOutput, format, args...)
|
||||
}
|
||||
@@ -1,85 +0,0 @@
|
||||
package archied
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRunCodesearchHelper(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "a.txt"), []byte("alpha needle omega\n"), 0o600); err != nil {
|
||||
t.Fatalf("write fixture: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "b.txt"), []byte("nothing here\n"), 0o600); err != nil {
|
||||
t.Fatalf("write fixture: %v", err)
|
||||
}
|
||||
indexPath := filepath.Join(t.TempDir(), "workspace.csearch")
|
||||
|
||||
t.Run("build writes an index sidecar", func(t *testing.T) {
|
||||
var out strings.Builder
|
||||
code := RunCodesearchHelper([]string{"build", "--root", root, "--index", indexPath}, &out)
|
||||
if code != 0 {
|
||||
t.Fatalf("build exit code = %d, want 0", code)
|
||||
}
|
||||
if _, err := os.Stat(indexPath); err != nil {
|
||||
t.Fatalf("index sidecar not written: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("candidates emits the JSON contract the manager decodes", func(t *testing.T) {
|
||||
var out strings.Builder
|
||||
code := RunCodesearchHelper([]string{
|
||||
"candidates", "--index", indexPath, "--pattern", "needle",
|
||||
}, &out)
|
||||
if code != 0 {
|
||||
t.Fatalf("candidates exit code = %d, want 0", code)
|
||||
}
|
||||
var files []string
|
||||
if err := json.Unmarshal([]byte(out.String()), &files); err != nil {
|
||||
t.Fatalf("decode candidates output %q: %v", out.String(), err)
|
||||
}
|
||||
if !slices.ContainsFunc(files, func(p string) bool {
|
||||
return strings.HasSuffix(p, "a.txt")
|
||||
}) {
|
||||
t.Errorf("candidates = %v, want the file containing the needle", files)
|
||||
}
|
||||
})
|
||||
|
||||
invalid := []struct {
|
||||
name string
|
||||
args []string
|
||||
}{
|
||||
{name: "no subcommand", args: nil},
|
||||
{name: "unknown subcommand", args: []string{"frobnicate"}},
|
||||
{name: "build without root", args: []string{"build", "--index", indexPath}},
|
||||
{name: "build without index", args: []string{"build", "--root", root}},
|
||||
{name: "candidates without index", args: []string{"candidates", "--pattern", "x"}},
|
||||
{name: "candidates without pattern", args: []string{"candidates", "--index", indexPath}},
|
||||
}
|
||||
for _, tc := range invalid {
|
||||
t.Run(tc.name+" is rejected", func(t *testing.T) {
|
||||
var out strings.Builder
|
||||
if code := RunCodesearchHelper(tc.args, &out); code == 0 {
|
||||
t.Errorf("exit code = 0, want non-zero for %v", tc.args)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The manager shells out to argv[0] with this exact subcommand name, so a
|
||||
// rename on either side silently disables indexed search.
|
||||
func TestCodesearchHelperSubcommandIsRecognised(t *testing.T) {
|
||||
if !IsCodesearchHelperArgs([]string{codesearchHelperCommand, "build"}) {
|
||||
t.Errorf("IsCodesearchHelperArgs() did not recognise %q", codesearchHelperCommand)
|
||||
}
|
||||
if IsCodesearchHelperArgs([]string{"-once"}) {
|
||||
t.Error("IsCodesearchHelperArgs() claimed a normal daemon flag")
|
||||
}
|
||||
if IsCodesearchHelperArgs(nil) {
|
||||
t.Error("IsCodesearchHelperArgs() claimed an empty argument list")
|
||||
}
|
||||
}
|
||||
@@ -533,8 +533,6 @@ type Config struct {
|
||||
|
||||
// Tools holds MCP server and tool policy configuration (from config.tools.yaml).
|
||||
Tools ToolsConfig `toml:"tools" yaml:"tools"`
|
||||
// Indexing locates the workspace codesearch index.
|
||||
Indexing IndexingConfig `toml:"indexing" yaml:"indexing"`
|
||||
|
||||
// Identities declares multi-identity configurations. When non-empty,
|
||||
// each identity runs its own poll loop with its own forge client,
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
package config
|
||||
|
||||
import "path/filepath"
|
||||
|
||||
// IndexingConfig locates the workspace codesearch index.
|
||||
//
|
||||
// The index is a large, fully rebuildable artifact, so both paths are
|
||||
// settable to allow putting it on different storage from the task work tree.
|
||||
type IndexingConfig struct {
|
||||
// IndexDir holds the codesearch sidecar files.
|
||||
IndexDir string `toml:"index_dir" yaml:"index_dir" json:"index_dir,omitempty"`
|
||||
// DBPath is the SQLite database holding index lifecycle metadata.
|
||||
DBPath string `toml:"db_path" yaml:"db_path" json:"db_path,omitempty"`
|
||||
}
|
||||
|
||||
// WithDefaults fills unset paths from the daemon's work directory.
|
||||
//
|
||||
// Deriving from workDir rather than a shared data home is deliberate: two
|
||||
// archied instances run on one host, each with its own work_dir, so this
|
||||
// keeps their indexes separate without either having to configure anything.
|
||||
func (c IndexingConfig) WithDefaults(workDir string) IndexingConfig {
|
||||
if c.IndexDir == "" {
|
||||
c.IndexDir = filepath.Join(workDir, "indexes")
|
||||
}
|
||||
if c.DBPath == "" {
|
||||
c.DBPath = filepath.Join(workDir, "workspace-indexes.db")
|
||||
}
|
||||
return c
|
||||
}
|
||||
@@ -1,557 +0,0 @@
|
||||
// Package indexing provides language-neutral workspace text indexing.
|
||||
package indexing
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"regexp/syntax"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/google/codesearch/index"
|
||||
"github.com/google/uuid"
|
||||
_ "modernc.org/sqlite"
|
||||
)
|
||||
|
||||
const (
|
||||
// MaxCandidateFiles bounds helper output and grep argv construction.
|
||||
MaxCandidateFiles = 2000
|
||||
maxCandidateArgBytes = 24 * 1024
|
||||
buildTimeout = 10 * time.Minute
|
||||
metadataTimeout = 5 * time.Second
|
||||
)
|
||||
|
||||
// CandidateProvider returns files which may match a grep query. A false
|
||||
// handled result tells the caller to use its normal unindexed search path.
|
||||
type CandidateProvider interface {
|
||||
Candidates(context.Context, string, bool, bool) ([]string, bool)
|
||||
}
|
||||
|
||||
// Manager owns the workspace index lifecycle. The codesearch implementation
|
||||
// runs in a child process so upstream fatal exits and mmap lifetime cannot
|
||||
// terminate or leak inside the interactive agent process.
|
||||
type Manager struct {
|
||||
root string
|
||||
indexPath string
|
||||
dbPath string
|
||||
runner helperRunner
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// logger is nil-safe: a Manager may be constructed as a struct literal
|
||||
// (the tests do), so defaulting at the point of use rather than in the
|
||||
// constructor is what actually holds.
|
||||
func (m *Manager) logger() *slog.Logger {
|
||||
if m == nil || m.log == nil {
|
||||
return slog.Default()
|
||||
}
|
||||
return m.log
|
||||
}
|
||||
|
||||
type helperRunner interface {
|
||||
Build(context.Context, string, string) error
|
||||
Candidates(context.Context, string, string, bool, bool) ([]string, error)
|
||||
}
|
||||
|
||||
type processRunner struct{}
|
||||
|
||||
// Config locates the index storage for a Manager.
|
||||
//
|
||||
// Archied resolves these in its config layer ([indexing] in config.toml,
|
||||
// defaulted off work_dir by config.finalize) and passes them down. Defaulting
|
||||
// deliberately does NOT live here so this package stays free of archie-core's
|
||||
// config.
|
||||
type Config struct {
|
||||
// IndexDir holds the mmap-friendly codesearch sidecar files.
|
||||
IndexDir string
|
||||
// DBPath is the SQLite database holding index lifecycle metadata.
|
||||
DBPath string
|
||||
// Log receives index build failures. Without it a failed build is
|
||||
// invisible: the manager degrades to unindexed search and the reason
|
||||
// is only ever written to a database column nobody reads.
|
||||
Log *slog.Logger
|
||||
}
|
||||
|
||||
// NewManager prepares lifecycle storage and starts an asynchronous atomic
|
||||
// refresh. An existing index remains available while the refresh runs.
|
||||
func NewManager(ctx context.Context, cfg Config, root string) (*Manager, error) {
|
||||
if strings.TrimSpace(cfg.IndexDir) == "" {
|
||||
return nil, fmt.Errorf("indexing: IndexDir is required")
|
||||
}
|
||||
if strings.TrimSpace(cfg.DBPath) == "" {
|
||||
return nil, fmt.Errorf("indexing: DBPath is required")
|
||||
}
|
||||
absRoot, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("resolve workspace root: %w", err)
|
||||
}
|
||||
if err := os.MkdirAll(cfg.IndexDir, 0o700); err != nil {
|
||||
return nil, fmt.Errorf("create workspace index directory: %w", err)
|
||||
}
|
||||
m := &Manager{
|
||||
root: filepath.Clean(absRoot),
|
||||
indexPath: filepath.Join(cfg.IndexDir, workspaceKey(absRoot)+".csearch"),
|
||||
dbPath: cfg.DBPath,
|
||||
runner: processRunner{},
|
||||
log: cfg.Log,
|
||||
}
|
||||
if err := m.ensureSchema(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.refreshAsync(ctx)
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// Candidates implements the tools.GrepIndex contract.
|
||||
func (m *Manager) Candidates(ctx context.Context, pattern string, literal, caseSensitive bool) ([]string, bool) {
|
||||
state, err := m.state(ctx)
|
||||
if err != nil || state.IndexedAt.IsZero() {
|
||||
return nil, false
|
||||
}
|
||||
if _, err := os.Stat(state.IndexPath); err != nil {
|
||||
return nil, false
|
||||
}
|
||||
files, err := m.runner.Candidates(ctx, state.IndexPath, pattern, literal, caseSensitive)
|
||||
if err != nil {
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// The sidecar is a snapshot. Include files changed or created since the
|
||||
// build began so indexed search cannot miss live workspace edits.
|
||||
merged, ok := m.mergeCandidateFiles(ctx, files, state)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
slices.Sort(merged)
|
||||
return merged, true
|
||||
}
|
||||
|
||||
// mergeCandidateFiles combines sidecar results with workspace files that
|
||||
// were changed or created since the index was built.
|
||||
func (m *Manager) mergeCandidateFiles(ctx context.Context, files []string, state indexState) ([]string, bool) {
|
||||
workspaceFiles, err := WorkspaceFiles(ctx, m.root)
|
||||
if err != nil {
|
||||
return nil, false
|
||||
}
|
||||
changedFiles := changedGitFiles(ctx, m.root)
|
||||
seen := make(map[string]struct{}, len(files))
|
||||
merged := make([]string, 0, len(files))
|
||||
argumentBytes := 0
|
||||
add := func(path string) {
|
||||
path, ok := confinedWorkspaceFile(m.root, path)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if _, ok := seen[path]; ok {
|
||||
return
|
||||
}
|
||||
seen[path] = struct{}{}
|
||||
merged = append(merged, path)
|
||||
argumentBytes += len(path) + 1
|
||||
}
|
||||
for _, path := range files {
|
||||
add(path)
|
||||
}
|
||||
for _, path := range workspaceFiles {
|
||||
info, statErr := os.Stat(path)
|
||||
_, changed := changedFiles[path]
|
||||
if statErr == nil && (changed || !info.ModTime().Before(state.IndexedAt)) {
|
||||
add(path)
|
||||
}
|
||||
}
|
||||
if len(merged) == 0 || len(merged) > MaxCandidateFiles || argumentBytes > maxCandidateArgBytes {
|
||||
return nil, false
|
||||
}
|
||||
return merged, true
|
||||
}
|
||||
|
||||
func changedGitFiles(ctx context.Context, root string) map[string]struct{} {
|
||||
changed := map[string]struct{}{}
|
||||
commands := [][]string{
|
||||
{"-C", root, "ls-files", "-m", "-o", "--exclude-standard", "-z"},
|
||||
{"-C", root, "diff", "--cached", "--name-only", "--diff-filter=ACMR", "-z"},
|
||||
}
|
||||
for _, args := range commands {
|
||||
out, err := exec.CommandContext(ctx, "git", args...).Output()
|
||||
if err != nil {
|
||||
return map[string]struct{}{}
|
||||
}
|
||||
for name := range strings.SplitSeq(string(out), "\x00") {
|
||||
if name != "" {
|
||||
path := filepath.Join(root, filepath.FromSlash(name))
|
||||
if confined, ok := confinedWorkspaceFile(root, path); ok {
|
||||
changed[confined] = struct{}{}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return changed
|
||||
}
|
||||
|
||||
type indexState struct {
|
||||
IndexPath string
|
||||
IndexedAt time.Time
|
||||
}
|
||||
|
||||
func (m *Manager) refreshAsync(parent context.Context) {
|
||||
go func() {
|
||||
ctx, cancel := context.WithTimeout(parent, buildTimeout)
|
||||
defer cancel()
|
||||
startedAt := time.Now().UTC()
|
||||
buildID := uuid.NewString()
|
||||
previous, _ := m.state(ctx)
|
||||
claimed, err := m.claimBuild(ctx, startedAt, buildID)
|
||||
if err != nil {
|
||||
// Degrading to unindexed search is safe but slow, and until
|
||||
// this is logged the only record of why is a database column
|
||||
// with no reader.
|
||||
m.logger().Warn("workspace index build not claimed", "root", m.root, "error", err)
|
||||
return
|
||||
}
|
||||
if !claimed {
|
||||
// Another process holds the build claim; not an error.
|
||||
return
|
||||
}
|
||||
finalPath := m.indexPath + "." + buildID
|
||||
tmp := finalPath + ".tmp"
|
||||
_ = os.Remove(tmp)
|
||||
if err := m.runner.Build(ctx, m.root, tmp); err != nil {
|
||||
_ = os.Remove(tmp)
|
||||
m.logger().Warn("workspace index build failed; falling back to unindexed search",
|
||||
"root", m.root, "error", err)
|
||||
m.finishBuild(parent, buildID, "error", time.Time{}, "", err.Error())
|
||||
return
|
||||
}
|
||||
if err := os.Rename(tmp, finalPath); err != nil {
|
||||
_ = os.Remove(tmp)
|
||||
m.logger().Warn("workspace index could not be published",
|
||||
"root", m.root, "error", err)
|
||||
m.finishBuild(parent, buildID, "error", time.Time{}, "", err.Error())
|
||||
return
|
||||
}
|
||||
if !m.finishBuild(parent, buildID, "ready", startedAt, finalPath, "") {
|
||||
_ = os.Remove(finalPath)
|
||||
m.logger().Warn("workspace index build superseded before publication", "root", m.root)
|
||||
return
|
||||
}
|
||||
m.logger().Info("workspace index ready", "root", m.root, "index", finalPath)
|
||||
if previous.IndexPath != "" && previous.IndexPath != finalPath &&
|
||||
(previous.IndexPath == m.indexPath || strings.HasPrefix(previous.IndexPath, m.indexPath+".")) {
|
||||
_ = os.Remove(previous.IndexPath)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
func (m *Manager) finishBuild(parent context.Context, buildID, status string, indexedAt time.Time, indexPath, lastError string) bool {
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(parent), metadataTimeout)
|
||||
defer cancel()
|
||||
db, err := openIndexDB(m.dbPath)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
// Each helper opens its own single-connection handle and the write has
|
||||
// already been committed by the time this runs, so a Close error carries
|
||||
// no recoverable information.
|
||||
defer func() { _ = db.Close() }()
|
||||
indexed := ""
|
||||
if !indexedAt.IsZero() {
|
||||
indexed = indexedAt.Format(time.RFC3339Nano)
|
||||
}
|
||||
result, err := db.ExecContext(ctx, `UPDATE workspace_indexes SET
|
||||
index_path=CASE WHEN ? = 'ready' THEN ? ELSE index_path END,
|
||||
status=?, indexed_at=CASE WHEN ? = 'ready' THEN ? ELSE indexed_at END,
|
||||
last_error=?, updated_at=?, generation=generation + CASE WHEN ? = 'ready' THEN 1 ELSE 0 END
|
||||
WHERE root=? AND build_id=?`, status, indexPath, status, status, indexed,
|
||||
lastError, time.Now().UTC().Format(time.RFC3339Nano), status, m.root, buildID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
rows, err := result.RowsAffected()
|
||||
return err == nil && rows == 1
|
||||
}
|
||||
|
||||
func (m *Manager) claimBuild(ctx context.Context, startedAt time.Time, buildID string) (bool, error) {
|
||||
db, err := openIndexDB(m.dbPath)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
// Each helper opens its own single-connection handle and the write has
|
||||
// already been committed by the time this runs, so a Close error carries
|
||||
// no recoverable information.
|
||||
defer func() { _ = db.Close() }()
|
||||
cutoff := startedAt.Add(-buildTimeout).Format(time.RFC3339Nano)
|
||||
result, err := db.ExecContext(ctx, `INSERT INTO workspace_indexes (root, index_path, status, indexed_at, build_started_at, build_id, last_error, updated_at)
|
||||
VALUES (?, ?, 'building', '', ?, ?, '', ?)
|
||||
ON CONFLICT(root) DO UPDATE SET status='building',
|
||||
build_started_at=excluded.build_started_at, build_id=excluded.build_id, last_error='', updated_at=excluded.updated_at
|
||||
WHERE workspace_indexes.status != 'building' OR workspace_indexes.updated_at < ?`,
|
||||
m.root, m.indexPath, startedAt.Format(time.RFC3339Nano), buildID, startedAt.Format(time.RFC3339Nano), cutoff)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("claim workspace index build: %w", err)
|
||||
}
|
||||
rows, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("inspect workspace index build claim: %w", err)
|
||||
}
|
||||
return rows > 0, nil
|
||||
}
|
||||
|
||||
func (m *Manager) ensureSchema(ctx context.Context) error {
|
||||
db, err := openIndexDB(m.dbPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Each helper opens its own single-connection handle and the write has
|
||||
// already been committed by the time this runs, so a Close error carries
|
||||
// no recoverable information.
|
||||
defer func() { _ = db.Close() }()
|
||||
_, err = db.ExecContext(ctx, `CREATE TABLE IF NOT EXISTS workspace_indexes (
|
||||
root TEXT PRIMARY KEY,
|
||||
index_path TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
format_version TEXT NOT NULL DEFAULT 'codesearch-v2',
|
||||
generation INTEGER NOT NULL DEFAULT 0,
|
||||
indexed_at TEXT NOT NULL DEFAULT '',
|
||||
build_started_at TEXT NOT NULL DEFAULT '',
|
||||
build_id TEXT NOT NULL DEFAULT '',
|
||||
last_error TEXT NOT NULL DEFAULT '',
|
||||
updated_at TEXT NOT NULL
|
||||
)`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create workspace index metadata: %w", err)
|
||||
}
|
||||
for _, migration := range []string{
|
||||
`ALTER TABLE workspace_indexes ADD COLUMN format_version TEXT NOT NULL DEFAULT 'codesearch-v2'`,
|
||||
`ALTER TABLE workspace_indexes ADD COLUMN generation INTEGER NOT NULL DEFAULT 0`,
|
||||
`ALTER TABLE workspace_indexes ADD COLUMN build_started_at TEXT NOT NULL DEFAULT ''`,
|
||||
`ALTER TABLE workspace_indexes ADD COLUMN build_id TEXT NOT NULL DEFAULT ''`,
|
||||
} {
|
||||
if _, migrationErr := db.ExecContext(ctx, migration); migrationErr != nil && !strings.Contains(migrationErr.Error(), "duplicate column name") {
|
||||
return fmt.Errorf("migrate workspace index metadata: %w", migrationErr)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Manager) state(ctx context.Context) (indexState, error) {
|
||||
db, err := openIndexDB(m.dbPath)
|
||||
if err != nil {
|
||||
return indexState{}, err
|
||||
}
|
||||
// Each helper opens its own single-connection handle and the write has
|
||||
// already been committed by the time this runs, so a Close error carries
|
||||
// no recoverable information.
|
||||
defer func() { _ = db.Close() }()
|
||||
var indexed, indexPath string
|
||||
err = db.QueryRowContext(ctx, `SELECT index_path, indexed_at FROM workspace_indexes WHERE root = ?`, m.root).Scan(&indexPath, &indexed)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return indexState{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return indexState{}, fmt.Errorf("read workspace index metadata: %w", err)
|
||||
}
|
||||
at, _ := time.Parse(time.RFC3339Nano, indexed)
|
||||
return indexState{IndexPath: indexPath, IndexedAt: at}, nil
|
||||
}
|
||||
|
||||
func openIndexDB(path string) (*sql.DB, error) {
|
||||
// modernc.org/sqlite only applies pragmas passed via repeated _pragma
|
||||
// params; unrecognized keys like _journal_mode/_busy_timeout are
|
||||
// silently ignored, which previously left the busy timeout at 0 and
|
||||
// caused SQLITE_BUSY on any lock contention instead of a bounded wait.
|
||||
db, err := sql.Open("sqlite", path+"?_pragma=busy_timeout(5000)&_pragma=journal_mode(WAL)")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open workspace index metadata: %w", err)
|
||||
}
|
||||
db.SetMaxOpenConns(1)
|
||||
return db, nil
|
||||
}
|
||||
|
||||
func workspaceKey(root string) string {
|
||||
sum := sha256.Sum256([]byte(filepath.Clean(root)))
|
||||
return hex.EncodeToString(sum[:12])
|
||||
}
|
||||
|
||||
func (processRunner) Build(ctx context.Context, root, target string) error {
|
||||
_, err := runHelper(ctx, "build", "--root", root, "--index", target)
|
||||
return err
|
||||
}
|
||||
|
||||
func (processRunner) Candidates(ctx context.Context, indexPath, pattern string, literal, caseSensitive bool) ([]string, error) {
|
||||
args := []string{"candidates", "--index", indexPath, "--pattern", pattern}
|
||||
if literal {
|
||||
args = append(args, "--literal")
|
||||
}
|
||||
if caseSensitive {
|
||||
args = append(args, "--case-sensitive")
|
||||
}
|
||||
out, err := runHelper(ctx, args...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var files []string
|
||||
if err := json.Unmarshal(out, &files); err != nil {
|
||||
return nil, fmt.Errorf("decode codesearch candidates: %w", err)
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func runHelper(ctx context.Context, args ...string) ([]byte, error) {
|
||||
executable, err := os.Executable()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("locate archied executable: %w", err)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, executable, append([]string{"workspace-codesearch"}, args...)...)
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
if exitErr, ok := errors.AsType[*exec.ExitError](err); ok {
|
||||
return nil, fmt.Errorf("codesearch helper: %s", strings.TrimSpace(string(exitErr.Stderr)))
|
||||
}
|
||||
return nil, fmt.Errorf("run codesearch helper: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// BuildIndex creates a codesearch sidecar for root. It is called only inside
|
||||
// the isolated helper process.
|
||||
func BuildIndex(ctx context.Context, root, target string) error {
|
||||
files, err := WorkspaceFiles(ctx, root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
w := index.Create(target)
|
||||
w.AddRoots([]index.Path{index.MakePath(root)})
|
||||
for _, path := range files {
|
||||
if err := w.AddFile(path); err != nil {
|
||||
continue
|
||||
}
|
||||
}
|
||||
w.Flush()
|
||||
return nil
|
||||
}
|
||||
|
||||
// IndexCandidates returns the conservative trigram candidate set. It is
|
||||
// called only inside the isolated helper process.
|
||||
func IndexCandidates(indexPath, pattern string, literal, caseSensitive bool) (files []string, err error) {
|
||||
defer func() {
|
||||
if recovered := recover(); recovered != nil {
|
||||
files = nil
|
||||
err = fmt.Errorf("open codesearch index: %v", recovered)
|
||||
}
|
||||
}()
|
||||
if literal {
|
||||
pattern = regexp.QuoteMeta(pattern)
|
||||
}
|
||||
if !caseSensitive && !hasUppercase(pattern) {
|
||||
pattern = "(?i:" + pattern + ")"
|
||||
}
|
||||
re, err := syntax.Parse(pattern, syntax.Perl)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse search pattern: %w", err)
|
||||
}
|
||||
ix := index.Open(indexPath)
|
||||
ids := ix.PostingQuery(index.RegexpQuery(re))
|
||||
files = make([]string, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
files = append(files, ix.Name(id).String())
|
||||
}
|
||||
return files, nil
|
||||
}
|
||||
|
||||
// WorkspaceFiles lists searchable workspace files, preferring Git's ignore
|
||||
// semantics and falling back to a conservative filesystem walk.
|
||||
func WorkspaceFiles(ctx context.Context, root string) ([]string, error) {
|
||||
cmd := exec.CommandContext(ctx, "git", "-C", root, "ls-files", "-co", "--exclude-standard", "-z")
|
||||
out, err := cmd.Output()
|
||||
if err == nil {
|
||||
parts := strings.Split(string(out), "\x00")
|
||||
files := make([]string, 0, len(parts))
|
||||
for _, part := range parts {
|
||||
if part != "" {
|
||||
path := filepath.Join(root, filepath.FromSlash(part))
|
||||
if confined, ok := confinedWorkspaceFile(root, path); ok {
|
||||
files = append(files, confined)
|
||||
}
|
||||
}
|
||||
}
|
||||
slices.Sort(files)
|
||||
return files, nil
|
||||
}
|
||||
|
||||
var files []string
|
||||
err = filepath.WalkDir(root, func(path string, entry os.DirEntry, walkErr error) error {
|
||||
if walkErr != nil {
|
||||
return nil //nolint:nilerr // an unreadable entry skips that path and continues the walk; a best-effort index must not abort on one bad file
|
||||
}
|
||||
if path != root && strings.HasPrefix(entry.Name(), ".") {
|
||||
if entry.IsDir() {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !entry.IsDir() && entry.Type().IsRegular() {
|
||||
if confined, ok := confinedWorkspaceFile(root, path); ok {
|
||||
files = append(files, confined)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("walk workspace files: %w", err)
|
||||
}
|
||||
slices.Sort(files)
|
||||
return files, nil
|
||||
}
|
||||
|
||||
func confinedWorkspaceFile(root, path string) (string, bool) {
|
||||
absRoot, err := filepath.Abs(root)
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
absPath, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
// Reject a symlink at the final component before resolving parent links.
|
||||
info, err := os.Lstat(absPath)
|
||||
if err != nil || !info.Mode().IsRegular() {
|
||||
return "", false
|
||||
}
|
||||
resolvedRoot, err := filepath.EvalSymlinks(absRoot)
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
resolvedPath, err := filepath.EvalSymlinks(absPath)
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
rel, err := filepath.Rel(resolvedRoot, resolvedPath)
|
||||
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return "", false
|
||||
}
|
||||
resolvedInfo, err := os.Lstat(resolvedPath)
|
||||
if err != nil || !resolvedInfo.Mode().IsRegular() {
|
||||
return "", false
|
||||
}
|
||||
return filepath.Clean(resolvedPath), true
|
||||
}
|
||||
|
||||
func hasUppercase(value string) bool {
|
||||
for _, r := range value {
|
||||
if r >= 'A' && r <= 'Z' {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -1,224 +0,0 @@
|
||||
package indexing
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestBuildIndexAndCandidates(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeTestFile(t, root, "a.txt", "alpha needle omega\n")
|
||||
writeTestFile(t, root, "b.txt", "nothing here\n")
|
||||
indexPath := filepath.Join(t.TempDir(), "workspace.csearch")
|
||||
|
||||
if err := BuildIndex(context.Background(), root, indexPath); err != nil {
|
||||
t.Fatalf("BuildIndex() error = %v", err)
|
||||
}
|
||||
files, err := IndexCandidates(indexPath, "needle", false, false)
|
||||
if err != nil {
|
||||
t.Fatalf("IndexCandidates() error = %v", err)
|
||||
}
|
||||
if len(files) != 1 || files[0] != filepath.Join(root, "a.txt") {
|
||||
t.Fatalf("candidates = %v", files)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkspaceFilesRespectsGitIgnore(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
runGit(t, root, "init", "-q")
|
||||
writeTestFile(t, root, ".gitignore", "ignored.txt\n")
|
||||
writeTestFile(t, root, "tracked.txt", "tracked\n")
|
||||
writeTestFile(t, root, "untracked.txt", "untracked\n")
|
||||
writeTestFile(t, root, "ignored.txt", "ignored\n")
|
||||
runGit(t, root, "add", ".gitignore", "tracked.txt")
|
||||
|
||||
files, err := WorkspaceFiles(context.Background(), root)
|
||||
if err != nil {
|
||||
t.Fatalf("WorkspaceFiles() error = %v", err)
|
||||
}
|
||||
if slices.Contains(files, filepath.Join(root, "ignored.txt")) {
|
||||
t.Fatalf("ignored file returned: %v", files)
|
||||
}
|
||||
for _, name := range []string{"tracked.txt", "untracked.txt"} {
|
||||
if !slices.Contains(files, filepath.Join(root, name)) {
|
||||
t.Fatalf("%s missing from %v", name, files)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkspaceFilesRejectsTrackedSymlinkOutsideRoot(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
runGit(t, root, "init", "-q")
|
||||
outside := filepath.Join(t.TempDir(), "secret.txt")
|
||||
if err := os.WriteFile(outside, []byte("secret"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
link := filepath.Join(root, "link.txt")
|
||||
if err := os.Symlink(outside, link); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runGit(t, root, "add", "link.txt")
|
||||
|
||||
files, err := WorkspaceFiles(context.Background(), root)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, accepted := confinedWorkspaceFile(root, link)
|
||||
if slices.Contains(files, link) || accepted {
|
||||
t.Fatalf("outside symlink accepted: %v", files)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfinedWorkspaceFileRejectsIntermediateSymlinkEscape(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
outside := t.TempDir()
|
||||
writeTestFile(t, outside, "secret.txt", "secret")
|
||||
if err := os.Symlink(outside, filepath.Join(root, "linkdir")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if path, ok := confinedWorkspaceFile(root, filepath.Join(root, "linkdir", "secret.txt")); ok {
|
||||
t.Fatalf("intermediate symlink escape accepted as %s", path)
|
||||
}
|
||||
}
|
||||
|
||||
func TestManagerRefreshBuildsAtomicallyAndRecordsGeneration(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
dir := t.TempDir()
|
||||
m := &Manager{
|
||||
root: root, indexPath: filepath.Join(dir, "workspace.csearch"),
|
||||
dbPath: filepath.Join(dir, "indexes.db"), runner: writingRunner{},
|
||||
}
|
||||
if err := m.ensureSchema(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := t.Context()
|
||||
m.refreshAsync(ctx)
|
||||
|
||||
// Poll for the async build to complete. Use a deadline longer than SQLite's
|
||||
// busy timeout (5 s) so a contended CI runner doesn't trigger a false timeout.
|
||||
deadline := time.Now().Add(8 * time.Second)
|
||||
var installedPath string
|
||||
for {
|
||||
state, stateErr := m.state(context.Background())
|
||||
if stateErr == nil && !state.IndexedAt.IsZero() {
|
||||
if _, err := os.Stat(state.IndexPath); err == nil {
|
||||
installedPath = state.IndexPath
|
||||
break
|
||||
}
|
||||
}
|
||||
// If the goroutine errored, fail fast with the last error.
|
||||
if stateErr == nil && state.IndexedAt.IsZero() {
|
||||
// Check if status indicates an error.
|
||||
db, dbErr := openIndexDB(m.dbPath)
|
||||
if dbErr == nil {
|
||||
var status, lastErr string
|
||||
_ = db.QueryRowContext(context.Background(),
|
||||
`SELECT status, last_error FROM workspace_indexes WHERE root = ?`, m.root).Scan(&status, &lastErr)
|
||||
_ = db.Close()
|
||||
if status == "error" {
|
||||
t.Fatalf("build failed: %s", lastErr)
|
||||
}
|
||||
}
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("index was not atomically installed")
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
if installedPath == m.indexPath {
|
||||
t.Fatalf("build did not install a generation-specific sidecar: %s", installedPath)
|
||||
}
|
||||
db, err := openIndexDB(m.dbPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = db.Close() }()
|
||||
var status string
|
||||
var generation int
|
||||
if err := db.QueryRowContext(context.Background(), `SELECT status, generation FROM workspace_indexes WHERE root = ?`, root).Scan(&status, &generation); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if status != "ready" || generation != 1 {
|
||||
t.Fatalf("state = %q generation %d", status, generation)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStaleBuilderCannotCompleteNewerLease(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
m := &Manager{root: t.TempDir(), indexPath: filepath.Join(dir, "workspace.csearch"), dbPath: filepath.Join(dir, "indexes.db")}
|
||||
if err := m.ensureSchema(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
oldStart := time.Now().UTC().Add(-2 * buildTimeout)
|
||||
claimed, err := m.claimBuild(context.Background(), oldStart, "build-a")
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("first claim = %v, %v", claimed, err)
|
||||
}
|
||||
newStart := time.Now().UTC()
|
||||
claimed, err = m.claimBuild(context.Background(), newStart, "build-b")
|
||||
if err != nil || !claimed {
|
||||
t.Fatalf("replacement claim = %v, %v", claimed, err)
|
||||
}
|
||||
if m.finishBuild(context.Background(), "build-a", "ready", oldStart, filepath.Join(dir, "a"), "") {
|
||||
t.Fatal("stale builder completed replacement lease")
|
||||
}
|
||||
pathB := filepath.Join(dir, "b")
|
||||
if !m.finishBuild(context.Background(), "build-b", "ready", newStart, pathB, "") {
|
||||
t.Fatal("current builder could not complete lease")
|
||||
}
|
||||
state, err := m.state(context.Background())
|
||||
if err != nil || state.IndexPath != pathB {
|
||||
t.Fatalf("state = %#v, %v", state, err)
|
||||
}
|
||||
}
|
||||
|
||||
type writingRunner struct{}
|
||||
|
||||
func (writingRunner) Build(_ context.Context, _, target string) error {
|
||||
return os.WriteFile(target, []byte("index"), 0o600)
|
||||
}
|
||||
|
||||
func (writingRunner) Candidates(context.Context, string, string, bool, bool) ([]string, error) {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
func writeTestFile(t *testing.T, dir, name, content string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func runGit(t *testing.T, dir string, args ...string) {
|
||||
t.Helper()
|
||||
cmd := exec.CommandContext(context.Background(), "git", append([]string{"-C", dir}, args...)...)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("git %v: %v: %s", args, err, out)
|
||||
}
|
||||
}
|
||||
|
||||
// A Manager built as a struct literal has no logger. Logging on the build
|
||||
// path must not panic, since that path runs in a background goroutine where
|
||||
// a panic takes the whole daemon down.
|
||||
func TestManagerLoggerIsNilSafe(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
m *Manager
|
||||
}{
|
||||
{name: "zero manager", m: &Manager{}},
|
||||
{name: "nil manager", m: nil},
|
||||
}
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := tc.m.logger(); got == nil {
|
||||
t.Fatal("logger() returned nil")
|
||||
}
|
||||
tc.m.logger().Warn("must not panic")
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -231,7 +231,6 @@ func (l *Loader) applyGeneralDefaults(cfg *config.Config) {
|
||||
if cfg.DBPath == "" {
|
||||
cfg.DBPath = filepath.Join(l.dataHome(), "archie", "archie.db")
|
||||
}
|
||||
cfg.Indexing = cfg.Indexing.WithDefaults(cfg.WorkDir)
|
||||
if cfg.PollInterval == 0 {
|
||||
cfg.PollInterval = config.Duration(defaultPollInterval)
|
||||
}
|
||||
|
||||
@@ -904,55 +904,6 @@ func TestLoadBytesDoesNotExist(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestFinalizeIndexingDefaults(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
extra string
|
||||
wantIndex string
|
||||
wantDBIsSet bool
|
||||
}{
|
||||
{
|
||||
name: "defaults derive from work_dir",
|
||||
// Both paths must land under the daemon's own work_dir so two
|
||||
// archied instances on one host cannot share an index.
|
||||
extra: "\nwork_dir = \"/srv/archie/work\"\n",
|
||||
wantIndex: filepath.Join("/srv/archie/work", "indexes"),
|
||||
wantDBIsSet: true,
|
||||
},
|
||||
{
|
||||
name: "explicit index_dir is preserved",
|
||||
extra: "\nwork_dir = \"/srv/archie/work\"\n" +
|
||||
"[indexing]\nindex_dir = \"/mnt/fast/idx\"\n",
|
||||
wantIndex: "/mnt/fast/idx",
|
||||
wantDBIsSet: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config.toml")
|
||||
// tc.extra precedes the table sections: a top-level key written
|
||||
// after [[repos]] would be parsed as a member of that table.
|
||||
contents := "bot_user = \"widget\"\n" + tc.extra +
|
||||
"[forge]\ntype = \"gitea\"\nhost = \"https://git.example.test\"\ntoken_env = \"T\"\n" +
|
||||
"[[repos]]\nowner = \"acme\"\nname = \"app\"\n"
|
||||
if err := os.WriteFile(path, []byte(contents), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := loadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Indexing.IndexDir != tc.wantIndex {
|
||||
t.Errorf("Indexing.IndexDir = %q, want %q", cfg.Indexing.IndexDir, tc.wantIndex)
|
||||
}
|
||||
if tc.wantDBIsSet && cfg.Indexing.DBPath == "" {
|
||||
t.Error("Indexing.DBPath is empty; want a default derived from work_dir")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdentitiesConfigParsesTwoIdentities(t *testing.T) {
|
||||
cfg, err := loadBytes([]byte(`
|
||||
bot_user = "default-legacy"
|
||||
|
||||
@@ -155,7 +155,7 @@ func TestApply_Idempotent_SecondRunLeavesOtherEditByteIdentical(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// [models], [indexing], [web], and [notify] in the real template all carry
|
||||
// [models], [web], and [notify] in the real template all carry
|
||||
// a trailing "# ..." comment on the header line itself
|
||||
// ("[models] # role -> ..."). The header regexes were anchored with no
|
||||
// allowance for that, so Apply's table-tracking never recognised these as
|
||||
|
||||
@@ -904,7 +904,8 @@ func TestUpdateInstallBacksUpTheSiblingTaskDatabase(t *testing.T) {
|
||||
assertCallAbsent(t, calls, "--setenv=ARCHIE_TASK_DB_PATH="+configured+" ")
|
||||
}
|
||||
|
||||
// db_path also appears in other sections -- [indexing] owns its own database.
|
||||
// db_path can also appear under a TOML table -- the retired [indexing] section
|
||||
// once owned its own database there, and a future section could do the same.
|
||||
// Reading the first match in the file can select an unrelated database, so the
|
||||
// value must come from the daemon's own top-level key.
|
||||
func TestUpdateInstallIgnoresDbPathFromOtherSections(t *testing.T) {
|
||||
@@ -929,7 +930,7 @@ func TestUpdateInstallIgnoresDbPathFromOtherSections(t *testing.T) {
|
||||
assertCallContains(t, calls, "go run ./cmd/archie-state-store backup", "-db "+realStore)
|
||||
for _, call := range calls {
|
||||
if strings.Contains(call, "go run ./cmd/archie-state-store backup") && strings.Contains(call, indexStore) {
|
||||
t.Errorf("backed up the indexing database %s: %q", indexStore, call)
|
||||
t.Errorf("backed up the other section's database %s: %q", indexStore, call)
|
||||
}
|
||||
}
|
||||
assertCallAbsent(t, calls, "--setenv=ARCHIE_TASK_DB_PATH="+indexStore)
|
||||
|
||||
@@ -7,6 +7,8 @@
|
||||
// - the docs tool is not registered and docs.go was not lifted. It serves
|
||||
// tau's own embedded documentation via the tau/docs package, which has
|
||||
// no meaning here. The pluginDocs parameter went with it.
|
||||
// - the workspace codesearch index parameter was removed along with the
|
||||
// index itself; RegisterBuiltins takes only the registry and cwd.
|
||||
//
|
||||
// Refresh by diffing against that path at a newer tau commit. Do not
|
||||
// edit without recording the change above.
|
||||
@@ -14,7 +16,7 @@ package builtin
|
||||
|
||||
// RegisterBuiltins registers all built-in tools into the given registry.
|
||||
// The cwd parameter sets the working directory for file and shell operations.
|
||||
func RegisterBuiltins(reg *Registry, cwd string, indexes ...GrepIndex) error {
|
||||
func RegisterBuiltins(reg *Registry, cwd string) error {
|
||||
mq := NewMutationQueue()
|
||||
rt := NewReadTracker()
|
||||
|
||||
@@ -23,7 +25,7 @@ func RegisterBuiltins(reg *Registry, cwd string, indexes ...GrepIndex) error {
|
||||
NewWriteTool(cwd, mq, rt),
|
||||
NewEditTool(cwd, mq, rt),
|
||||
NewShellTool(cwd, mq),
|
||||
NewGrepTool(cwd, indexes...),
|
||||
NewGrepTool(cwd),
|
||||
NewFindTool(cwd),
|
||||
}
|
||||
|
||||
|
||||
+28
-122
@@ -14,6 +14,10 @@
|
||||
// smaller helpers (resolveGrepTargets, runGrepBinary, grepExplicitTargets,
|
||||
// setGrepResultsRelPath, grepWalkVisitor) to satisfy golangci-lint
|
||||
// cyclop/gocognit limits. No behavioural change.
|
||||
// - the workspace codesearch index integration was removed: GrepIndex, the
|
||||
// candidate-narrowing path, the codesearch backend retry, and the
|
||||
// search_backend metric are gone, and grep always walks the requested
|
||||
// path directly.
|
||||
//
|
||||
// Refresh by diffing against that path at a newer tau commit. Do not
|
||||
// edit without recording the change above.
|
||||
@@ -67,12 +71,6 @@ type GrepParams struct {
|
||||
Limit int `json:"limit,omitempty"` // max matches to return
|
||||
}
|
||||
|
||||
// GrepIndex provides conservative workspace-wide candidate files. The grep
|
||||
// tool remains responsible for authoritative matching and output formatting.
|
||||
type GrepIndex interface {
|
||||
Candidates(context.Context, string, bool, bool) ([]string, bool)
|
||||
}
|
||||
|
||||
var grepSchema = Schema{
|
||||
Name: "grep",
|
||||
Description: fmt.Sprintf("Search file contents for a regex pattern using ripgrep (rg). Respects .gitignore. Returns matching lines with file paths and line numbers. Supports alternation (e.g. 'foo|bar') and full regex syntax. Use context_before/context_after to show surrounding lines (capped at %d each). Output is capped at %d matches (adjustable via limit) and long lines are truncated to %d chars.", grepMaxContext, grepDefaultLimit, grepMaxLineChars),
|
||||
@@ -118,20 +116,16 @@ var grepSchema = Schema{
|
||||
|
||||
// NewGrepTool creates the built-in grep tool.
|
||||
|
||||
func NewGrepTool(cwd string, indexes ...GrepIndex) Tool {
|
||||
var workspaceIndex GrepIndex
|
||||
if len(indexes) > 0 {
|
||||
workspaceIndex = indexes[0]
|
||||
}
|
||||
func NewGrepTool(cwd string) Tool {
|
||||
return Tool{
|
||||
Schema: grepSchema,
|
||||
Source: "builtin",
|
||||
Emoji: "🔍",
|
||||
Execute: makeGrepExecutor(cwd, workspaceIndex),
|
||||
Execute: makeGrepExecutor(cwd),
|
||||
}
|
||||
}
|
||||
|
||||
func makeGrepExecutor(cwd string, workspaceIndex GrepIndex) Executor {
|
||||
func makeGrepExecutor(cwd string) Executor {
|
||||
return func(ctx context.Context, params json.RawMessage, _ UIBridge) (Result, error) {
|
||||
var p GrepParams
|
||||
if err := json.Unmarshal(params, &p); err != nil {
|
||||
@@ -156,8 +150,6 @@ func makeGrepExecutor(cwd string, workspaceIndex GrepIndex) Executor {
|
||||
return Result{Content: "error: path escapes working directory", IsError: true, ErrorKind: "sandbox_escape"}, nil
|
||||
}
|
||||
|
||||
searchTargets, searchBackend := resolveGrepTargets(ctx, workspaceIndex, p, cwd, searchPath)
|
||||
|
||||
limit := p.Limit
|
||||
if limit <= 0 {
|
||||
limit = grepDefaultLimit
|
||||
@@ -165,55 +157,37 @@ func makeGrepExecutor(cwd string, workspaceIndex GrepIndex) Executor {
|
||||
|
||||
binary, err := grepBinary()
|
||||
if err != nil {
|
||||
searchBackend = "direct"
|
||||
// No external binary available - use pure-Go fallback.
|
||||
output, err := grepFallback(ctx, p, searchPath, cwd, searchTargets)
|
||||
output, err := grepFallback(ctx, p, searchPath, cwd)
|
||||
if err != nil {
|
||||
return grepBackendResult(Result{Content: fmt.Sprintf("grep error: %v", err), IsError: true}, searchBackend, clamped), nil
|
||||
return applyClampNotice(Result{Content: fmt.Sprintf("grep error: %v", err), IsError: true}, clamped), nil
|
||||
}
|
||||
if output == "" {
|
||||
return grepBackendResult(Result{Content: "no matches found"}, searchBackend, clamped), nil
|
||||
return applyClampNotice(Result{Content: "no matches found"}, clamped), nil
|
||||
}
|
||||
return grepBackendResult(capGrepResult(output, limit), searchBackend, clamped), nil
|
||||
return applyClampNotice(capGrepResult(output, limit), clamped), nil
|
||||
}
|
||||
|
||||
result, backend := runGrepBinary(ctx, binary, cwd, p, searchPath, searchTargets, searchBackend, limit)
|
||||
return grepBackendResult(result, backend, clamped), nil
|
||||
result := runGrepBinary(ctx, binary, cwd, p, searchPath, limit)
|
||||
return applyClampNotice(result, clamped), nil
|
||||
}
|
||||
}
|
||||
|
||||
// resolveGrepTargets picks the search targets and backend label: the
|
||||
// workspace index's narrowed candidates when it can serve the request, or a
|
||||
// direct walk of searchPath otherwise.
|
||||
//
|
||||
// The index is workspace-wide, so it can serve any search scoped at or below
|
||||
// cwd once its candidates are narrowed to the requested subtree. Requiring
|
||||
// searchPath == cwd meant subdirectory searches silently fell back to a full
|
||||
// walk, which is why the index served only 5% of searches across analysed
|
||||
// sessions.
|
||||
func resolveGrepTargets(ctx context.Context, workspaceIndex GrepIndex, p GrepParams, cwd, searchPath string) ([]string, string) {
|
||||
if workspaceIndex != nil && isConfined(cwd, searchPath) {
|
||||
if candidates, ok := workspaceIndex.Candidates(ctx, p.Pattern, p.Literal, p.CaseSensitive); ok {
|
||||
if scoped := filterUnder(searchPath, candidates); len(scoped) > 0 {
|
||||
return scoped, "codesearch"
|
||||
}
|
||||
// An empty scoped set is NOT evidence of no matches: the index
|
||||
// may simply be cold or stale for this subtree, so fall through
|
||||
// to the authoritative direct walk rather than reporting "no
|
||||
// matches found" from an empty candidate list.
|
||||
}
|
||||
// applyClampNotice appends the context-clamp notice to a successful result.
|
||||
func applyClampNotice(result Result, clampNotice string) Result {
|
||||
if clampNotice != "" && !result.IsError {
|
||||
result.Content += "\n" + clampNotice
|
||||
}
|
||||
return []string{searchPath}, "direct"
|
||||
return result
|
||||
}
|
||||
|
||||
// runGrepBinary runs the external rg binary against searchTargets, retrying
|
||||
// against the authoritative searchPath if a codesearch-backed run fails, and
|
||||
// returns a Result plus the backend that actually produced it.
|
||||
// runGrepBinary runs the external rg binary against searchPath and returns
|
||||
// the Result it produced.
|
||||
func runGrepBinary(
|
||||
ctx context.Context, binary, cwd string, p GrepParams,
|
||||
searchPath string, searchTargets []string, searchBackend string, limit int,
|
||||
) (Result, string) {
|
||||
args := append(buildGrepArgs(p), searchTargets...)
|
||||
searchPath string, limit int,
|
||||
) Result {
|
||||
args := append(buildGrepArgs(p), searchPath)
|
||||
cmd := exec.CommandContext(ctx, binary, args...)
|
||||
cmd.Dir = cwd
|
||||
|
||||
@@ -222,22 +196,7 @@ func runGrepBinary(
|
||||
cmd.Stderr = &stderr
|
||||
|
||||
err := cmd.Run()
|
||||
|
||||
output := stdout.String()
|
||||
if searchBackend == "codesearch" && err != nil {
|
||||
// Snapshot candidates can disappear, or an unusually large candidate
|
||||
// argv can fail to spawn. Retry the authoritative workspace path.
|
||||
searchBackend = "direct"
|
||||
args = append(buildGrepArgs(p), searchPath)
|
||||
cmd = exec.CommandContext(ctx, binary, args...)
|
||||
cmd.Dir = cwd
|
||||
stdout.Reset()
|
||||
stderr.Reset()
|
||||
cmd.Stdout = &stdout
|
||||
cmd.Stderr = &stderr
|
||||
err = cmd.Run()
|
||||
output = stdout.String()
|
||||
}
|
||||
if output == "" && err != nil {
|
||||
// Exit 1 means "no matches"; exit 2 means a real failure (bad
|
||||
// pattern, unreadable path). Reporting the latter as an empty
|
||||
@@ -246,31 +205,16 @@ func runGrepBinary(
|
||||
// error type.
|
||||
var exitErr *exec.ExitError
|
||||
if errors.As(err, &exitErr) && exitErr.ExitCode() == 1 {
|
||||
return Result{Content: "no matches found"}, searchBackend
|
||||
return Result{Content: "no matches found"}
|
||||
}
|
||||
errMsg := stderr.String()
|
||||
if errMsg == "" {
|
||||
errMsg = err.Error()
|
||||
}
|
||||
return Result{Content: fmt.Sprintf("grep error: %s", errMsg), IsError: true}, searchBackend
|
||||
return Result{Content: fmt.Sprintf("grep error: %s", errMsg), IsError: true}
|
||||
}
|
||||
|
||||
return capGrepResult(output, limit), searchBackend
|
||||
}
|
||||
|
||||
// filterUnder returns the candidates at or below root. Paths are compared
|
||||
// after cleaning so that a candidate is only kept when root is a genuine path
|
||||
// prefix, not merely a string prefix ("/a/bc" is not under "/a/b").
|
||||
func filterUnder(root string, candidates []string) []string {
|
||||
root = filepath.Clean(root)
|
||||
|
||||
out := make([]string, 0, len(candidates))
|
||||
for _, c := range candidates {
|
||||
if isConfined(root, filepath.Clean(c)) {
|
||||
out = append(out, c)
|
||||
}
|
||||
}
|
||||
return out
|
||||
return capGrepResult(output, limit)
|
||||
}
|
||||
|
||||
// clampGrepContext bounds the context window in place and returns a notice
|
||||
@@ -294,14 +238,6 @@ func clampGrepContext(p *GrepParams) string {
|
||||
)
|
||||
}
|
||||
|
||||
func grepBackendResult(result Result, backend, clampNotice string) Result {
|
||||
result.MetricLabels = map[string]string{"search_backend": backend}
|
||||
if clampNotice != "" && !result.IsError {
|
||||
result.Content += "\n" + clampNotice
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func capGrepResult(output string, limit int) Result {
|
||||
lines := strings.Split(strings.TrimSuffix(output, "\n"), "\n")
|
||||
kept := make([]string, 0, len(lines))
|
||||
@@ -349,23 +285,13 @@ func truncationBoundary(s string, limit int) int {
|
||||
}
|
||||
|
||||
// grepFallback performs a pure-Go file scan for when ripgrep is not available.
|
||||
// Works on all platforms including Windows. When targets has explicit file paths
|
||||
// (not just the searchPath directory), only those files are searched.
|
||||
func grepFallback(ctx context.Context, p GrepParams, searchPath, cwd string, targets []string) (string, error) {
|
||||
// Works on all platforms including Windows.
|
||||
func grepFallback(ctx context.Context, p GrepParams, searchPath, cwd string) (string, error) {
|
||||
matcher, err := buildMatcher(p)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
// When explicit files are provided (index candidates), search only those.
|
||||
if len(targets) > 0 && targets[0] != searchPath {
|
||||
results, err := grepExplicitTargets(ctx, targets, cwd, matcher, p)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return formatGrepResults(results), nil
|
||||
}
|
||||
|
||||
var results []grepResult
|
||||
info, err := os.Stat(searchPath)
|
||||
if err != nil {
|
||||
@@ -386,26 +312,6 @@ func grepFallback(ctx context.Context, p GrepParams, searchPath, cwd string, tar
|
||||
return formatGrepResults(results), nil
|
||||
}
|
||||
|
||||
// grepExplicitTargets searches only the given files (index candidates),
|
||||
// skipping any that fail to read.
|
||||
func grepExplicitTargets(ctx context.Context, targets []string, cwd string, matcher func(string) bool, p GrepParams) ([]grepResult, error) {
|
||||
var results []grepResult
|
||||
for _, path := range targets {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
res, fErr := grepFile(ctx, path, matcher, p.ContextBefore, p.ContextAfter)
|
||||
if fErr != nil {
|
||||
continue
|
||||
}
|
||||
setGrepResultsRelPath(res, cwd, path)
|
||||
results = append(results, res...)
|
||||
}
|
||||
return results, nil
|
||||
}
|
||||
|
||||
// setGrepResultsRelPath fills in each result's relPath relative to cwd,
|
||||
// falling back to the file's base name when the relative path collapses to
|
||||
// "" or ".".
|
||||
|
||||
@@ -11,27 +11,6 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
type staticGrepIndex struct{ files []string }
|
||||
|
||||
func (s staticGrepIndex) Candidates(context.Context, string, bool, bool) ([]string, bool) {
|
||||
return s.files, true
|
||||
}
|
||||
|
||||
func TestGrepUsesIndexCandidatesAsAdvisoryFileSet(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
indexed := filepath.Join(tmp, "indexed.txt")
|
||||
createGrepTestFile(t, tmp, "indexed.txt", "needle\n")
|
||||
createGrepTestFile(t, tmp, "excluded.txt", "needle\n")
|
||||
tool := NewGrepTool(tmp, staticGrepIndex{files: []string{indexed}})
|
||||
result, err := tool.Execute(context.Background(), json.RawMessage(`{"pattern":"needle"}`), nil)
|
||||
if err != nil || result.IsError {
|
||||
t.Fatalf("grep result = %#v, err = %v", result, err)
|
||||
}
|
||||
if !strings.Contains(result.Content, "indexed.txt") || strings.Contains(result.Content, "excluded.txt") {
|
||||
t.Fatalf("candidate-filtered output = %s", result.Content)
|
||||
}
|
||||
}
|
||||
|
||||
// TestGrepReportsSearchFailureRatherThanNoMatches guards the distinction
|
||||
// between ripgrep's exit 1 (no matches) and exit 2 (the search never ran). An
|
||||
// errorlint autofix once rewrote the check to match any *exec.ExitError and
|
||||
@@ -42,7 +21,7 @@ func TestGrepUsesIndexCandidatesAsAdvisoryFileSet(t *testing.T) {
|
||||
func TestGrepReportsSearchFailureRatherThanNoMatches(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
createGrepTestFile(t, tmp, "haystack.txt", "needle\n")
|
||||
tool := NewGrepTool(tmp, nil)
|
||||
tool := NewGrepTool(tmp)
|
||||
|
||||
result, err := tool.Execute(context.Background(), json.RawMessage(`{"pattern":"(unclosed"}`), nil)
|
||||
if err != nil {
|
||||
@@ -56,36 +35,6 @@ func TestGrepReportsSearchFailureRatherThanNoMatches(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestGrepRetriesDirectSearchWhenIndexedCandidatesAreStale(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
createGrepTestFile(t, tmp, "live.txt", "needle\n")
|
||||
missing := filepath.Join(tmp, "deleted.txt")
|
||||
tool := NewGrepTool(tmp, staticGrepIndex{files: []string{missing}})
|
||||
result, err := tool.Execute(context.Background(), json.RawMessage(`{"pattern":"needle"}`), nil)
|
||||
if err != nil || result.IsError {
|
||||
t.Fatalf("grep result = %#v, err = %v", result, err)
|
||||
}
|
||||
if !strings.Contains(result.Content, "live.txt") || result.MetricLabels["search_backend"] != "direct" {
|
||||
t.Fatalf("fallback output = %s, labels = %v", result.Content, result.MetricLabels)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGrepDiscardsPartialIndexedOutputOnCandidateError(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
live := filepath.Join(tmp, "live.txt")
|
||||
createGrepTestFile(t, tmp, "live.txt", "needle\n")
|
||||
createGrepTestFile(t, tmp, "also-live.txt", "needle\n")
|
||||
missing := filepath.Join(tmp, "deleted.txt")
|
||||
tool := NewGrepTool(tmp, staticGrepIndex{files: []string{live, missing}})
|
||||
result, err := tool.Execute(context.Background(), json.RawMessage(`{"pattern":"needle"}`), nil)
|
||||
if err != nil || result.IsError {
|
||||
t.Fatalf("grep result = %#v, err = %v", result, err)
|
||||
}
|
||||
if !strings.Contains(result.Content, "also-live.txt") || result.MetricLabels["search_backend"] != "direct" {
|
||||
t.Fatalf("partial indexed output was not replaced: %s, labels = %v", result.Content, result.MetricLabels)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGrepFallback(t *testing.T) {
|
||||
// Create a temporary directory with test files.
|
||||
tmp := t.TempDir()
|
||||
@@ -196,7 +145,7 @@ func TestGrepFallback(t *testing.T) {
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, err := grepFallback(ctx, tc.params, tc.params.Path, tmp, nil)
|
||||
got, err := grepFallback(ctx, tc.params, tc.params.Path, tmp)
|
||||
if err != nil {
|
||||
t.Fatalf("grepFallback error: %v", err)
|
||||
}
|
||||
@@ -231,7 +180,7 @@ func TestGrepFallback_ContextDedup(t *testing.T) {
|
||||
defer cancel()
|
||||
|
||||
params := GrepParams{Pattern: "Hello", Path: tmp, ContextBefore: 1, ContextAfter: 1}
|
||||
got, err := grepFallback(ctx, params, tmp, tmp, nil)
|
||||
got, err := grepFallback(ctx, params, tmp, tmp)
|
||||
if err != nil {
|
||||
t.Fatalf("grepFallback error: %v", err)
|
||||
}
|
||||
@@ -305,7 +254,7 @@ func TestGrepFallback_NonExistentPath(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
|
||||
_, err := grepFallback(ctx, GrepParams{Pattern: "foo"}, "/nonexistent/path", "/nonexistent/path", nil)
|
||||
_, err := grepFallback(ctx, GrepParams{Pattern: "foo"}, "/nonexistent/path", "/nonexistent/path")
|
||||
if err == nil {
|
||||
t.Fatal("expected error for non-existent path")
|
||||
}
|
||||
@@ -334,7 +283,7 @@ func TestGrepClampsContextLines(t *testing.T) {
|
||||
}
|
||||
createGrepTestFile(t, tmp, "f.txt", strings.Join(lines, "\n")+"\n")
|
||||
|
||||
tool := NewGrepTool(tmp, nil)
|
||||
tool := NewGrepTool(tmp)
|
||||
res, err := tool.Execute(context.Background(), json.RawMessage(
|
||||
`{"pattern":"needle","context_before":40,"context_after":40}`,
|
||||
), nil)
|
||||
@@ -362,7 +311,7 @@ func TestGrepLeavesModestContextAlone(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
createGrepTestFile(t, tmp, "f.txt", "a\nb\nneedle\nd\ne\n")
|
||||
|
||||
tool := NewGrepTool(tmp, nil)
|
||||
tool := NewGrepTool(tmp)
|
||||
res, err := tool.Execute(context.Background(), json.RawMessage(
|
||||
`{"pattern":"needle","context_before":2,"context_after":2}`,
|
||||
), nil)
|
||||
@@ -378,60 +327,3 @@ func TestGrepLeavesModestContextAlone(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The index served only 5% of searches across the analysed sessions
|
||||
// (codesearch 11, direct 199) because it engaged only when the search path was
|
||||
// exactly cwd. Any subdirectory-scoped search silently fell back to a walk.
|
||||
func TestGrepUsesIndexForSubdirectorySearch(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
createGrepTestFile(t, tmp, "sub/inside.txt", "needle\n")
|
||||
createGrepTestFile(t, tmp, "other/outside.txt", "needle\n")
|
||||
|
||||
index := staticGrepIndex{files: []string{
|
||||
filepath.Join(tmp, "sub", "inside.txt"),
|
||||
filepath.Join(tmp, "other", "outside.txt"),
|
||||
}}
|
||||
|
||||
tool := NewGrepTool(tmp, index)
|
||||
res, err := tool.Execute(context.Background(), json.RawMessage(
|
||||
`{"pattern":"needle","path":"sub"}`,
|
||||
), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if res.IsError {
|
||||
t.Fatalf("unexpected tool error: %s", res.Content)
|
||||
}
|
||||
if got := res.MetricLabels["search_backend"]; got != "codesearch" {
|
||||
t.Fatalf("search_backend = %q, want codesearch", got)
|
||||
}
|
||||
if !strings.Contains(res.Content, "inside.txt") {
|
||||
t.Fatalf("expected the in-scope match, got:\n%s", res.Content)
|
||||
}
|
||||
if strings.Contains(res.Content, "outside.txt") {
|
||||
t.Fatalf("candidates outside the search path must be filtered out, got:\n%s", res.Content)
|
||||
}
|
||||
}
|
||||
|
||||
// If the index has no candidates inside the requested subtree we must not
|
||||
// conclude "no matches" from an empty candidate list - fall back to a walk.
|
||||
func TestGrepFallsBackToDirectWhenNoCandidatesInSubtree(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
createGrepTestFile(t, tmp, "sub/inside.txt", "needle\n")
|
||||
|
||||
index := staticGrepIndex{files: []string{filepath.Join(tmp, "elsewhere.txt")}}
|
||||
|
||||
tool := NewGrepTool(tmp, index)
|
||||
res, err := tool.Execute(context.Background(), json.RawMessage(
|
||||
`{"pattern":"needle","path":"sub"}`,
|
||||
), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if got := res.MetricLabels["search_backend"]; got != "direct" {
|
||||
t.Fatalf("search_backend = %q, want direct", got)
|
||||
}
|
||||
if !strings.Contains(res.Content, "inside.txt") {
|
||||
t.Fatalf("an empty candidate set must not hide real matches, got:\n%s", res.Content)
|
||||
}
|
||||
}
|
||||
@@ -108,7 +108,7 @@ func TestGrepRejectsGoModCache(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
tool := NewGrepTool(cwd, nil)
|
||||
tool := NewGrepTool(cwd)
|
||||
res, err := tool.Execute(context.Background(), json.RawMessage(
|
||||
`{"pattern":"Needle","path":`+quote(cache)+`}`,
|
||||
), nil)
|
||||
|
||||
@@ -32,11 +32,6 @@ type Provider struct {
|
||||
// config.ChatConfig.UnrestrictedFilesystem for when that is wanted.
|
||||
unrestricted bool
|
||||
|
||||
// workspaceIndex narrows grep's candidate files. Optional: nil leaves
|
||||
// grep on its direct walk, which is the behaviour when no index is
|
||||
// configured or its build has not finished.
|
||||
workspaceIndex toolsbuiltin.GrepIndex
|
||||
|
||||
// registry holds the constructed tools. Built once in Start so
|
||||
// Discover cannot partially fail.
|
||||
registry *toolsbuiltin.Registry
|
||||
@@ -48,15 +43,8 @@ type Provider struct {
|
||||
//
|
||||
// unrestricted lets the file tools reach absolute paths outside workspace;
|
||||
// relative paths still resolve against it either way.
|
||||
// A workspace index is optional and variadic to match
|
||||
// toolsbuiltin.RegisterBuiltins, which has always taken one the same way.
|
||||
// Only the first is used.
|
||||
func New(workspace string, unrestricted bool, indexes ...toolsbuiltin.GrepIndex) *Provider {
|
||||
p := &Provider{workspace: workspace, unrestricted: unrestricted}
|
||||
if len(indexes) > 0 {
|
||||
p.workspaceIndex = indexes[0]
|
||||
}
|
||||
return p
|
||||
func New(workspace string, unrestricted bool) *Provider {
|
||||
return &Provider{workspace: workspace, unrestricted: unrestricted}
|
||||
}
|
||||
|
||||
// Manifest declares the adapter's tool capability.
|
||||
@@ -84,14 +72,7 @@ func (p *Provider) Start(ctx context.Context) error {
|
||||
toolsbuiltin.SetPathConfinement(!p.unrestricted)
|
||||
|
||||
registry := toolsbuiltin.NewRegistry()
|
||||
// Forwarded only when present. Passing a nil interface would still work
|
||||
// today (grep nil-checks it), but an explicit branch keeps a typed-nil
|
||||
// index from reaching a call on a nil receiver.
|
||||
var indexes []toolsbuiltin.GrepIndex
|
||||
if p.workspaceIndex != nil {
|
||||
indexes = append(indexes, p.workspaceIndex)
|
||||
}
|
||||
if err := toolsbuiltin.RegisterBuiltins(registry, p.workspace, indexes...); err != nil {
|
||||
if err := toolsbuiltin.RegisterBuiltins(registry, p.workspace); err != nil {
|
||||
return fmt.Errorf("workspace tool provider: register builtins: %w", err)
|
||||
}
|
||||
p.registry = registry
|
||||
|
||||
@@ -5,7 +5,6 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
provider "github.com/samcharles93/archie-core/internal/tools/provider/builtin"
|
||||
@@ -234,78 +233,6 @@ func TestUnrestrictedFilesystemReachesOutsideWorkspace(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
// stubGrepIndex records whether the grep tool consulted a supplied workspace
|
||||
// index, and serves a fixed candidate set.
|
||||
type stubGrepIndex struct {
|
||||
mu sync.Mutex
|
||||
calls int
|
||||
pattern string
|
||||
files []string
|
||||
}
|
||||
|
||||
func (s *stubGrepIndex) Candidates(_ context.Context, pattern string, _, _ bool) ([]string, bool) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.calls++
|
||||
s.pattern = pattern
|
||||
return s.files, true
|
||||
}
|
||||
|
||||
func (s *stubGrepIndex) observed() (int, string) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return s.calls, s.pattern
|
||||
}
|
||||
|
||||
// TestGrepConsultsSuppliedWorkspaceIndex covers the wiring this provider is
|
||||
// the only production owner of. internal/tools/builtin.RegisterBuiltins has
|
||||
// always accepted a GrepIndex, and nothing ever passed one, so every grep ran
|
||||
// the direct walk regardless of configuration. A provider that accepts an
|
||||
// index and then drops it on the floor is the exact failure worth a test:
|
||||
// it looks wired from the outside and changes nothing.
|
||||
func TestGrepConsultsSuppliedWorkspaceIndex(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "hit.go"), []byte("package p\n\nfunc Target() {}\n"), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
idx := &stubGrepIndex{files: []string{filepath.Join(dir, "hit.go")}}
|
||||
|
||||
p := provider.New(dir, false, idx)
|
||||
if err := p.Start(context.Background()); err != nil {
|
||||
t.Fatalf("Start: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = p.Stop(context.Background()) })
|
||||
|
||||
entries, err := p.Discover(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("Discover: %v", err)
|
||||
}
|
||||
var grep func(context.Context, map[string]any) (any, error)
|
||||
for _, e := range entries {
|
||||
if e.Name == "grep" {
|
||||
grep = e.Handler
|
||||
}
|
||||
}
|
||||
if grep == nil {
|
||||
t.Fatal("no grep tool discovered")
|
||||
}
|
||||
|
||||
if _, err := grep(context.Background(), map[string]any{"pattern": "Target"}); err != nil {
|
||||
t.Fatalf("grep handler: %v", err)
|
||||
}
|
||||
|
||||
calls, pattern := idx.observed()
|
||||
if calls == 0 {
|
||||
t.Fatal("grep never consulted the supplied workspace index, so the index is decoded, constructed and ignored")
|
||||
}
|
||||
if pattern != "Target" {
|
||||
t.Errorf("index asked for pattern %q, want the caller's pattern %q", pattern, "Target")
|
||||
}
|
||||
}
|
||||
|
||||
// TestWorkspaceToolsCarryTheirOwnIcon pins that each builtin declares an icon
|
||||
// and that the conversion carries it. A chat surface renders what the tool
|
||||
// registered, so an icon missing here is an icon missing on screen.
|
||||
|
||||
@@ -50,8 +50,9 @@ configured_agent_image() {
|
||||
# rollback would then restore that empty file over the database and delete its
|
||||
# -wal/-shm -- destroying the task store it exists to protect.
|
||||
#
|
||||
# Only the top-level key counts. db_path also appears in other sections
|
||||
# ([indexing] owns its own database), and the first match in the file can
|
||||
# Only the top-level key counts. db_path can also appear under a TOML table
|
||||
# (the retired [indexing] section once owned its own database), and the first
|
||||
# match in the file can
|
||||
# easily be one of those.
|
||||
configured_task_db_path() {
|
||||
local base
|
||||
|
||||
Reference in new issue
Block a user