The agent could produce a file -- a transcript, a log dump, a screenshot --
and had no way to hand it over. Asked to send one it offered a "download"
link; nothing arrived and it had no signal that anything had failed. The
defect was not that it lied: success and silent non-delivery were
indistinguishable to it.
Three places made a local file undeliverable, and each is fixed here:
- liveReply.Media skipped any attachment with an empty URL, so a local
file was dropped before it ever reached a sender.
- telegramMediaSender built every attachment as models.InputFileString,
which tells Telegram to FETCH the string as a URL. A host path is not
fetchable, so the send did nothing while reporting success. It now
chooses by what the attachment carries: URL -> fetch, Path -> upload
via models.InputFileUpload over an open file.
- Nothing let the agent ask for a send at all. New send_file tool
(internal/tools/sendfile) validates the file and returns a
MultimodalResult naming it; the turn's channel delivers it.
Meaning changes, called out because the names did not:
- MediaAttachment gains Path, the alternative to URL. Exactly one is
meaningful for an outbound send.
- tools.MultimodalResult.URLs used to mean remote-hosted media only; it
now means media this result wants delivered, which may be local. JSON
name unchanged so existing producers keep working.
- errNoURL is now errNoSource: having neither source, not having no URL,
is what makes an attachment undeliverable.
Failures are visible at every layer, which is the actual fix. send_file
errors on an absent, irregular, unreadable, oversized or confinement-
refused path, so the model sees it and can report it. The sender checks
Telegram's ceilings (10 MB photos, 50 MB otherwise) before the request and
reports the real size rather than leaving a 400 nobody reads. A delivery
that still fails now appends "could not send <name>" to the reply instead
of an empty link, which read as a delivered file.
Exposing a public URL for local files was explicitly rejected: it trades a
missing feature for a hosting and access-control problem.
send_file resolves paths through builtin.ResolveReadable -- the read tool's
own policy, exported rather than reimplemented -- so sending cannot reach
what reading cannot, whatever [chat] unrestricted_filesystem is set to.
53 lines
2.5 KiB
Go
53 lines
2.5 KiB
Go
package tools
|
|
|
|
// MultimodalResult wraps a tool result that includes non-text content
|
|
// (images, audio, video, documents). Summary is the text fed back into
|
|
// the model's context; the actual media is written to files under
|
|
// SubdirHint rather than inlined as base64, since inlining media into
|
|
// the context window would bloat token usage for content the model
|
|
// cannot directly consume anyway.
|
|
type MultimodalResult struct {
|
|
// IsMultimodal is always true for this type; present so a JSON
|
|
// consumer can distinguish a MultimodalResult from a plain string
|
|
// result without type-sniffing.
|
|
IsMultimodal bool `json:"is_multimodal"`
|
|
// Summary is the text description shown to the model -- any text
|
|
// content blocks from the tool result, or a default note when the
|
|
// result is media-only.
|
|
Summary string `json:"summary"`
|
|
// SubdirHint is the directory the media files were written under,
|
|
// relative to the caller's configured media root. Empty when no
|
|
// media root was configured, so the files could not be persisted.
|
|
SubdirHint string `json:"subdir_hint,omitempty"`
|
|
// Files lists the full paths written under SubdirHint, in the order
|
|
// the source content blocks appeared.
|
|
Files []string `json:"files,omitempty"`
|
|
// URLs lists media to deliver to the user's channel. Each ref carries
|
|
// either a remote URL -- a generation API's result, for example,
|
|
// which has nowhere to go in Files because nothing was downloaded --
|
|
// or a local Path the channel uploads.
|
|
//
|
|
// The field used to mean remote-hosted media only; it now means "media
|
|
// this result wants delivered", because a locally produced file has no
|
|
// URL and exposing one for it was rejected. The JSON name is unchanged
|
|
// so existing producers keep working.
|
|
URLs []MediaRef `json:"urls,omitempty"`
|
|
}
|
|
|
|
// MediaRef is one piece of remote-hosted media within a MultimodalResult.
|
|
type MediaRef struct {
|
|
// Type is "image", "video", "audio", or "document" -- the same
|
|
// vocabulary gateway.MediaAttachment.Type uses, so a caller can build
|
|
// one directly from this without translation.
|
|
Type string `json:"type"`
|
|
// URL is where the media is hosted, for media the channel should
|
|
// fetch. Empty when the ref names a local file instead.
|
|
URL string `json:"url,omitempty"`
|
|
// Path is an absolute local filesystem path, for media the channel
|
|
// should upload. Empty when the ref names a hosted URL instead.
|
|
Path string `json:"path,omitempty"`
|
|
// FileName is the name to present the upload under. Ignored for a
|
|
// URL ref, which carries its own name.
|
|
FileName string `json:"file_name,omitempty"`
|
|
}
|