LLM prompts and tool execution
The LLM interaction layer turns a user task into provider-ready messages and turns model tool requests into useful work. It keeps prompt structure, tool descriptions, model calls, tool results, and command-line selection in a shared flow so that different providers can support the same task shape.
Prompt construction
Prompt construction combines caller messages with optional system guidance and tool context. PromptConfig defines this prompt policy, while PromptBuilder produces one shared message shape for the runtime. Callers can provide either a config object or equivalent builder options, which keeps prompt setup explicit (src/…/builder.py• L11-17, src/…/builder.py• L19-54).
The builder preserves an existing system message by joining it with configured guidance and any tool text that belongs in the system message. If no system content applies, it leaves the caller's messages unchanged. This keeps prompt policy separate from the user request while avoiding a blank system message (src/…/builder.py• L56-79).
Provider prompt forms
Provider prompt forms describe the same task in the shape expected by each model service. The provider builder map leaves tool details to native request fields for Claude and OpenAI, while it adds readable tool text to the system message for Ollama-style prompts (src/…/builder.py• L104-117). Provider adapters still own the final translation to their service format.
The shared input model carries messages, model choice, response limits, generation settings, and tool definitions. Provider adapters then serialize that shared input into their own request shape (src/…/types.py• L96-118, src/…/openai.py• L71-83, src/…/claude.py• L58-86).
References: src/…/builder.py• L11-17 src/…/builder.py• L19-79 src/…/builder.py• L104-131 src/…/types.py• L96-118 src/…/openai.py• L71-83 src/…/claude.py• L58-86
Template registry
The template registry provides named prompt text that callers can add, replace, read, or remove at runtime. It acts as a small shared store for reusable prompt content; the builder remains the component that places such content into a message sequence (src/…/__init__.py• L5-7, src/…/__init__.py• L17-40).
Template names and content must be non-empty. A missing name can return a caller-provided default, which lets a workflow keep a safe prompt path when an optional template is not registered (src/…/__init__.py• L9-15, src/…/__init__.py• L34-40).
References: src/…/__init__.py• L5-40 src/…/__init__.py• L3-15
Tool contract
The tool contract gives the model a stable description of an action and gives the runtime a stable way to call it. A ToolDefinition contains a name, a plain-language description, a parameter schema, and a strictness setting. A ToolCall carries the model's selected action and arguments. A ToolResult carries the result back with the call identifier (src/…/types.py• L48-93).
The same definition can be represented as a generic record, an OpenAI function tool, or an Anthropic input schema. This translation keeps provider differences at the boundary while the execution loop works with one internal contract (src/…/types.py• L55-79).
Tool registry
The tool registry connects each public tool name to both its model-facing definition and its callable behavior. Registering a tool makes it discoverable by the prompt path and executable by the runtime (src/…/executor.py• L21-40).
The registry replaces an existing entry when the same name is registered again. This makes the tool name the contract key, so a definition and its callable must remain aligned under that name (src/…/executor.py• L26-28).
References: src/…/types.py• L48-93 src/…/executor.py• L21-40
Tool-assisted execution
Tool-assisted execution is a bounded ReAct loop. ReAct means the model alternates between reasoning through a response and requesting an action. The ReActAgent sends the current conversation to a provider, runs any returned calls, appends the assistant request and tool results, then asks the provider for the next response (src/…/executor.py• L71-117).
The flow ends when the model returns no tool calls. It also ends safely after the configured iteration limit and returns a stop result rather than continuing without bound (src/…/executor.py• L82-122).
This diagram shows how one request becomes a sequence of model turns and tool results.
flowchart TD
A[LLM input: messages and tools] --> B[Build provider-ready messages]
B --> C[Provider generates output]
C --> D{Tool calls present?}
D -- No --> E[Return final LLM output]
D -- Yes --> F[Append assistant tool request]
F --> G[Registry resolves each tool]
G --> H[Execute callable]
H --> I[Append tool results]
I --> J{Iteration limit reached?}
J -- No --> C
J -- Yes --> K[Return max-iterations result]
Tool result handling
Tool execution converts callable output to text so that every result can return to the model as a tool message. An unknown tool or a callable error becomes an error result tied to the original call rather than terminating the whole loop (src/…/executor.py• L43-65).
Multiple calls from one model response run in their returned order. Each result keeps its tool_call_id, which lets the next provider turn associate the result with the requested action (src/…/executor.py• L67-68, src/…/executor.py• L108-117).
References: src/…/executor.py• L43-68 src/…/executor.py• L82-122 src/…/types.py• L27-45 src/…/types.py• L82-93
Provider execution boundary
Provider adapters are the boundary between the shared execution flow and a model service. They receive LLMInput, send messages and optional tools to the selected service, then normalize text, tool calls, usage, and stop information into LLMOutput (src/…/interface.py• L11-30, src/…/types.py• L121-147).
OpenAI-compatible services return function calls with serialized arguments. Claude returns text and tool-use blocks. The adapters normalize both forms into the same ToolCall structure, so the execution loop does not need provider-specific branches (src/…/openai.py• L88-113, src/…/claude.py• L88-124).
Provider failures are classified at this boundary when they match authentication, rate-limit, or context-length conditions. Other failures continue as provider errors for the caller to handle (src/…/interface.py• L33-60, src/…/openai.py• L114-122, src/…/claude.py• L125-133).
References: src/…/interface.py• L11-30 src/…/interface.py• L33-60 src/…/types.py• L121-147 src/…/openai.py• L71-122 src/…/claude.py• L58-133
Command-line interfaces
The Python CLI gives an engineer an interactive way to select a provider and model. It shows the available choices, handles invalid input, saves the selection to .llm.env, and can load the selection into the current process (src/…/selector.py• L52-106, src/…/selector.py• L109-169).
The CLI is a setup surface rather than the tool-execution loop. The resolver later uses an explicit provider, the LLM_PROVIDER environment value, or the saved .llm.env choice when it creates a provider (src/…/resolver.py• L35-75).
The Node.js ecc CLI is the repository's command dispatcher. It maps a command name to a script, forwards arguments, preserves child output, and returns the child exit status. This gives operator commands a common process boundary without mixing their argument parsing into the dispatcher (scripts/ecc.js• L9-20, scripts/ecc.js• L264-339).
Command execution safeguards
The dispatcher supports a global dry-run flag for commands that honor it. It also gives interactive setup and login commands inherited terminal streams, while other commands use captured streams so their output can be relayed consistently (scripts/ecc.js• L134-152, scripts/ecc.js• L212-226, scripts/ecc.js• L269-289).
The Itô command is a separate bridge to an explicitly configured canonical CLI. It filters the child environment and validates the allowed command surface before launch. It is not part of managed LLM inference (scripts/ecc.js• L38-44, scripts/ecc.js• L269-287).
References: src/…/__main__.py• L1-7 src/…/selector.py• L19-25 src/…/selector.py• L52-106 src/…/selector.py• L109-169 src/…/resolver.py• L35-75 scripts/ecc.js• L9-20 scripts/ecc.js• L134-152 scripts/ecc.js• L205-339
End-to-end request path
An end-to-end request begins with a message sequence and optional tool definitions. When the caller applies prompt adaptation, it prepares the sequence for the chosen provider. The provider returns either a final answer or a structured tool request. The runtime executes the request and feeds its result into the next turn until it has a final answer or reaches the loop bound.
The main relationships are:
- Prompt layer: adds reusable system guidance and provider-appropriate tool context.
- Shared type layer: carries messages, tool definitions, calls, results, inputs, and outputs.
- Provider layer: translates the shared request to a service API and normalizes its response.
- Tool layer: maps tool names to callables and converts their output into model-readable results.
- CLI layer: selects provider settings or dispatches operator commands around the runtime.
The parent page, LLM runtime and provider routing, covers provider creation and routing. The sibling page, LLM providers and model selection, covers provider capabilities and model choice.
Change checklist
When these files or sections are changed, remember to consider these:
- Keep the shared
ToolDefinitionfields aligned with every provider serializer. A change to the common schema can alter both OpenAI-compatible and Anthropic requests. - Keep
ToolCallandToolResultidentifiers aligned with the message shape. A mismatch can make a valid tool result unusable on the next model turn. - Keep provider prompt policy aligned with actual provider support. Moving tool descriptions between system text and native fields can change behavior for local text-only models.
- Keep the tool registry name stable across the model-facing definition and callable lookup. Renaming only one side causes a runtime “not found” result.
- Keep CLI provider names aligned with
ProviderTypeand resolver registration. A saved or exported provider name that is not registered cannot create a provider. - Keep bounded-loop behavior aligned with callers that expect either a final answer or a
max_iterationsstop result. - Keep builder configuration entry points mutually exclusive. A caller that combines a
PromptConfigwith keyword options now fails before prompt construction.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A model requests a name not in the registry | The loop appends an error tool result and the model must recover or finish without that tool | Register the definition and callable under the same stable name (src/…/executor.py• L47-55) |
| A registered callable raises an exception | The exception becomes an error result instead of a successful tool result | Treat ToolResult.is_error as part of the tool contract (src/…/executor.py• L56-65, src/…/types.py• L89-93) |
| The model keeps requesting tools beyond the loop bound | The agent stops with max_iterations and does not continue indefinitely | Set and review ReActAgent.max_iterations for the workflow (src/…/executor.py• L71-77, src/…/executor.py• L119-122) |
| A provider returns no usable response choice | The provider raises the shared empty-response error rather than producing a false final answer | Handle provider errors at the request boundary (src/…/openai.py• L83-86, src/…/claude.py• L86-90) |
| A local model lacks native tool support | Tool descriptions can use the text prompt form, but structured tool calls are not guaranteed | Check model capability before relying on tool execution (src/…/builder.py• L104-117, src/…/ollama.py• L33-57) |
A caller passes a PromptConfig and builder keyword options together | Prompt setup fails before a message sequence is built | Choose one prompt configuration form for each builder (src/…/builder.py• L19-54) |
| The CLI receives an unknown command or provider | Dispatch or provider creation fails with an explicit error | Use the command registry and provider resolver as the supported names (scripts/ecc.js• L239-261, src/…/resolver.py• L62-75) |
Key files
| File | Why to read it |
|---|---|
| src/…/types.py | Defines the shared message, prompt input, tool, and output contracts. |
| src/…/builder.py | Shows how system guidance, tool context, and provider policy become provider-ready messages. |
| src/…/__init__.py | Shows the named template registry and its fallback behavior. |
| src/…/executor.py | Shows registry lookup, tool result handling, and the bounded model-tool loop. |
| src/…/openai.py | Illustrates an OpenAI-compatible request and tool-call response boundary. |
| src/…/claude.py | Illustrates a provider with separate system content and tool-use blocks. |
| src/…/selector.py | Shows interactive provider/model selection and saved runtime configuration. |
| src/…/resolver.py | Shows how CLI and environment choices become a provider instance. |
| scripts/ecc.js | Shows the shared Node.js command dispatch and child-process execution path. |