Introducing a new way to review your PRs.

Read the announcement

affaan-m / ecc

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

View on GitHub

30 documents · Updated Nov 20, 2026

docs / llm-providers-and-model-selection

LLM providers and model selection

The provider layer gives the LLM runtime one common way to use hosted and local language models. It selects a provider from explicit input or saved runtime settings, then lets that provider choose the requested model and translate the common request into its service format. See LLM runtime and provider routing for the parent runtime view and LLM prompts and tool execution for prompt and tool behavior.

Provider abstraction

An LLM provider is an adapter that connects the runtime to one model service. Every adapter exposes generation, model information, configuration validation, a default model, and capability flags through the shared LLMProvider contract.

The runtime exchanges common LLMInput and LLMOutput values with the adapter. This keeps provider choice separate from the rest of the request flow. A ModelInfo record describes whether a model supports tools or vision and can state its token limits.

References: src/…/interface.py• L11-30 src/…/types.py• L97-167

Supported providers

The runtime supports managed APIs, OpenAI-compatible gateways, and a local model service. The provider name is the stable value used by the resolver and configuration.

Provider nameService roleDefault model or model sourceMain configuration need
claudeAnthropic Messages APIclaude-sonnet-4-6ANTHROPIC_API_KEY
openaiOpenAI chat completionsgpt-4o-miniOPENAI_API_KEY
ollamaLocal Ollama serviceOLLAMA_MODEL or llama3.2Reachable OLLAMA_BASE_URL
astraflowUModelVerse global OpenAI-compatible endpointASTRAFLOW_MODEL, or gpt-4o-miniASTRAFLOW_API_KEY
astraflow_cnUModelVerse China OpenAI-compatible endpointASTRAFLOW_CN_MODEL, then ASTRAFLOW_MODEL, or gpt-4o-miniASTRAFLOW_CN_API_KEY
atlasAtlas Cloud OpenAI-compatible endpointATLAS_MODEL or deepseek-ai/deepseek-v4-proATLAS_API_KEY or ATLASCLOUD_API_KEY

Claude and OpenAI publish fixed model catalogs in this layer. The gateway adapters expose the configured default model as their available model entry, so the configured model name remains the routing choice.

References: src/…/types.py• L17-24 src/…/claude.py• L18-56 src/…/openai.py• L27-69 src/…/ollama.py• L23-58 src/…/astraflow.py• L20-23 src/…/astraflow.py• L133-151 src/…/atlas.py• L20-24 src/…/atlas.py• L40-80

Provider resolution

The provider resolver turns a provider name into a configured adapter. It accepts an explicit provider, reads the process environment when no provider is given, and then reads .llm.env; if none of these sources names a provider, it uses Claude.

The resolution order is therefore:

  1. Explicit provider passed to get_provider.
  2. LLM_PROVIDER in the process environment.
  3. LLM_PROVIDER in the current working directory's .llm.env file.
  4. claude as the built-in fallback.

Provider values from environment sources are trimmed and converted to lower case. An unknown value fails before a request is sent. The resolver also permits registration of another adapter under a ProviderType, which extends the routing table without changing the caller contract.

The following diagram shows how provider input becomes an adapter instance.

flowchart TD
    A[Provider request] --> B{Explicit provider?}
    B -->|Yes| D[Normalize provider type]
    B -->|No| C{LLM_PROVIDER set?}
    C -->|Yes| D
    C -->|No| E[Read .llm.env]
    E --> F{Saved provider found?}
    F -->|Yes| D
    F -->|No| G[Use claude]
    G --> H[Resolve registered adapter]
    D --> H
    H --> I[Create provider]

References: src/…/resolver.py• L16-23 src/…/resolver.py• L35-75 src/…/resolver.py• L78-79 tests/test_resolver.py• L50-102

Model selection

Model routing has two levels. The provider is chosen by the resolver, while the model is chosen for each generation request or falls back to the provider's configured default.

Request model precedence

When a request contains LLMInput.model, adapters use that model for the call. When it is absent, the adapter uses its own default. This means a request can route to a different model without changing the selected provider, while provider-specific environment settings control the normal path.

The interactive selector is a separate convenience path. It presents provider and model choices, writes both values to .llm.env, and can also place them in the current process when persistence is requested. The resolver consumes the saved provider, but provider adapters do not read the generic LLM_MODEL value as their default; model defaults come from the adapter-specific settings described below.

References: src/…/types.py• L97-104 src/…/openai.py• L71-75 src/…/claude.py• L58-60 src/…/ollama.py• L60-67 src/…/astraflow.py• L69-73 src/…/atlas.py• L82-86 src/…/selector.py• L94-106 src/…/selector.py• L109-158

Interactive model choices

The command-line selector offers guided choices for Claude, OpenAI, and Ollama. Its lists describe a capability trade-off such as power, speed, cost, or code focus. Astraflow and Atlas remain available through direct provider configuration even though they are not in the selector's default menu.

References: src/…/selector.py• L109-141 src/…/selector.py• L143-158

Provider configuration

Configuration supplies credentials, service endpoints, and default models to each adapter. Constructor arguments take priority over environment values where the adapter supports both. The values below are the provider-specific controls that affect routing.

ProviderCredentialEndpoint overrideModel override
ClaudeANTHROPIC_API_KEYOptional constructor base_urlNo provider-specific model environment value; use the request model or adapter default
OpenAIOPENAI_API_KEYOptional constructor base_urlNo provider-specific model environment value; use the request model or adapter default
OllamaNone requiredOLLAMA_BASE_URL; default http://localhost:11434OLLAMA_MODEL; default llama3.2
AstraflowASTRAFLOW_API_KEYASTRAFLOW_BASE_URL; global UModelVerse endpoint by defaultASTRAFLOW_MODEL; default gpt-4o-mini
Astraflow ChinaASTRAFLOW_CN_API_KEYASTRAFLOW_CN_BASE_URL; China ModelVerse endpoint by defaultASTRAFLOW_CN_MODEL, then ASTRAFLOW_MODEL; default gpt-4o-mini
AtlasATLAS_API_KEY or ATLASCLOUD_API_KEYATLAS_BASE_URL; Atlas Cloud endpoint by defaultATLAS_MODEL; default deepseek-ai/deepseek-v4-pro

Validation checks whether the adapter has the minimum value it needs to attempt a request. For managed providers this is normally an API key. For Ollama it is a non-empty service URL, so validation does not prove that a local service is running.

References: src/…/claude.py• L26-30 src/…/claude.py• L135-142 src/…/openai.py• L27-35 src/…/openai.py• L124-131 src/…/ollama.py• L26-32 src/…/ollama.py• L111-118 src/…/astraflow.py• L39-59 src/…/astraflow.py• L123-130 src/…/atlas.py• L48-72 src/…/atlas.py• L140-147

Capability routing

Capability metadata helps the runtime and callers decide whether a selected model can support a requested interaction. The current catalogs mark Claude, OpenAI, Astraflow, and Atlas models as tool-capable, while the listed Ollama models are not marked as tool-capable; Claude and OpenAI models also expose vision support.

OpenAI-compatible adapters share the same broad request shape, but each gateway still owns its endpoint and model defaults. Claude uses its own message service and handles system content and adaptive thinking according to the selected Claude model. Atlas applies a minimum output-token floor for reasoning models. These differences affect which model is safe for a request even when the common runtime interface is the same.

References: src/…/interface.py• L23-30 src/…/claude.py• L60-85 src/…/openai.py• L36-68 src/…/ollama.py• L33-57 src/…/astraflow.py• L60-67 src/…/atlas.py• L73-80 src/…/atlas.py• L90-97

Error boundaries

Adapters map common request failures into typed LLM errors where the service response makes the cause clear. Authentication failures, rate limits, and context-length failures retain the provider identity so the caller can report or handle the problem without guessing which backend failed.

An empty or filtered response is treated as a failed generation for the OpenAI-compatible adapters that check response choices. Malformed gateway tool arguments are preserved as raw data rather than silently discarded, which lets the higher-level tool path decide how to handle them.

References: src/…/interface.py• L33-60 src/…/openai.py• L83-122 src/…/claude.py• L86-133 src/…/astraflow.py• L82-121 src/…/atlas.py• L99-138

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep the ProviderType values, resolver map, exported provider names, and configuration names aligned. A mismatch can make a provider selectable but not constructible.
  • Keep provider defaults consistent with the selector's model labels when the selector presents that provider. The selector writes the generic LLM_MODEL, but adapters use their own model settings.
  • Update both the global and China Astraflow endpoint or model fallback rules together. The China adapter intentionally falls back to the global model setting.
  • Preserve the ModelInfo capability flags when changing a model catalog. Callers may use them before sending tools or vision input.
  • Keep provider error types tied to the correct ProviderType. The runtime uses that identity to report backend-specific failures.

Important failure modes

TriggerConsequenceGuard
LLM_PROVIDER names a value outside the registered provider typesProvider creation stops with an unknown-provider errorUse one of the resolver's registered names or register a matching adapter
A managed provider has no API keyConfiguration validation fails and a request cannot authenticateSet the provider key or pass it to the adapter
Ollama has a valid-looking URL but no running serviceValidation can pass, then generation fails on connectionCheck the local service at the configured base URL
A request sends tools to a model marked as not tool-capableThe backend may reject the request or return no usable tool callCheck ModelInfo.supports_tools before routing tool work
The selector saves a model only as LLM_MODELThe selected provider may still use its adapter defaultSet the provider-specific model variable or pass LLMInput.model
Atlas receives too little output-token space for reasoningThe answer can be truncated or emptyAllow the adapter's minimum token floor or provide a larger limit

Key files

FileWhy to read it
src/…/interface.pyDefines the provider contract and typed error boundary.
src/…/types.pyDefines provider names, model metadata, and the request field that selects a model.
src/…/resolver.pyExplains provider precedence, default selection, and adapter registration.
src/…/selector.pyShows the user-facing provider and model choice flow and its saved configuration.
src/…/claude.pyShows the Anthropic adapter and Claude model-specific behavior.
src/…/openai.pyShows the OpenAI adapter and its fixed model catalog.
src/…/ollama.pyShows local service routing and local model defaults.
src/…/astraflow.pyShows the global and China OpenAI-compatible gateway variants.
src/…/atlas.pyShows Atlas Cloud routing and reasoning-model token protection.