Context, memory, and compression
Hermes keeps an agent useful across long tasks by combining live conversation history with project context, user-supplied references, durable memory, and controlled compression. Each source has a different lifetime and cost, so the runtime can preserve important facts without sending every past byte to the model on every request.
Context model
The context model separates information that should remain stable from information that changes during a turn or across sessions. This separation supports both model quality and prompt-cache reuse.
Context layers
The agent builds a request from several context layers. The system prompt carries stable identity and project guidance, the conversation carries the current task, and request-time additions carry material that is useful for one turn or one provider call.
| Layer | Main purpose | Lifetime |
|---|---|---|
| System prompt | Identity, project rules, skills, and memory snapshots | Session |
| Conversation history | User goals, model actions, tool results, and prior decisions | Session |
| Attached context | Files, diffs, folders, commits, URLs, and provider references | Current message and its history row |
| Recalled memory | Relevant facts from built-in or external stores | Current turn or new session |
| Compression summary | A compact record of older work | Until the next summary update |
The system prompt is intended to stay stable during a conversation. Mid-session context that must change is normally added to a user message or tool result. Compression is the controlled exception because it rewrites the conversation history when the prompt becomes too large.
References: agent/prompt_builder.py• L192-241 agent/prompt_builder.py• L1809-1834 agent/context_engine.py• L121-154
Conversation history
Conversation history is the working record for the active task. It includes user requests, assistant replies, tool calls, tool results, and the context that those messages produced.
History lifecycle
Each turn reads the current history, adds the new request, and sends the assembled context to the model. Tool results become part of the history because later model calls may need the evidence that tools returned. At a real session boundary, memory providers can extract durable facts and then bind themselves to the next session.
The history is not the same as persistent memory. History preserves task detail for the active session, while memory keeps a small set of facts that should remain useful after the session ends. Past sessions remain searchable when a fact was not promoted to memory.
The main turn flow and provider request handling are described in Agent turn execution. This page focuses on how the resulting context is retained and reduced.
References: agent/context_engine.py• L181-202 agent/memory_provider.py• L165-178 website/…/memory.md• L59-65
Files and references
Files and inline references let the user add task-specific source material without turning every project file into standing prompt context. They are useful for facts that matter now but may not matter in the next task.
Project context files
Project context files provide standing instructions for work in a directory. Hermes discovers one project-context family in priority order and loads it into the system prompt; SOUL.md is separate and defines the agent identity for the active Hermes home.
The project context is bounded and scanned before it enters the prompt. A large file is reduced to a head and tail with a marker, and a suspicious project file is blocked. Subdirectory guidance can appear later when the agent reaches that directory, which avoids loading the whole repository at session start.
| Context source | Role |
|---|---|
| .hermes.md or HERMES.md | Hermes project guidance |
| AGENTS.override.md or AGENTS.md | Directory and repository rules |
| CLAUDE.md | Compatible project guidance |
.cursorrules and .cursor/rules/*.mdc | Cursor-compatible rules |
| SOUL.md | Global agent identity and tone |
References: agent/prompt_builder.py• L1605-1646 agent/prompt_builder.py• L1669-1737 agent/prompt_builder.py• L1809-1834
Inline context references
Inline @ references attach selected material to a user message. They can provide file text, a line range, a folder listing, Git changes, recent commits, a web page, or a plugin-defined source. Hermes expands the material before the model sees the message, so the attached content becomes part of the conversation history and later compression can summarize it.
Reference expansion uses a soft budget and a hard budget based on the model context length. A large attachment receives a warning or is refused instead of consuming the whole request. Path checks, binary checks, and sensitive-path checks keep an attachment from becoming an unintended credential or workspace escape.
References: agent/context_references.py• L152-248 agent/context_references.py• L287-386 website/…/context-references.md• L65-110
Memory
Memory carries selected facts across session boundaries. Hermes keeps a small built-in store for facts that apply broadly and can add one external provider for richer recall.
Built-in memory
Built-in memory has two stores with different purposes: MEMORY.md holds environment and agent notes, while USER.md holds user identity and preferences. Both are bounded, loaded from the active Hermes home, and placed into the next session's system-prompt snapshot.
Writes update the files at once, but the running session keeps its original snapshot. This preserves a stable prompt prefix. A later session sees the new entries. If a store reaches its character limit, the write fails and the agent must shorten, replace, or remove old entries before it can save the new fact.
Use memory for durable facts that apply across tasks. Use project files for repository rules, skills for task-specific procedures, and session search for detailed past discussion.
References: tools/memory_tool_store.py• L141-159 tools/memory_tool_store.py• L187-218 tools/memory_tool_store.py• L334-369 website/…/memory.md• L11-20
External memory
An external memory provider adds recall and storage beyond the built-in limits without replacing the built-in stores. The provider may add a static prompt block, fetch relevant context before a turn, save completed turns in the background, extract facts at session end, and expose memory tools.
Only one external provider is active at a time. Provider context is treated as reference data rather than as a new user request, and outbound provider data is scrubbed for recognizable secrets before storage or recall. A provider can also checkpoint useful facts before compression so the summary pass does not become the only copy of those facts.
References: agent/memory_manager.py• L391-455 agent/memory_manager.py• L492-557 agent/memory_manager.py• L733-834 agent/memory_provider.py• L116-140
Prompt budgets
Prompt budgets turn a model's context-window limit into decisions about what to load, when to compress, and how much recent history to keep. The active model's window is the source of truth for these decisions.
Context window resolution
Hermes resolves the model context length from explicit configuration, endpoint or provider metadata, learned values, and fallback metadata. An explicit model.context_length value is a pin: it wins over provider information and is shown as pinned so an operator can see that the normal value is overridden.
The resolved window is shared by the main agent and its context engine. When the model or provider changes, the runtime recalculates the usable budget and compression threshold for the new window.
References: agent/model_metadata.py• L2245-2289 agent/context_pin.py• L17-30 agent/context_pin.py• L56-71 agent/context_engine.py• L233-250
Compression thresholds
The built-in compressor triggers from the effective input budget rather than the raw window. The effective budget accounts for any reserved output space, then applies the configured threshold and safe limits for small windows. The default agent threshold is 50 percent, while gateway session hygiene provides a later safety net at 85 percent.
The threshold is a trigger, not a target size. Compression chooses a protected head and recent tail, then replaces the middle with a summary. A per-model threshold can change when compression starts when the active models have different windows.
References: agent/context_compressor.py• L2863-2896 agent/context_compressor.py• L2898-2949 website/…/context-compression-and-caching.md• L83-105
Compression
Compression preserves the active task when the full transcript no longer fits comfortably in the model window. It trades old verbatim detail for a structured handoff while keeping the current work available.
Compression flow
The built-in compressor follows a bounded sequence. The sequence removes low-value bulk first, keeps important boundaries intact, asks an auxiliary model for a structured summary when possible, and then rebuilds a valid history.
flowchart LR
A[Growing conversation] --> B{Budget threshold reached}
B -- no --> C[Send current context]
B -- yes --> D[Prune old tool results]
D --> E[Keep protected head and recent tail]
E --> F[Summarize the middle]
F --> G[Join head, summary, and tail]
G --> H[Continue the same task]
The head protects the system prompt and early task framing. The tail is selected by token budget and keeps recent user and assistant work, with tool-call groups kept together. The middle summary records goals, constraints, progress, decisions, relevant files, and next steps. Later compression updates the prior summary instead of starting with an empty record.
References: agent/context_compressor.py• L5630-5736 agent/context_compressor.py• L5738-5767 website/…/context-compression-and-caching.md• L492-599
Compression continuity
Lean compression keeps a small recent tail and puts more continuity into the summary. It also keeps direct user anchors, identifiers, and a recovery pointer so the agent can find older detail when the summary is not enough. Legacy tail mode keeps a larger verbatim tail instead.
The compressor avoids committing a partial or unsafe summary. A failed summary can preserve the original transcript, wait through a cooldown, retry through a bounded route, or use a deterministic fallback according to policy. Manual compression can bypass the normal failure cooldown, while provider-proven overflow gets a bounded recovery attempt.
Compression normally rewrites the live message list in place under one stable session identity. Older turns remain archived and searchable. Provider-owned runtimes may use native compaction instead, in which case Hermes records the boundary and preserves the provider's opaque checkpoint rather than rewriting the local mirror.
References: agent/context_compressor.py• L4072-4140 agent/context_compressor.py• L4142-4178 website/…/context-compression-and-caching.md• L280-289 website/…/context-compression-and-caching.md• L425-458
Prompt caching
Prompt caching reduces the cost of repeating a stable prefix. Hermes marks the system prompt and recent completed message boundaries for providers that support cache control, while keeping the stored transcript free of request-only cache markers.
Cache boundary
The cache boundary explains why system-prompt mutation is restricted. Identity, project guidance, skills, and memory snapshots are assembled into a stable session prefix. New turn-specific context is appended outside that prefix. Compression is allowed to break the history portion of the cache, but the stable system portion can remain reusable.
Changing the model or provider route changes the provider cache identity and normally causes a cold read. A request-time context selection hook can also change the cache prefix for that request, so engines should keep selections stable when the selected context has not changed.
References: agent/prompt_caching.py• L298-362 agent/context_engine.py• L121-141 website/…/prompt-assembly.md• L265-280 website/…/context-compression-and-caching.md• L659-723
Change checklist
When these files or sections are changed, remember to consider these:
- Changes to system-prompt layers must preserve the stable-prefix boundary and the frozen memory snapshot rule.
- Changes to context-length resolution must update both the displayed window and every compression threshold derived from it.
- Changes to
@expansion must preserve the soft and hard budgets so one attachment cannot crowd out the user request. - Changes to compression summaries must keep tool-call groups valid and retain a recovery path for information removed from the live tail.
- Changes to external memory hooks must preserve the distinction between provider recall, durable writes, and pre-compression checkpoints.
- Changes to cache planning must keep request-only markers out of persisted messages and must account for model or route changes.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A pinned context length is larger than the provider's real window | Requests can reach the provider before local compression starts | The pin is explicit and a one-time disagreement warning identifies the override |
| An attachment exceeds the hard reference budget | The message is sent without the expanded attachment | The expansion reports a warning and leaves the original reference visible |
| Built-in memory reaches its character limit | The requested fact is not saved | The tool returns the current entries so the agent can consolidate or remove stale facts |
| A summary model cannot produce a complete summary | Older history must not be silently replaced by partial text | The compressor preserves the transcript or uses a bounded fallback path |
| Context grows past the normal compressor trigger between turns | The next provider request may fail for size | Gateway session hygiene checks at a higher safety threshold |
| A project context file contains a prompt-injection pattern | Its instructions must not enter the system prompt | Project context is scanned and blocked before injection |
Key files
| File | Why to read it |
|---|---|
| agent/prompt_builder.py | Shows how identity, project files, and memory guidance enter the system prompt |
| agent/context_references.py | Shows how user-selected files, diffs, folders, URLs, and plugin references become bounded context |
| tools/memory_tool_store.py | Shows the bounded built-in memory model and frozen session snapshot |
| agent/memory_manager.py | Shows how built-in and external memory providers cooperate across turns and session boundaries |
| agent/model_metadata.py | Shows how the active model context window is resolved |
| agent/context_pin.py | Shows how an explicit context-window pin is identified and surfaced |
| agent/context_engine.py | Defines the contract for compression and alternate context engines |
| agent/context_compressor.py | Shows the default budget, summary, tail, and failure behavior |
| agent/prompt_caching.py | Shows how provider cache boundaries are applied to request copies |
| website/…/context-compression-and-caching.md | Gives the broader compression and provider-cache model |