Introducing a new way to review your PRs.

Read the announcement

nousresearch / hermes-agent

The agent that grows with you

View on GitHub

38 documents · Updated Nov 20, 2026

docs / state-session-data-management

Session data management

Session data management keeps each Hermes conversation durable and recoverable across turns, clients, profiles, processes, and restarts. It connects session identity to transcript history, usage records, search views, and live-work coordination without treating any derived or temporary record as the conversation itself.

Durable record model

The session record is the durable envelope for a conversation. It gives downstream systems one stable object from which they can find the transcript, display state, usage, ownership, and recovery information.

Session identity

Each session has a durable session ID that identifies its stored record. Source and origin fields describe where the conversation came from, while a session key can connect a platform chat or gateway peer to the current conversation. The owning profile remains part of the identity boundary, so the same ID is not read through an unrelated profile store.

A session can also point to a parent session. This link describes history lineage rather than simple ownership. Compression continuations extend one logical conversation, while branches and delegated sessions represent separate work and must not inherit routing as if they were the same chat.

References: hermes_state_common.py• L382-447 hermes_state_sessions.py• L330-395 hermes_state_profile_repair.py• L32-47

Transcript records

Message records preserve the ordered conversation used for model context and client display. The record keeps the message role and content with tool, reasoning, platform, and display metadata so a restored session can serve both the model and the user interface.

The durable row identity is separate from message text. Row order uses the stored message ID because timestamps can move backward. A message occurrence also has a stable message identity, and tool calls have their own occurrence identity, which lets replay and delivery repair avoid creating duplicates.

Active, inactive, and compacted flags give the transcript more than one view. Normal resume reads live context, display reads can include compacted history, and audit or export reads can include inactive rows without making those rows part of new model context.

References: hermes_state_common.py• L449-480 hermes_state_messages.py• L267-334 hermes_state_messages.py• L1392-1439

Usage records

Usage records explain the resource cost of a session. The session row carries broad totals for tokens, calls, and cost, while the model-usage records preserve attribution by model, provider, and task.

Main-loop usage updates contribute to session totals and can also build a per-model view. Auxiliary work such as vision, compression, or title generation is kept under a task label, so it can be reported without changing the session's main model route.

References: hermes_state_common.py• L402-419 hermes_state_common.py• L482-502 hermes_state_usage.py• L273-367 hermes_state_usage.py• L369-420

Profile ownership

Profile ownership keeps a session with the profile that owns its credentials, routing namespace, and state store. Gateway session keys carry profile information for routed conversations, while keyless records can remain unowned until a caller supplies a valid owner.

Profile repair identifies rows whose stored label, routing key, parent, or store disagree. It can relabel rows, sever unsafe cross-profile parent links, or move a complete session with its messages and usage to the correct store. The target is written before the source is removed, so an interrupted move leaves recoverable duplicate state instead of losing the conversation.

References: hermes_state_profile_repair.py• L1-20 hermes_state_profile_repair.py• L66-100 hermes_state_profile_repair.py• L180-256

Session lifecycle

Session lifecycle management turns a durable row into a resumable conversation and later makes it eligible for cleanup. The lifecycle preserves history when live work ends, distinguishes intentional boundaries from failures, and keeps lineage visible as one logical conversation where appropriate.

Creation and resumption

Creation records the session before work depends on it. Later callers can safely fill missing details without replacing identity already written by another path. If a child is created from a parent, ownership and workspace metadata can flow down when the child has no explicit value.

Resumption clears the ended state and restores the conversation to active use. A resume target is resolved to the deepest continuation that holds messages, so a caller does not resume an old compression parent while newer history lives in its child.

References: hermes_state_sessions.py• L359-467 hermes_state_sessions.py• L520-585 hermes_state_messages.py• L1471-1510

Lineage boundaries

Compression splits a long conversation into linked session segments. The parent keeps earlier history and the child receives later turns, but retrieval can merge the compression lineage into one transcript. Explicit branches copy history into an independent conversation, so branch reads and usage do not absorb later data from the source.

Intentional reset, branch, delegate, and compression boundaries have different recovery meaning. The boundary type is stored with the session relationship and controls which descendants can inherit routing, which rows belong in a display lineage, and which records can be reclaimed together.

References: hermes_state_sessions.py• L300-323 hermes_state_messages.py• L1392-1417 hermes_state_messages.py• L1520-1536 hermes_state_usage.py• L393-420

End and cleanup states

Ending a session records the first meaningful end reason. A session can later be reopened for resume, while archiving hides old work without removing its history. Automatic orphan handling closes stale state-owned sessions but keeps their messages and makes them resumable.

Pruning is different from ending or archiving. It removes eligible ended sessions and their related records after retention checks. Active work, pinned sessions, guarded sessions, and recoverable lineage remain protected from cleanup.

References: hermes_state_sessions.py• L520-585 hermes_state_maintenance.py• L155-212 hermes_state_maintenance.py• L214-229 hermes_state_maintenance.py• L421-442

Transcript persistence

Transcript persistence makes a turn durable as one coordinated state change. The write path admits the operation against the current session guards, repairs the incoming sequence when needed, stores the rows, and updates session counters together.

This diagram shows how a live turn becomes durable history and then becomes available to later readers.

flowchart TD
    A[Create or resume session] --> B[Read session and transcript]
    B --> C[Acquire turn ownership]
    C --> D[Run agent work]
    D --> E[Append message rows]
    E --> F[Update session totals and usage]
    F --> G[Refresh search and client views]
    F --> H[Resume after restart]

Message write guarantees

Single-message and batch writes use the same transcript rules. A batch either persists its accepted rows together or does not persist them, and message counters follow the rows that were actually inserted. A failed transcript-critical write stops the turn rather than reporting work that cannot be resumed.

Replay repair handles repeated flushes, missing ordering, and tool-call pairing before the rows become the canonical transcript. It preserves inactive history when a later repair or import needs to explain what happened.

References: hermes_state_messages.py• L210-265 hermes_state_messages.py• L376-421 hermes_state_messages.py• L458-492

Compression persistence

Compression changes the shape of a conversation without deleting its meaning. It archives or summarizes older display generations and carries concurrent tail messages into the continuation, so new turns can use a smaller context while the full user-visible history remains recoverable.

Reads choose the view they need. Model restore excludes compacted display history, while display and export paths can merge and deduplicate the lineage so a user sees one coherent transcript.

References: hermes_state_messages.py• L1392-1439 hermes_state_messages.py• L1520-1558 hermes_state_portability.py• L292-317

Retrieval and portability

Retrieval provides several views of the same durable records. Session lists support recent-work discovery, transcript reads support resume and display, search supports content discovery, and export/import supports backup or controlled movement without reviving live runtime ownership.

Session discovery

Session listing returns a display-ready summary with a preview, activity time, lifecycle flags, unread state, and filters for source, workspace, profile-related scope, archive state, and search terms. Compression roots can project to their current tip so the user sees one logical conversation instead of every physical segment.

Session ID lookup also resolves an old compression root to the surfaced continuation. This keeps links and resume commands useful after the conversation has crossed a lineage boundary.

References: hermes_state_sessions.py• L1348-1466 hermes_state_sessions.py• L1549-1572 hermes_state_search.py• L1238-1244

Transcript retrieval

Transcript reads can return one session or its compression ancestors. They support chronological paging, latest-page reads, message anchors, display projections, and model-shaped conversation output. The caller must select the right view because compacted and inactive rows have different meaning for display, audit, and model restore.

References: hermes_state_messages.py• L1392-1439 hermes_state_messages.py• L1452-1469 hermes_state_messages.py• L1520-1536

Content search finds message matches and returns session metadata, a short snippet, and bounded nearby context. It searches canonical message meaning through derived indexes when available, then uses a canonical-row fallback when an index is stale or a query needs data that the index does not cover.

Search results are discovery data, not a second transcript. A missing or degraded index can reduce speed or ranking, but it must not remove the canonical messages or block their normal persistence.

References: hermes_state_search.py• L930-969 hermes_state_search.py• L1008-1018 hermes_state_search.py• L1063-1120

Export and import

Export packages session metadata with message history and timing data. A lineage export can represent several physical segments as one logical session, while inactive and compacted flags can be retained when the goal is backup rather than display.

Import restores history but not live ownership. Existing IDs are skipped, missing parents are detached, cycles are rejected, and routing, handoff, rewind, and live activity state are reset. This makes an imported record safe to resume as data without pretending that the old process or channel still exists.

References: hermes_state_portability.py• L285-344 hermes_state_portability.py• L700-777

Coordination records

Coordination records control access to live work around a durable session. They explain which process may write, compress, route, or deliver work now; they do not replace the session or transcript and can expire without destroying either.

Turn ownership

A turn lease gives one active writer ownership of a conversation across processes. Transcript mutations check the lease when a caller claims one, and stale or expired ownership can be reclaimed and fenced so a late writer cannot append after another process takes control.

A compression lock protects the narrower operation of changing history structure. Ordinary appends can continue during a compression operation, while destructive mutations must respect the active lock and lease.

References: hermes_state_common.py• L568-580 hermes_state_messages.py• L210-265 hermes_state_compression.py• L598-696

Gateway routing

Gateway routing records map an external session key to the session identity and origin needed to find it again. The mapping is refreshed when a gateway handles a peer and can be repaired when routing publication and session creation finish in the wrong order.

Routing metadata follows compression ancestors only when the operation is an explicit continuation. Branches and delegates keep separate routing boundaries so a reconnect cannot send a platform message into another task's transcript.

References: hermes_state_gateway.py• L234-308 hermes_state_common.py• L509-515 hermes_state_sessions.py• L347-357

Liveness and delegation

Gateway heartbeats identify backends that can still own open sessions. Async delegation records keep detached work, its result, and delivery attempts durable until the result reaches the originating conversation or is safely retried.

These records let cleanup distinguish a dead owner from an idle owner. They also let a restarted client recover pending coordination without treating an absent in-memory object as proof that the underlying session data is gone.

References: hermes_state_common.py• L551-566 hermes_state_common.py• L582-610 hermes_state_maintenance.py• L161-175

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep session ID, session key, source, and profile ownership aligned across session rows, gateway routing, parent links, and profile-store selection.
  • Keep compression, branch, reset, and delegation markers aligned with lineage reads, resume resolution, usage aggregation, and deletion scope.
  • Keep message identity, tool-call identity, active/compacted flags, counters, and replay repair consistent across single writes, batch writes, imports, and delivery retries.
  • Keep search projections and fallback behavior dependent on canonical messages rather than making search indexes a second source of truth.
  • Keep exports faithful to history while imports reset routing, activity, and live ownership state.
  • Keep lease, lock, heartbeat, and orphan-sweep rules aligned so cleanup cannot close or mutate live work.

Important failure modes

TriggerConsequenceGuard
A session row has the wrong profile or routing keyResume and profile views can use the wrong store or hide the conversationStamp store ownership, validate key/profile identity, and run profile repair
A replayed or partial message batch is accepted as new historyThe transcript can duplicate turns or lose tool-call pairingUse transactional batch persistence and identity repair before insertion
A compression parent is resumed instead of its message-bearing childNew work can attach to stale context or miss recent turnsResolve resume targets through the compression lineage tip
A search index is stale or damagedSearch can miss or reject matches while canonical history is still soundMark derived search state stale and fall back to canonical message reads
Orphan cleanup cannot distinguish a dead backend from an idle oneCleanup can interrupt live work or leave abandoned sessions openCheck leases, locks, heartbeats, activity age, and pinned state together
Imported history keeps old activity or routing stateA restored record can look live or route into a retired processReset live runtime fields and detach unavailable parents during import

Key files

FileWhy to read it
hermes_state_common.pyDefines the durable session, message, usage, routing, and coordination record model.
hermes_state_sessions.pyExplains session identity, listing, lineage, resume state, and deletion boundaries.
hermes_state_messages.pyShows transcript persistence, message views, replay repair, and compression-aware reads.
hermes_state_usage.pyShows session totals and model/task usage attribution.
hermes_state_search.pyShows content search results and the canonical-row fallback contract.
hermes_state_portability.pyShows export, lineage transfer, import, and live-state reset rules.
hermes_state_gateway.pyShows gateway peer identity and routing persistence.
hermes_state_compression.pyShows turn lease ownership and coordination around history changes.
hermes_state_profile_repair.pyShows profile identity checks and safe cross-store session movement.
hermes_state_maintenance.pyShows orphan closure, retention, pruning, and cleanup protection.
State and persistenceProvides the broader state-store model and links to storage integrity.
State storage integrityExplains database health, derived-index recovery, and safe repair.