Introducing a new way to review your PRs.

Read the announcement

nousresearch / hermes-agent

The agent that grows with you

View on GitHub

38 documents · Updated Nov 20, 2026

docs / state-and-persistence

State and persistence

The shared persistent-state layer gives Hermes one durable record of conversations and runtime work. It keeps sessions, messages, profiles, search indexes, usage data, and coordination records available across turns, clients, processes, and restarts. It also detects damaged or stale state and repairs it without silently discarding the canonical conversation data.

Shared state layer

The state layer is centered on a per-profile SQLite store. SessionDB presents one common boundary to the agent, gateway, CLI, desktop, and maintenance paths, so these callers use the same rules for reads, writes, ownership, and recovery. A store may be opened for observation or for updates, but both modes read the same durable model.

Store boundary

The store is the durable source for user-visible conversation state. It separates canonical records from derived data and keeps profile ownership with the state it serves. This lets a resumed session retain its identity even when the live agent or the process that created it no longer exists.

The process-wide registry shares one writable store per resolved path. When a file is replaced during recovery, the old generation is retired and remains valid only for its current holders. This prevents two physical connections from racing over the same database lifecycle.

References: hermes_state.py• L460-486 hermes_state.py• L579-625 hermes_state_registry.py• L1-29 hermes_state_registry.py• L195-273

Durable data domains

The store joins several kinds of state. Each kind serves a different downstream need, but the session record connects them into one recoverable unit.

Data domainPurposeMain relationship
SessionsConversation identity, source, profile, model route, workspace, lifecycle, and display flagsOwns message and usage history
MessagesOrdered user, assistant, tool, and system historyRebuilds model context and client views
ProfilesOwnership and routing scope for isolated stores and gateway sessionsPrevents one profile from reading or writing another profile's state
UsageToken, call, cost, model, provider, and auxiliary-task totalsSupports status, billing, and insight views
IndexesSearch and display projections derived from canonical rowsSpeeds lookup without becoming the source of truth
Coordination recordsLeases, locks, heartbeats, routing, handoffs, and delegation stateCoordinates active work without replacing the transcript

References: hermes_state_common.py• L372-447 hermes_state_common.py• L449-610 hermes_state_profile_repair.py• L1-20

Session and transcript state

Session rows describe the durable conversation envelope. They hold the stable identity used by gateways and clients, the profile and source that own the conversation, its parent link when history is split, and its lifecycle such as ended, archived, hidden, or pinned.

Message rows hold the ordered transcript and the metadata needed to restore it. Writes are designed to be repeatable: batch persistence repairs message order and identity before rows become visible, while counters and message rows stay aligned. Compression creates related session segments, so a conversation can span a parent and continuation without losing its history.

The state layer also keeps presentation data separate from model-facing content. This allows clients to show reactions, tool results, previews, and hidden records without changing what the model receives when a session resumes.

References: hermes_state_sessions.py• L327-467 hermes_state_messages.py• L120-141 hermes_state_messages.py• L458-492 hermes_state_common.py• L382-480

Usage and profile state

Usage state records both session totals and per-model or per-task attribution. The session summary supports broad reporting, while the detailed rows preserve spend and token ownership when a conversation changes model or uses auxiliary model work.

Profile state keeps a session tied to the store and routing namespace that own it. Profile repair can identify crossed rows, correct labels, sever unsafe parent links, and move a complete session between stores. The move order favors preserving data: the target receives a verified copy before the source is deleted.

References: hermes_state_usage.py• L273-367 hermes_state_usage.py• L369-435 hermes_state_profile_repair.py• L61-100 hermes_state_profile_repair.py• L178-256

Runtime coordination records

Coordination records protect live work around the durable conversation. Turn leases provide one active writer for a conversation, compression locks prevent conflicting history changes, heartbeats identify live backends, and routing records reconnect an external peer to the right session.

These records are not a replacement for the transcript. They explain who may act now and how to recover ownership after a disconnect or restart. Their separation lets a session remain resumable after its live runtime is gone.

References: hermes_state_common.py• L509-580 hermes_state_gateway.py• L234-260 hermes_state_sessions.py• L520-585 gateway-session-lifecycle.md• L88-104

State lifecycle

Persistent state moves through a small set of broad stages: open the profile store, reconcile its shape, read or write canonical records, maintain derived data, and close or recover the store. The same lifecycle serves interactive turns and background work.

Read and write flow

A normal turn reads the session envelope and the visible transcript, performs work, then writes messages and usage through the store boundary. Runtime coordination records admit the write before the turn changes shared state. Readers can continue while a writer commits, and bounded read budgets keep a busy process from exhausting its file descriptors.

This diagram shows how canonical session data supports runtime work while derived views and reports consume the committed result.

flowchart TD
    A[Profile store opens] --> B[Session and transcript read]
    B --> C[Runtime coordination]
    C --> D[Agent or gateway work]
    D --> E[Message and session write]
    E --> F[Usage and coordination update]
    F --> G[Search and client views]
    F --> H[Resume after restart]

References: hermes_state.py• L579-625 hermes_state_messages.py• L389-421 hermes_state_usage.py• L316-337 hermes_state_readpool.py• L140-235

Schema and index lifecycle

On open, the store brings its schema to the current shape and reconciles additions that older stores do not yet have. Full-text search indexes are derived from messages, so they can be rebuilt or detached while canonical message reads and writes remain available.

Search improves discovery but does not define session truth. If an index is stale or unavailable, the store preserves the message tables and uses a safe fallback until recovery can rebuild the derived view.

For the detailed integrity and recovery contract, see State storage integrity.

References: hermes_state_schema.py• L756-785 hermes_state_schema.py• L564-687 hermes_state_common.py• L634-644

Repair and maintenance lifecycle

Repair handles malformed schemas, damaged search structures, unwritable files, and unsafe database generations. It serializes repair work, preserves a forensic copy when needed, uses bounded retry rules, and promotes a repaired snapshot only after it passes health checks.

Maintenance handles the normal cost of long-lived state. It closes stale state-owned sessions, prunes ended history according to retention rules, protects pinned or active work, and runs space reclamation only when the store is safe to rewrite. A failure in optional housekeeping must not block startup or the next user turn.

References: hermes_state_repair.py• L857-890 hermes_state_repair.py• L648-669 hermes_state_maintenance.py• L155-212 hermes_state_maintenance.py• L421-529

Restart and replacement behavior

Restart recovery treats durable evidence as stronger than in-memory assumptions. A stale open session can be closed by a conservative orphan sweep, while a live backend heartbeat or active lease protects its row. A replaced database file causes old handles to stop acting as current writers, and new callers wait until the replacement lifecycle is safe.

This model preserves two priorities: never let an old process overwrite a newer state generation, and never treat a missing live runtime as proof that conversation data should be deleted.

References: hermes_state_maintenance.py• L155-176 hermes_state_common.py• L551-566 hermes_state_registry.py• L276-309 gateway-session-lifecycle.md• L132-158

Child page map

The state layer has focused follow-up pages for the two main ways engineers use it:

PageScope
State storage integrityDatabase health, repair, corruption handling, WAL safety, and recovery limits
Session data managementSession creation, transcript persistence, lineage, import/export, retention, and profile ownership

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep session ownership, profile names, and gateway session-key namespaces aligned across session rows, routing records, and cross-profile repair.
  • Keep message writes, session counters, display projections, and search triggers consistent when the transcript shape changes.
  • Keep usage summary updates and per-model usage rows aligned when model routes, providers, or auxiliary tasks change.
  • Keep lease, lock, heartbeat, handoff, and orphan-sweep rules aligned so cleanup cannot race a live writer.
  • Keep replacement and repair paths compatible with the process-wide registry and with readers holding older database generations.
  • Keep retention and VACUUM rules from deleting active, pinned, or still-recoverable conversation lineage.

Important failure modes

TriggerConsequenceGuard
A session is written without its profile ownershipResume or routing can select the wrong store or fail closedStore ownership stamps, profile-key checks, and crossed-profile repair
A message batch is partially applied or replayedThe transcript can contain duplicates, broken role order, or wrong countersTransactional batch writes, identity repair, and counter updates in the write boundary
Search indexes become stale or corruptSearch misses results or message writes fail through index hooksStale markers, detached fallback, health probes, and admitted rebuilds
A dead backend leaves an open session or leaseCleanup can either leak state forever or interrupt live workHeartbeats, liveness checks, lease expiry, and conservative orphan sweeps
A database file is replaced while handles remain openOld writers can damage the replacement or lose WAL stateGeneration retirement, path teardown barriers, and replacement guards
Retention runs across an active or pinned lineageA live conversation or its recoverable history can disappearWrite-guard checks, pinned protection, and continuation-aware pruning

Key files

FileWhy to read it
hermes_state.pyDefines the shared SessionDB boundary and composes the state capabilities.
hermes_state_common.pyDefines the canonical schema and the shared session-lineage rules.
hermes_state_sessions.pyShows session identity, lifecycle, lineage, and profile-aware metadata.
hermes_state_messages.pyShows transcript persistence, replay repair, and visible-history rules.
hermes_state_usage.pyShows session totals and detailed model/task usage accounting.
hermes_state_gateway.pyShows routing, heartbeats, handoffs, and gateway recovery records.
hermes_state_registry.pyShows shared handles, database generations, and safe teardown.
hermes_state_schema.pyShows schema reconciliation and derived-index recovery.
hermes_state_repair.pyShows corruption checks, backups, repair admission, and promotion safety.
hermes_state_maintenance.pyShows retention, orphan cleanup, and space reclamation policy.
hermes_state_profile_repair.pyShows profile ownership checks and cross-store session repair.