Introducing a new way to review your PRs.

Read the announcement

thedotmack / claude-mem

Persistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More

View on GitHub

47 documents · Updated Nov 20, 2026

docs / remote-worker-capture

Remote worker capture

Remote worker capture keeps hook events available when the local processing path is unavailable. Deferred events use a durable spool and a remote acceptance boundary, while some live observer events use a bounded direct handoff. The worker accepts deferred events before it starts normal observation or session work, then applies bounded live processing with recovery rules for provider outages. This keeps capture separate from the hook's small time budget and lets later processing use the same ingest path as local capture. See Capture and memory lifecycle, CLI hook event ingestion, Transcript observation processing, Memory ingestion and reinforcement, and Native memory bridge.

Capture boundary

The capture boundary is the point where a hook stops depending on immediate worker availability. Deferred capture accepts observations, file edits, summaries, session ends, and advisor calls as one durable event stream. The event keeps its session identity, source, payload, and capture time so delayed processing can still attach it to the right work. Live observer events can instead use a bounded direct handoff when the hook can reach the worker within its time budget.

Deferred hook-event spool

The hook spool is a file-backed hand-off for short host hooks. Each event has its own atomic file, and its stable name makes repeated delivery replace or identify the same event instead of creating an unbounded duplicate. The spool sorts events by their enqueue time and keeps newer work safe when a drain stops part way through.

The spool is not searchable memory. It is an outbox that protects capture until a worker accepts the event. Accepted events then enter the normal ingest path, which can queue observation generation or session work.

References: src/…/hook-spool.ts• L9-32 src/…/hook-spool.ts• L35-92 src/…/hook-spool.ts• L252-327

Capture-time ordering

The spool carries the time at which the hook saw an event. Worker ingestion uses that time when it links a delayed observation to the prompt that was active during capture. Direct live handoffs carry the same time value. This prevents an outage from moving an old tool action under a later prompt.

Summary and session-end events wait behind retained observations for the same session. An observation can create the session that those closing events need, so this order prevents a temporary outage from stranding the session close.

References: src/…/hook-spool-drain.ts• L17-54 src/…/hook-spool.ts• L362-393 src/…/shared.ts• L1-112 src/…/live-observer-event.ts• L62-73

Remote transport

Remote transport moves the durable outbox from a client process to a worker over HTTP. The client sends one versioned envelope to POST /api/spool/ingest, and the worker acknowledges only after it has written the event to its own spool.

Transport selection

The transport uses the filesystem for loopback workers and HTTP for workers on another host. The setting CLAUDE_MEM_HOOK_SPOOL_TRANSPORT can force either choice. In automatic mode, loopback addresses stay local while other hosts use the remote path. A token is required on both sides before remote capture is enabled. Live observer events select the same local or remote worker boundary, but they do not create a disk outbox.

References: src/…/hook-spool-remote.ts• L6-8 src/…/hook-spool-remote.ts• L56-74 src/…/SettingsDefaultsManager.ts• L168-184 src/…/SettingsDefaultsManager.ts• L400-414

Live observer handoff

The live observer handoff is a short direct request for an event that is already in the hook process. Remote delivery uses the versioned spool endpoint, while loopback delivery uses the session routes for observations and summaries. The hook waits only for a bounded handoff and does not keep a disk copy or retry an ambiguous result, so deferred spooling remains the reliable path when delivery cannot be confirmed.

References: src/…/live-observer-event.ts• L9-18 src/…/live-observer-event.ts• L33-46 src/…/live-observer-event.ts• L48-79

Delivery acknowledgement

The worker validates the protocol version, event shape, and bearer token before it writes anything. It returns the event's deterministic receipt only after the server spool write and its receipt marker succeed. The client removes its copy only when the acknowledgement names that exact receipt, so a lost response or a wrong response leaves the event available for retry.

This boundary gives remote delivery at-least-once transport with duplicate suppression. The worker's receipt marker makes a repeated request harmless even when the client did not receive the first acknowledgement.

References: src/…/HookSpoolRoutes.ts• L17-47 src/…/hook-spool.ts• L121-123 src/…/hook-spool.ts• L166-172 src/…/hook-spool.ts• L319-327

Remote delivery flow

The diagram shows how a captured event crosses the client and worker boundaries before normal memory processing begins.

flowchart LR
    H[Short-lived hook] --> S[Client durable spool]
    S --> U[Detached uploader]
    U -->|versioned HTTPS request| R[Worker ingest route]
    R --> D[Worker durable spool]
    D --> I[Shared worker ingest]
    I --> M[Queued observation or session work]
    U -. retry on timeout or bad acknowledgement .-> S

References: src/…/spool-hook-event.ts• L23-75 src/…/HookSpoolRoutes.ts• L21-46 src/…/hook-spool-drain.ts• L17-54

Upload scheduling

Upload scheduling keeps deferred remote work outside the hook's main action. A hook records the event first, then requests a detached uploader. The hook only waits for the short scheduling attempt; it does not wait for remote ingestion or model work. The separate live handoff path waits for a bounded request instead of starting an uploader.

Burst coalescing

Many events can arrive during one agent turn. The client shares one uploader across a burst and records that a follow-up is needed when another event arrives during delivery. This limits process creation while still giving late events a later upload pass.

Each upload pass has a short transport budget and a bounded batch size. Older entries go first, and entries that do not fit remain in the outbox for the next hook or later retry.

References: src/…/spool-hook-event.ts• L14-21 src/…/spool-hook-event.ts• L30-75 src/…/spool-hook-event.ts• L77-143 src/…/hook-spool-remote.ts• L6-9

Retry scheduling

An unavailable worker, missing credential, failed process launch, timeout, or invalid acknowledgement keeps the client event in place. The next hook can trigger another upload, so the capture path does not need a long-running client daemon. The worker also has its own drain triggers after an event arrives.

References: src/…/spool-hook-event.ts• L23-29 src/…/spool-hook-event.ts• L49-73 src/…/spool-hook-event.ts• L101-143 src/…/spool-hook-event.ts• L145-193

Handoff timeout

The live handoff timeout is capped below the shortest host hook deadline. An invalid or oversized value falls back to the default or to the safe maximum. A timeout or ambiguous result is logged and skipped rather than resent because the worker may already have accepted the event.

References: src/…/live-observer-event.ts• L9-30 src/…/live-observer-event.ts• L33-46 src/…/live-observer-event.ts• L48-78

Worker ingestion

Worker ingestion turns both local spool files and remote spool files into the same capture actions. The worker does not need a separate remote processing model; remote delivery ends at durable acceptance, after which the normal worker ingest functions decide whether to queue an observation, summary, advisor call, or session close. Queue acceptance is not the same as durable memory storage: the live processing buffer may still need a provider run before the event becomes an observation or summary.

Drain triggers

The worker drains at boot, after a remote ingest request, after a local nudge, when the spool directory changes, and during a periodic safety sweep. A single-flight drain prevents concurrent passes. A new request during a pass schedules one more pass instead of running two drains at once.

References: src/…/worker-service.ts• L469-484 src/…/worker-service.ts• L542-555 src/…/hook-spool-drain.ts• L75-105 src/…/hook-spool-drain.ts• L108-157

Ingest hand-off

The drain marks an event when the shared ingest path has accepted it. The marker is written before later generator work begins. If the worker stops after acceptance but before file removal, the next worker removes the marked event without sending it again. If the worker stops before acceptance, the event remains available for a full retry.

References: src/…/hook-spool.ts• L329-347 src/…/hook-spool.ts• L395-466 src/…/SessionStore.ts• L2042-2069

Live processing buffer

The live processing buffer holds accepted work only while the worker can process it. It uses per-session and global bounds, removes expired unsent work, and suppresses repeated tool-use identities during the worker lifetime. The buffer is not a restart-safe backlog; the transcript remains the durable source for recovery workflows.

References: src/…/SessionMessageBuffer.ts• L47-65 src/…/SessionMessageBuffer.ts• L75-105 src/…/SessionMessageBuffer.ts• L267-317 src/…/SessionMessageBuffer.ts• L349-361

Processing recovery

Processing recovery keeps accepted work from blocking newer events when a provider pauses. The session manager preserves pending batches, schedules paced transport resumes, and runs one unattended probe at a time. Provider dispatch can route work to a configured fallback while the selected provider is held. A streamed request is retryable only when it fails before output starts; a failure after output starts is not resent because the provider may have charged for it.

References: src/…/SessionManager.ts• L108-149 src/…/SessionManager.ts• L158-235 src/…/provider-dispatch.ts• L193-223 src/…/provider-dispatch.ts• L487-559 src/…/streamed-chat-completion.ts• L1-23 src/…/streamed-chat-completion.ts• L367-495

Worker memory access

After ingestion stores memory, worker read paths expose observations, summaries, and prompts with project, source, and session context. Search can use semantic results with keyword fallback, while timeline and paged reads keep results bounded and preserve the matched record's project scope. These read paths let delayed capture become part of the same memory view as local capture.

References: src/…/SearchManager.ts• L124-193 src/…/SearchManager.ts• L501-723 src/…/ProgressiveMemorySearch.ts• L26-63 src/…/ProgressiveMemorySearch.ts• L65-127 src/…/PaginationHelper.ts• L58-123

Retention and quarantine

The worker keeps temporary failures for retry and moves malformed or permanently rejected events to corrupt/ instead of deleting them. Events that never become acceptable move to expired/ after the retry window. A session end for a session that never registered has a shorter grace period because it cannot succeed after the worker has confirmed that the session is absent.

References: src/…/hook-spool.ts• L131-146 src/…/hook-spool.ts• L427-455 src/…/hook-spool.ts• L524-547 src/…/hook-spool-drain.ts• L35-50

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep the remote envelope schema, protocol version, receipt value, and worker acknowledgement in sync.
  • Preserve the distinction between the client spool receipt marker and the worker's ingest hand-off marker.
  • Keep transport selection aligned with worker host settings so loopback delivery does not require HTTP credentials and remote delivery does not use an unshared filesystem.
  • Preserve capture timestamps and the ordering rule that keeps summaries and session ends behind retained observations.
  • Keep the live handoff timeout below the host hook deadline and do not turn an ambiguous direct handoff into an automatic resend.
  • Preserve the live buffer limits and its distinction from the durable spool. Do not make accepted in-memory work look restart-safe.
  • Keep provider fallback and stream retry outcomes aligned with paid-send behavior. Do not resend a request after provider output has started.
  • If a new spool kind is added, update its payload validation, receipt behavior, client upload path, worker drain path, and expiry outcome together.

Important failure modes

TriggerConsequenceGuard
Client or worker has no valid ingest tokenThe event stays in the client spool and remote delivery does not startConfigure a safe shared CLAUDE_MEM_WORKER_INGEST_TOKEN on both sides
Network timeout or missing acknowledgementThe client keeps the event and retries it laterUse the exact receipt check and bounded upload timeout
Direct live handoff times out after an ambiguous worker resultThe live event is skipped because it has no durable client outboxUse deferred spool capture when the event must survive an uncertain handoff
Remote payload fails validation or exceeds the accepted sizeThe event is not ingested and is moved to the client's corrupt/ areaValidate the versioned envelope before acknowledging it
Worker stops after ingest accepts an event but before file removalA retry could duplicate capture without a hand-off markerWrite the durable consumed marker before generator work and remove marked files without re-ingest
Live buffer reaches its session or global limitNew live input is skipped instead of creating an unbounded catch-up queueKeep the bounded buffer policy and use the transcript for recovery
Provider fails after streamed output startsA resend could charge twice or create a duplicate resultTreat post-output stream failure as non-retryable
Session end arrives before session creationThe end waits briefly or is later retired while the session remains unregisteredKeep closing events behind observations and apply the unknown-session grace window

Key files

FileWhy to read it
src/…/hook-spool.tsDefines durable event files, ordering, receipts, retry retention, and exactly-once hand-off rules.
src/…/hook-spool-remote.tsDefines the remote protocol, payload limits, token checks, and transport selection.
src/…/spool-hook-event.tsShows how hooks schedule local nudges or detached remote uploads without waiting for processing.
src/…/live-observer-event.tsShows the bounded direct handoff used for live observer events and its local-versus-remote route choice.
src/…/HookSpoolRoutes.tsShows the authenticated remote acceptance boundary and acknowledgement contract.
src/…/hook-spool-drain.tsShows how the worker schedules drains and maps spool entries to shared ingest actions.
src/…/SessionMessageBuffer.tsShows the bounded, non-durable buffer used after accepted live work reaches the worker.
src/…/SessionManager.tsShows queue admission, provider-pause recovery, and paced transport resumes.
src/…/provider-dispatch.tsShows provider selection and fallback when a provider cannot serve capture.
src/…/streamed-chat-completion.tsShows the boundary between retryable pre-output failures and non-retryable post-output failures.
src/…/worker-service.tsShows worker startup, nudge routing, remote spool route registration, and the uploader command.
src/…/SessionStore.tsDefines the durable hand-off markers that protect against duplicate processing across restarts.
src/…/SearchManager.tsShows how stored capture is read through semantic and keyword search with bounded fallbacks.
src/…/ProgressiveMemorySearch.tsShows how observations, summaries, and prompts become one bounded memory view.
src/…/PaginationHelper.tsShows bounded project, source, and session reads for stored capture.