Introducing a new way to review your PRs.

Read the announcement

thedotmack / claude-mem

Persistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More

View on GitHub

47 documents · Updated Nov 20, 2026

docs / deep-search-execution

Deep-search execution

Deep-search execution provides bounded, targeted access to stored memory and raw tool-use evidence without slowing the normal context reader. It combines a local child process for isolated raw searches with a progressive memory lookup that returns an index first and details only for selected records. For the wider delivery path, see Context delivery; related retrieval behavior is described in Search and memory retrieval and Context construction.

Execution boundary

The deep-search boundary separates expensive or sensitive retrieval from ordinary context work. The worker admits one raw search at a time, gives it a fixed time and size budget, and returns an explicit status when the search cannot complete.

Single-flight client

The DeepSearchClient is the local entry point for raw tool-use search. It rejects work when the client is closed, when another child is still active, or when the runtime cannot support a standalone child. A timeout completes the request for the caller, but the client keeps its slot until the operating system confirms that the child has exited. This prevents a late child from overlapping a new query.

The client shares child preparation across requests without keeping timed-out callers attached to the build. It passes only the query and supported search scope to the child. The worker gives this executor to SearchManager, which invokes it only for an eligible local observation search.

References: src/…/DeepSearchClient.ts• L25-39 src/…/DeepSearchClient.ts• L119-145 src/…/SearchManager.ts• L463-498 src/…/worker-service.ts• L1275-1285

Search eligibility

Raw search supplements an ordinary observation search; it does not replace it. The request must be a first-page observation search with a valid query and no filter that the raw tier cannot apply. A successful ordinary search with observations does not run the raw tier, while an ordinary error or an unsupported scope produces a clear non-success status.

The result keeps raw evidence separate from observations. This lets callers distinguish “the raw search was not applicable” or “it timed out” from “the raw search completed and found no matches.” Server search rejects the deep option before sending a local or server search request because this raw tier is local-only.

References: src/…/SearchManager.ts• L463-498 tests/…/mcp-tool-schemas.test.ts• L153-185

Child-process protocol

The client and child use one request and one reply over standard input and output. The protocol carries a version, request identity, parent identity, absolute deadline, database path, query, and supported search options. The reply carries the same version and identity plus a bounded DeepSearchResult.

The parent derives the child deadline from the remaining request budget. The child reads the wall-clock deadline once, then uses a monotonic clock for its own work. This prevents a clock change during startup from giving the child extra time or ending the search too early.

The child opens the database as read-only and does not create a missing file. It checks the raw index before searching. The search applies project, platform, date, and result-order scope, then returns bounded tool-use metadata and snippets rather than full raw bodies. A caller can use the numeric IDs to request selected tool-use bodies later.

The transport has separate request and reply byte limits. The client validates the reply, checks the exit result, reassembles split UTF-8 output, and treats malformed, oversized, mismatched, or abnormal replies as incomplete errors. This keeps untrusted child output from becoming an apparently complete search result.

References: src/…/protocol.ts• L17-40 src/…/child.ts• L6-40 src/…/DeepSearchClient.ts• L65-105 src/…/tool-use-search.ts• L179-228

Process guards

The child has an independent guard that watches both the parent process and the deadline. If the parent dies or the deadline passes while native database work is blocked, the guard kills the child. The parent also requests termination when a search times out or fails, then waits for confirmed process exit before releasing the single-flight slot.

The child keeps this guard armed until its reply has drained from standard output. This lets the transport finish without leaving a live child behind, including on hosts where pending pipe writes need time to complete.

This two-sided cleanup protects the normal reader and the database's read snapshot. It also makes host shutdown wait for the deep-search child instead of leaving a reader behind.

References: src/…/guard.ts• L1-29 src/…/child.ts• L33-40 src/…/DeepSearchClient.ts• L111-117 src/…/DeepSearchClient.ts• L136-145 src/…/worker-service.ts• L1610-1671

Runtime support

Deep search needs a runtime that can start the standalone child through the Bun launcher contract. Source runs build the child program when first needed. Release builds embed the same child program so the installed binary does not need a source tree, package directory, or writable temporary file.

Compiled binaries are checked separately from source execution. An embedded runtime below the supported Bun version is rejected before preparation or spawn because it cannot safely start the child. An unsupported runtime returns unsupported rather than attempting an inline fallback.

References: src/…/child-program.ts• L1-13 scripts/deep-search-build.js• L1-17 src/…/runtime-support.ts• L1-12 src/…/bun-be-bun-support.ts• L1-25

The following sequence shows how a raw query moves from the worker to isolated retrieval and back to the caller.

sequenceDiagram
    participant Caller
    participant Worker as SearchManager
    participant Client as DeepSearchClient
    participant Child as Read-only child
    participant Guard as Child guard
    participant DB as Memory database

    Caller->>Worker: Search with deep enabled
    Worker->>Worker: Run ordinary observation search
    Worker->>Client: Admit eligible raw query
    Client->>Child: Send versioned request over stdin
    Child->>Guard: Start parent/deadline guard
    Child->>DB: Open read-only and search raw index
    DB-->>Child: Bounded raw hits and snippets
    Child-->>Client: Versioned reply over stdout
    Child->>Guard: Drain output, then terminate guard
    Client-->>Worker: Complete result or explicit status
    Worker-->>Caller: Ordinary results plus formatted raw tier

Progressive memory retrieval

Progressive retrieval is the shallow memory path used when a caller needs fast targeted recall. It separates discovery from disclosure: the first step returns bounded index rows, the second adds nearby records around selected anchors, and the third fetches details only for relevant non-prompt records.

Retrieval stages

The ProgressiveSearch flow validates the query and bounds the project, result count, context depth, and detail count before it calls the backend. It ranks titles by query-word overlap. If no title is selected, it may make one normalized query refinement, but it does not perform hidden reasoning or unbounded retries.

When anchors exist, the flow reads nearby timeline context within the anchor project and ranks that context again. It then fetches only selected observation or summary details. Prompt rows can remain in the index, but they are not fetched as memory details.

References: src/…/progressive-search.ts• L15-63 src/…/progressive-search.ts• L193-242

Result shaping

Progressive results report the completed step, the query, the selected project, the operation trace, and a reason when retrieval stops early. The response has a fixed byte budget. The system first reduces the index and then shortens the largest detail, marking shortened content so the caller knows that the evidence is incomplete.

The formatter labels retrieved content as data rather than instructions. It shows the index when no details are selected and gives the caller the next Smart search workflow when shallow title matching is not enough. Backend errors are reduced to safe contract messages instead of exposing database or credential details.

References: src/…/progressive-search.ts• L162-191 src/…/progressive-search.ts• L245-370

Native-memory bridge

The native-memory hook supplements an agent's pending memory or code-search action without replacing the native tool result. It recognizes supported memory tools and memory paths, maps the lookup to the active project and source, and sends only a bounded index to the model. If the worker is unavailable, the native operation continues without supplemental context.

This bridge preserves the same staged recall rule: the model receives index fields first, then chooses timeline context and selected details. It also treats stored memory as evidence, not as instructions to execute.

References: src/…/memory-search.ts• L102-136 src/…/memory-search.ts• L138-183

Result formatting

Deep results are projected into the same text surfaces used by normal search and MCP. Raw hits use a distinct #T identity, show a short tool and project label with a bounded snippet, and state whether more matches exist. The formatter keeps incomplete statuses visible because a timeout or unsupported search is not proof that no raw evidence exists.

For completed raw results, the text points to get_tool_uses with numeric IDs and preserves the project and platform scope. It warns callers not to pass raw IDs to observation or timeline operations. MCP reserves space for this raw block even when ordinary summaries are large, so targeted raw evidence is not silently lost to response-size limits.

References: src/…/deep-search-format.ts• L1-18 src/…/mcp-text-format.ts• L139-184 src/…/mcp-text-format.ts• L213-230

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep the request and reply version, identity checks, byte limits, and result completeness rules aligned between the client and child.
  • Keep the raw-search eligibility rules aligned with the filters that searchToolUses can actually scope.
  • Keep embedded release child content and source-built child behavior equivalent when the child protocol changes.
  • Keep runtime support checks aligned with the Bun self-launch contract used by compiled binaries.
  • Keep raw #T IDs and the get_tool_uses follow-up separate from observation and timeline IDs in every text and JSON projection.
  • Keep progressive index, timeline, and detail budgets aligned with the backend's owner and project scope checks.

Important failure modes

TriggerConsequenceGuard
Raw index is missing, building, disabled, or not readyRaw search returns an explicit incomplete status instead of raw hitsCheck index readiness before the child query and preserve the status in the formatter
Query exceeds the request or search limitsThe raw tier returns invalid_query or the transport rejects the requestBound query terms and request bytes before opening or searching the child
Child exceeds its deadline or the parent diesThe child is killed and the request is incomplete; a live child keeps the slot busy until exitUse the child guard and confirmed-exit cleanup
Compiled Bun runtime cannot start a standalone childDeep search returns unsupported without building or spawningReject unsupported embedded runtimes before preparation
Ordinary MCP text consumes the response budgetRaw evidence could disappear from the caller's viewReserve space for the formatted raw tier and cap both projections
Progressive backend returns invalid, cross-project, or oversized detailsUnsafe or excessive memory content reaches the modelValidate identities, keep timeline context in the anchor project, and truncate within the response budget

Key files

FileWhy to read it
src/…/DeepSearchClient.tsDefines admission, deadlines, child lifecycle, reply validation, and shutdown behavior.
src/…/child.tsShows the child-side request validation, read-only database use, and reply lifecycle.
src/…/guard.tsDefines parent-death and deadline cleanup that protects the host and database reader.
src/…/protocol.tsDefines the transport contract and shared size and time limits.
src/…/tool-use-search.tsExplains raw index readiness, scope-aware search, bounded snippets, and raw hit shape.
src/…/SearchManager.tsConnects ordinary memory search to eligible local raw search and result projection.
src/…/progressive-search.tsDefines staged memory retrieval, validation, ranking, truncation, and safe text output.
src/…/deep-search-format.tsDefines the caller-facing raw evidence format and follow-up ID contract.
src/…/mcp-text-format.tsKeeps deep evidence visible and budgeted in MCP text responses.
src/…/memory-search.tsShows how native memory actions receive bounded supplemental index context.
scripts/deep-search-build.jsShows how release artifacts embed the child program for isolated execution.