Search and memory retrieval
This page covers retrieval orchestration, SQLite, vector, and hybrid search strategies, filtering and pagination, and the server-facing search operations that locate retained memory. Retrieval turns a user query or scope into small, useful sets of observations, session summaries, or prompts for context delivery and later detail reads.
The search surface sits below Context delivery and supports Context construction. The read-only MCP surface is described with MCP recall and tooling.
Retrieval model
Retrieval stages
The retrieval flow separates finding candidates from reading full memory. A request enters the worker search API, its filters are normalized, the orchestrator selects a search path, and the result is shaped for a caller such as context delivery, a timeline view, or an MCP client.
flowchart LR
A[Query or filters] --> B[Worker search API]
B --> C[Search orchestrator]
C --> D{Search path}
D --> E[SQLite keyword search]
D --> F[Vector search]
D --> G[Hybrid file search]
E --> H[Scoped retained memory]
F --> H
G --> H
H --> I[Search index, timeline, or context]
The normal search result is an index of retained memory. A caller can then ask for timeline context or fetch selected records. This staged model limits the amount of memory sent downstream and lets Context construction decide how much retrieved material to include. Search responses can also carry privacy-safe telemetry about the result count and selected strategy without exposing query text or result content.
References: src/…/SearchOrchestrator.ts• L69-123 src/…/SearchRoutes.ts• L163-210 src/…/SearchManager.ts• L464-849
Memory categories
The search system treats observations, session summaries, and user prompts as separate result categories. A category selector chooses which kind of retained memory to search. An observation type filter narrows observations inside that category.
| Category | Retrieval role |
|---|---|
| Observations | Detailed records of work, findings, decisions, and changes |
| Session summaries | Higher-level records of a session and its outcome |
| User prompts | The questions or requests that began work |
The result keeps these categories separate so callers can choose a compact index or combine records for a timeline. The local server also supports a newer project-scoped memory-item store with observation, summary, prompt, and manual item kinds; its search remains project-bound and returns stored memory items.
References: src/…/types.ts• L12-23 src/…/SQLiteSearchStrategy.ts• L41-47 src/…/memory-item.ts• L5-25 src/…/memory-items.ts• L236-259
Search strategies
SQLite retrieval
SQLite retrieval provides deterministic keyword and filter search over the local retained-memory store. Its full-text indexes cover observation text and session-summary text. The path also supports filter-only reads, which is important when a caller asks for records by project, date, type, concept, or file without a query string.
For ordinary text, SQLite uses FTS5 matching and relevance or date order. It uses a substring path when the query contains scripts that the tokenizer cannot split well. If FTS5 is not available, the local path does not pretend that an empty result means that no memory exists; vector search can serve as the other search path when it is configured.
References: src/…/SessionSearch.ts• L95-171 src/…/SessionSearch.ts• L413-478 src/…/SessionSearch.ts• L480-553 src/…/SQLiteSearchStrategy.ts• L25-95
Vector retrieval
Vector retrieval finds records by semantic similarity. The query and retained text use a matching embedding model and vector shape, and the local embedder bounds input size, shares model loading across callers, and returns normalized vectors for similarity search. The vector path then checks returned record metadata, hydrates matching IDs from SQLite, and uses the canonical SQLite rows in the response.
The canonical row also supplies the date used for the semantic date window. This prevents a stale timestamp in the vector index from allowing a row outside an explicit or default date range. The vector path applies a default 90-day window when the caller does not provide a date range. A stored corpus can disable that default because its saved filter defines its full time range. Explicit date bounds replace the default window.
If model or vector service work fails, the worker keeps a keyword path available rather than treating the failure as an empty memory set.
References: src/…/LocalEmbedder.ts• L40-61 src/…/LocalEmbedder.ts• L69-117 src/…/LocalEmbedder.ts• L129-179 src/…/SearchManager.ts• L131-193
Hybrid retrieval
Hybrid retrieval combines the strengths of the two search paths. For a file search, SQLite first defines which records really mention the file and the vector index ranks those records by meaning. The final rows are then read from SQLite, so ranking cannot return a record outside the file and project scope.
For a normal query, the orchestrator uses vector search when it is available and fills empty categories from SQLite. If every requested category has no vector result, it uses SQLite for the whole request. If the vector service is disabled, the request goes directly to SQLite. A vector service error is reported as a degraded semantic search rather than as a false claim that no memory matches. Single-category searches widen the semantic candidate set when filters or date bounds require more eligible rows, then add non-duplicate keyword matches when the semantic work budget is reached.
References: src/…/HybridSearchStrategy.ts• L22-90 src/…/SearchOrchestrator.ts• L94-170 src/…/SearchManager.ts• L131-193 src/…/SearchManager.ts• L556-655 src/…/SearchManager.ts• L1060-1093 src/…/ResultFormatter.ts• L2-8
Scope and result controls
Project and source scope
Project scope is a read boundary, not only a display filter. The search layer accepts one project or a project list, then expands the request to stored key spellings and fully merged project sources. SQLite and vector search use the same expanded scope, so a semantic match cannot be hydrated from a different project.
Comma-separated project text is parsed as a list while a lone key remains exact. The vector filter matches both the row's project and its merged-project marker; this keeps semantic results aligned with the SQLite rows they hydrate. An unscoped request can search across projects, while empty project entries do not widen the read.
The optional platform source keeps memory from different host agents separate. The request normalizes both camel-case and snake-case forms before the strategies run.
References: src/…/project-where-filter.ts• L5-35 src/…/project-read-keys.ts• L9-11 src/…/project-read-keys.ts• L34-89 src/…/project-list.ts• L1-19 src/…/SearchManager.ts• L501-537 src/…/SearchRoutes.ts• L464-501 src/…/SessionSearch.ts• L174-205
Filters and date bounds
The search API accepts query text or at least one row filter. Supported filters include project, platform source, observation type, concepts, files, and a start or end date. A category value such as observations selects a result set; an observation type value narrows the records inside it. The request boundary also accepts common singular, plural, camel-case, and snake-case forms, then gives the search layer one consistent set of filters.
The type input can select a memory category or act as an observation-type alias. Known category values keep their category meaning; other values scope the search to observations and join the observation-type filter. This keeps custom observation types searchable through the same server-facing operation.
Date bounds use inclusive start and end values. File filters match files read or changed by a record. Folder searches add a direct-child check after the broad path match so a parent folder does not return every nested path as a direct match.
References: src/…/SearchOrchestrator.ts• L37-67 src/…/SearchManager.ts• L315-416 src/…/SQLiteSearchStrategy.ts• L25-60 src/…/SessionSearch.ts• L207-257 src/…/SessionSearch.ts• L645-733
Ordering and pagination
Each search path accepts a limit, an offset, and an order. Relevance order favors the match score when an index supports it. Date-descending order is the default at the orchestration layer, while date-ascending order supports history review. Pagination applies the same limit and offset to the selected category search. The server applies the page after semantic supplementation and date merging, so the visible page represents the requested scope rather than only the first semantic candidates.
Vector search retrieves a larger candidate set before applying date and category checks. It hydrates rows before applying the date window, then unions semantic rows with SQLite keyword matches for date-ordered results. The worker removes duplicate IDs, sorts the union by date, and applies the requested page. Single-type searches widen semantic candidates in bounded steps because the vector service does not provide an offset. This keeps semantic ranking useful while preserving the caller's result shape and scope.
References: src/…/types.ts• L6-10 src/…/SQLiteSearchStrategy.ts• L25-39 src/…/ChromaSearchStrategy.ts• L92-147 src/…/SearchManager.ts• L417-462 src/…/SearchManager.ts• L564-655 src/…/SearchManager.ts• L131-193 src/…/SessionSearch.ts• L345-355
Server-facing operations
Worker search operations
The worker exposes one unified search route and focused routes for observations and file lookup. These routes translate HTTP query values into search arguments and return either structured records or a compact formatted index. The same route group also exposes timeline and context operations that use search results to locate the next read.
The main operations are:
GET /api/searchfor category-aware search.GET /api/search/observationsfor observation-focused search.GET /api/search/by-filefor file and folder history.GET /api/timelineandGET /api/timeline/by-queryfor context around a match.GET /api/context/semanticfor a small observation set formatted for context injection.
The unified search operation can request a deep result. It first completes the normal observation search, then adds raw tool-use detail only when the request has a direct query and compatible scope. This keeps broad memory retrieval separate from the more specific Deep search execution path.
The worker creates one SearchManager with the local search, storage, vector, formatting, and timeline services. This gives all routes the same retrieval rules and fallback behavior. The manager also owns the boundary between search results and timeline, recent-session, and optional deep-search responses.
References: src/…/SearchRoutes.ts• L133-175 src/…/SearchRoutes.ts• L178-210 src/…/SearchRoutes.ts• L416-460 src/…/SearchManager.ts• L464-849 src/…/SearchManager.ts• L1060-1119 src/…/worker-service.ts• L725-737
Timeline retrieval
Timeline retrieval turns a query or an anchor into nearby retained memory. Query mode first finds one observation by semantic search and then keyword search if needed. Anchor mode accepts an observation ID, a session ID, or a timestamp. The service then loads nearby observations, summaries, and prompts and trims them to the requested depth.
This operation is the middle step between a compact search index and full record reads. It preserves the time order that a relevance-ranked search does not provide. The result helps a caller understand what led to a match before it requests more detail. Direct timeline requests use a finite depth window capped at 50 records on either side, while query-based timeline search uses the same scoped semantic-to-SQLite hydration and keyword fallback as observation lookup.
References: src/…/SearchManager.ts• L860-1057 src/…/SearchManager.ts• L1248-1323 src/…/SearchRoutes.ts• L189-192
Read-only recall interface
The recall MCP server exposes a small read-only interface for server and remote clients. An injected backend performs the actual scoped search. The MCP layer validates the project and query, clamps the result limit, and returns serialized observations without adding another storage policy.
The interface provides search, context, and recent. search returns matching observations. context returns those observations plus one joined context string. recent lists the newest observations. The backend is responsible for team and project scope, so a recall link cannot read outside the authenticated key's boundary.
References: src/…/recall-mcp-server.ts• L5-13 src/…/recall-mcp-server.ts• L25-37 src/…/recall-mcp-server.ts• L39-85 src/…/recall-mcp-server.ts• L99-133
Recent session context
Recent session context provides a project-bounded view of the latest sessions when a caller needs continuity rather than a text match. It reports completed summaries and the current state of active or incomplete sessions, so context delivery can distinguish finished work from work that still has no summary.
References: src/…/SearchManager.ts• L1122-1246
Change checklist
When these files or sections are changed, remember to consider these:
- Keep project read-key expansion identical in SQLite filters and vector metadata filters. A change in one path can expose different memory for the same project.
- Keep single-key and comma-separated
projectorprojectsparsing aligned across request boundaries. Trimming or dropping values in only one path can change the read scope. - Keep result-category names aligned across request normalization, strategy output, route formatting, and MCP consumers.
- Keep the vector metadata ID and document type mapping aligned with SQLite hydration. A stale mapping can rank a record that cannot be read.
- Keep the embedding model, token limit, vector width, pooling, and normalization aligned between local embedding and the stored vector corpus. A silent mismatch can make all semantic scores unreliable.
- Keep the default vector recency window separate from explicit date bounds and corpus rebuild filters.
- Keep canonical SQLite timestamps as the source for date filtering after vector hydration. An index can contain an old timestamp for a row that has since changed.
- Keep file and folder matching rules aligned with the post-match direct-child check.
- Keep limit and offset behavior consistent when a vector result is hydrated through SQLite. Date ordering must union keyword rows before cutting the page.
- Keep the bounded semantic candidate expansion aligned with the keyword refill path. A cap must degrade to useful keyword results without repeating IDs.
- Keep the deep-search eligibility rules aligned with the normal observation result state. Deep detail must not run for incompatible filters, offsets, or failed observation search.
- Keep timeline depth validation aligned with the work done per returned row. A larger bound increases the amount of history one request can load and render.
- Keep telemetry values limited to counts, booleans, and closed strategy or fallback enums. Search content must not enter the telemetry envelope.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Vector service is disabled or unreachable | Results use keyword search and can miss meaning-based matches | SQLite fallback and a degraded-search message identify the reduced mode |
| Local embedding model cannot load or returns an invalid vector | Semantic indexing or lookup cannot use the vector path | Model loading is shared and retried after a delay; callers retain the keyword or no-vector path |
| Vector search returns no record in a requested category | That category appears empty even when keyword search can find it | The orchestrator supplements empty categories from SQLite |
| A vector index timestamp is stale | A semantic result can pass a date filter even though the current row is outside the requested window | The worker hydrates the row from SQLite and applies the date bounds to its canonical timestamp |
| Project scope is applied differently in vector and SQLite paths | A result can be ranked from one project and hydrated from another | Both paths use the same expanded project read keys |
| A request has neither query text nor a row filter | The server cannot define a meaningful search set | The request boundary returns INVALID_SEARCH_REQUEST |
| FTS5 cannot segment the query script | Normal token search returns no useful match | SQLite uses a substring fallback for unsegmented scripts |
| A folder path matches nested files only | File history can include unrelated deep descendants | Folder search checks direct-child paths before returning rows |
| Date ordering cuts the semantic top-N before date sorting | Newest or oldest matching rows can be missing from a page | The worker merges keyword matches by date before applying the page |
| A filtered semantic search reaches its candidate cap | Some eligible semantic rows remain unseen | The worker adds non-duplicate keyword matches after the bounded expansion |
| Recall MCP receives an empty project or query | The backend could receive an unbounded or useless read | The MCP layer requires both strings and clamps the limit |
| A caller requests deep search with filters or a non-zero offset that raw search cannot preserve | The response could imply that raw detail covers a scope it did not search | Deep search is marked not applicable unless the query and supported scope match the normal observation search |
| A timeline request uses an excessive depth | One request can scan and render an unsafe amount of project history | The worker rejects depths above 50 records before loading the timeline |
Key files
| File | Why to read it |
|---|---|
| src/…/SearchOrchestrator.ts | Defines request validation, strategy choice, and fallback behavior |
| src/…/project-where-filter.ts | Aligns project-list parsing, expanded read keys, and vector project scope |
| src/…/SQLiteSearchStrategy.ts | Shows the local keyword and filter search boundary |
| src/…/ChromaSearchStrategy.ts | Shows semantic candidate retrieval and SQLite hydration |
| src/…/HybridSearchStrategy.ts | Shows scoped file matching with semantic ranking |
| src/…/SessionSearch.ts | Defines SQLite full-text, substring, filter, file, and pagination behavior |
| src/…/LocalEmbedder.ts | Defines the local semantic vector contract and safe model use |
| src/…/SearchRoutes.ts | Maps server-facing routes to search and timeline operations |
| src/…/SearchManager.ts | Connects search results to date-safe hydration, timeline, recent-session, context, telemetry, and optional deep-search reads |
| src/…/recall-mcp-server.ts | Defines the read-only server recall contract |
| src/…/memory-items.ts | Shows project-scoped search for the server memory-item model |