History and prompt import
History and prompt import reads recent local agent histories, keeps prompts typed by the owner, measures agent token use, and passes the result to local memory and optional hosted extraction. It supports both new installs and upgrades where hooks already captured part of the history.
Prompt discovery
Supported history sources
The prompt importer treats each supported agent as a read-only history source. It finds the source on the local machine and converts its native records into one common prompt form. This gives the setup scan one view of prompts across different agents, even when a source uses files or a local data store.
The supported sources are:
- Claude Code
- Codex
- Gemini CLI
- Antigravity CLI
- OpenCode
Each source also reports the sessions that it can see. The scan uses this session view to tell the difference between a prompt missing from a transcript and a prompt that was never in a fully read session.
References: src/…/readers.ts• L17-53 src/…/readers.ts• L495-570
Scan window and source status
The normal scan covers the last 30 days. It selects prompt sources by file age, then reads a nine-day wider lookback for agent text. This wider range helps identify prompts that an agent wrote before the prompt appeared.
The scan reports detected agents, sessions, byte progress, and unreadable sources. A source that cannot be read is skipped and reported so that the other sources can still contribute results.
References: src/…/index.ts• L26-30 src/…/index.ts• L102-137
Owner prompt filtering
Typed prompt records
Each reader keeps interactive user text that represents a prompt typed into the agent. It excludes context, injected instructions, continuations, subagent sessions, and built-in commands when the source marks them as non-user input. The readers also remove generated file context from sources that store it beside the typed text.
Project exclusions apply before prompts enter the scan result. Memory tags are removed before a prompt reaches a consumer. Stable source, session, and native prompt identities remain available for later matching.
References: src/…/index.ts• L141-181 src/…/readers.ts• L184-210 src/…/readers.ts• L269-311 src/…/readers.ts• L414-437 src/…/readers.ts• L461-490 src/…/readers.ts• L532-550
Agent-written prompt exclusion
The owner-prompt filter removes text that the agent wrote first. It compares suitable prompt openings with earlier agent text and rejects other patterns that show the text came from an agent rather than the owner.
This check is a separate pass over agent text. The split lets the importer read large histories as streams while still applying the same ownership rule to transcript prompts and hook-recorded prompts. Agent text includes model replies and model-created tool input when the source exposes it.
References: src/…/index.ts• L158-180 src/…/readers.ts• L212-231 src/…/readers.ts• L313-333 src/…/readers.ts• L439-450 src/…/readers.ts• L552-565 src/…/setup-scan.ts• L321-349
Token usage accounting
Agent usage totals
The token scan reads usage records from the same local histories and time window as prompt discovery. It counts each agent's usage once, handles repeated or running totals, and excludes claude-mem observer sessions from the person's usage.
Antigravity CLI history has no readable token records. It can still be detected and contribute prompts, but it does not contribute to token totals. The result also records whether history reaches the start of the scan window and whether retention is long enough to support the full window.
References: src/…/token-usage.ts• L1-27 src/…/token-usage.ts• L250-279
Coverage and usage labels
The scan marks usage as complete only when the available records support the full window without read failures or short retention. Partial or missing coverage prevents a usage persona from being shown.
The persona uses total token bands. Efficiency uses claude-mem observation economics only when observation history has a positive measured result. These two measures describe different parts of the system: agent use comes from local histories, while efficiency comes from local memory observations.
References: src/…/token-usage.ts• L41-53 src/…/token-usage.ts• L294-316
Setup scan
Scan lifecycle
The setup scan coordinates prompt reading, agent-written checking, local saving, token accounting, and optional extraction. Only one scan runs at a time. A second start joins the current state instead of starting another scan.
The scan follows this sequence:
flowchart LR
A[Find local histories] --> B[Read prompts]
B --> C[Check agent-written text]
C --> D[Save owner prompts locally]
D --> E[Measure token use]
E --> F[Wait for upload choice]
F --> G[Extract selected prompts]
G --> H[Report the result]
Runner detection is separate from history detection. A history can be readable even when its installed agent cannot run setup extraction. The scan waits for runner detection, but runner detection does not decide whether local history can be read. If no extractor is available, the scan finishes after the local work without an upload choice.
References: src/…/setup-scan.ts• L18-21 src/…/setup-scan.ts• L198-209 src/…/setup-scan.ts• L268-297 src/…/setup-scan.ts• L354-364
Scan results
The result shows prompt counts by agent. It separates prompts already recorded by hooks from prompts found in transcripts. It also includes recent prompts, skipped sources, token usage, extraction readiness, and scan or usage errors.
On an upgrade, the scan matches transcript prompts to hook-recorded prompts within a time window. A match is counted once and is not stored twice. A typed prompt whose transcript is missing can still join extraction from local memory when the session was not fully read.
References: src/…/setup-scan.ts• L23-67 src/…/setup-scan.ts• L299-353
Upload consent
When an extractor is available and owner prompts exist, the scan holds the filtered prompt set in memory and waits for an explicit upload choice. The person can allow all prompts or select agents. A declined choice sends an empty extraction request so setup can finish while the prompts stay on the machine.
The choice also carries the optional product-improvement setting. A second scan joins the waiting or uploading state instead of starting another upload. If hosted extraction is temporarily unavailable, setup can finish locally and report that automatic organization was not completed. An upload error that cannot be handled locally leaves the scan in failed with the service error.
References: src/…/setup-scan.ts• L145-166 src/…/setup-scan.ts• L221-265 src/…/setup-scan.ts• L354-368
Import handoff
Local prompt memory
The scan hands owner prompts to local memory with their source, session, project, native identity, text, and transcript time. Local memory uses this identity and the hook match window to make rescans safe and to connect transcript records to prompts already captured by hooks.
After local saving, the scan builds one prompt set from transcript prompts and owner prompts that exist only in memory. Filtering, tag stripping, and redaction happen before this set reaches an extraction consumer.
References: src/…/setup-scan.ts• L72-80 src/…/setup-scan.ts• L299-346
Hosted extraction handoff
Hosted extraction is separate from local saving. It receives only the prompts allowed by the upload choice. A declined choice keeps source prompts local and completes extraction with an empty set. A temporary hosted-service limit can complete setup locally without sending a second request.
References: src/…/setup-scan.ts• L221-265 src/…/setup-scan.ts• L354-364
Existing observation memory
The separate history import sends this machine's observations to the remote memory service for a connected CMEM Pro account. It uses small batches and reports new versus already-known rows. Sessions, summaries, and prompts remain local during this handoff.
The operation is safe to retry because the remote service uses stable observation identities. A missing account token blocks the operation before it starts. A refused batch stops the run with the server reason.
References: src/…/history-import.ts• L1-10 src/…/history-import.ts• L81-90 src/…/history-import.ts• L100-137
Related systems
The wider capture and memory model is described in Capture and memory. For nearby flows, see Hook event ingestion, Transcript observation processing, Memory ingest and reinforcement, Native memory bridge, Remote worker capture, and Media artifact lifecycle.
Change checklist
When these files or sections are changed, remember to consider these:
- Keep each reader's normalized prompt identity aligned with hook matching and rescan deduplication.
- Keep source names, agent labels, and runner mappings aligned across discovery, scan results, and extraction readiness.
- Keep token coverage rules aligned with history retention and read-failure signals before showing a persona.
- Keep upload consent aligned with the rule that non-selected prompt data stays on this machine.
- Keep local prompt and session handling separate from observation upload behavior.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A history source cannot be read | That source is omitted and usage may become partial | Record the skipped source and continue with other sources |
| A prompt was written by an agent | Agent text can enter owner goals or extraction | Compare prompt text with earlier agent text before handoff |
| A hook prompt has no readable transcript copy | The prompt can be lost or counted incorrectly | Keep eligible memory prompts when the session is missing or incomplete |
| History retention is shorter than the scan window | Token totals cannot support a full-window persona | Mark coverage partial and omit the persona |
| Upload consent is declined or narrowed | Unselected prompts do not reach hosted extraction | Send only selected agents and keep the rest local |
| Hosted extraction is temporarily unavailable | Setup cannot organize prompts remotely | Finish the local setup and show a hosted-service notice |
| Remote observation import lacks a token or rejects a batch | Existing observations are not uploaded | Require CMEM Pro before start and stop with the server error |
Key files
| File | Why to read it |
|---|---|
| src/…/index.ts | Defines the scan window, common prompt model, filtering order, and source status. |
| src/…/readers.ts | Defines the supported history sources and their common reader boundary. |
| src/…/owner-prompt-filter.ts | Defines the owner-versus-agent-written decision. |
| src/…/token-usage.ts | Defines token totals, coverage, personas, and efficiency reporting. |
| src/…/setup-scan.ts | Coordinates discovery, local saving, consent, token accounting, and extraction handoff. |
| src/…/history-import.ts | Defines the separate observation upload used to bring existing memory to CMEM Pro. |