Smart file-read gate
The smart file-read system helps an agent inspect code without loading whole source files into context. It maps file types to language grammars, builds structural views and relevance matches, and uses those results to route large reads toward focused tools while keeping workspace access inside the project.
Source inspection model
The source inspection model turns a file into a language-aware outline. It exposes symbols, imports, signatures, documentation hints, line ranges, and nested ownership while leaving function bodies folded. This gives an agent a low-cost map before it asks for exact implementation text. The model supports common code and document formats through extension-based language detection, with a safe empty result when no grammar is available.
The main inspection actions are:
| Action | Agent-facing result |
|---|---|
smart_outline | Structural view with symbols and signatures. |
smart_unfold | Full text for one named symbol or document section. |
smart_search | Ranked symbol matches plus folded views for relevant files. |
References: src/…/language-map.ts• L1-51 src/…/parser.ts• L20-40 src/…/mcp-server.ts• L797-909
Language-aware parsing
Language-aware parsing selects a grammar from the file extension and extracts the structures that matter for navigation. The result keeps language-specific names and ownership, such as receivers for Go methods, class and method relationships, and namespace ownership for TypeScript declarations, so search and unfold can refer to symbols in the same way an engineer sees them.
The parser also treats non-code source as navigable structure. Markdown headings and code blocks become outline entries, while configuration and query languages expose their main declarations. Unsupported languages remain readable through normal file tools because the smart view does not claim that it can parse them.
References: src/…/language-map.ts• L4-50 src/…/parser.ts• L43-68 src/…/parser.ts• L99-122 src/…/parser.ts• L1078-1115 src/…/parser.ts• L1118-1168
Symbol navigation
Symbol navigation connects a compact outline to focused source retrieval. A symbol can include its owner, documentation comment, export status, and one or more source ranges. When a requested name is not found, the system returns a small ranked list of available names instead of making the agent read the file again.
The navigation flow moves from broad structure to exact code:
flowchart LR
A[Source file] --> B[Language outline]
B --> C[Named symbol]
C --> D[Focused source]
B --> E[Available-name hints]
E --> C
smart_unfold accepts qualified names where ownership matters and returns the source range for the selected symbol. It also supports document sections and symbols whose declaration and implementation use separate ranges. TypeScript namespaces and external modules form owners for their members, so nested symbols can be unfolded through qualified names such as Validation.Inner.deep.
References: src/…/parser.ts• L111-118 src/…/parser.ts• L737-1076 src/…/parser.ts• L1300-1457 src/…/mcp-server.ts• L835-877
Relevance search
Relevance search scans a bounded set of project source files and ranks results by symbol identity before weaker text matches. A query can find a symbol name, signature, documentation text, or file path. The result keeps symbol matches separate from path-only matches so a common path word does not flood the agent's context with unrelated folded files.
Search follows this high-level sequence:
- Walk the workspace while skipping generated, dependency, cache, and hidden tool directories.
- Read only non-empty text files within the size limit.
- Parse files in language groups so the same language work can be processed together.
- Rank symbol matches and retain the most relevant folded files.
- Add path-only matches as compact file entries and report an estimated context cost.
The search path can skip symbol extraction when the native parser is unavailable. It then keeps the file shape needed for path matching while returning no symbol or folded-view results. This preserves file discovery without spending time on parser attempts that cannot succeed. In the normal case, qualified names and language-specific ownership rules keep matches useful for the next smart_unfold action.
Search results therefore support both discovery and the next action. The agent can use a folded result to choose smart_unfold or use a path-only result to request smart_outline.
References: src/…/search.ts• L8-40 src/…/search.ts• L115-171 src/…/search.ts• L178-306 src/…/parser.ts• L1118-1168 src/…/parser.ts• L1300-1385
Native Tree-sitter tooling
Native Tree-sitter tooling supplies the command-line parser and language grammars used by outlines, unfolds, and search. The runtime looks for a package-local executable first, then uses a working compatible executable on PATH when the local file cannot run. It checks the result when needed so a worker can use a newly provisioned or repaired executable without a restart.
The runtime distinguishes an executable that exists from one that can run. It probes the executable with its version command and treats a missing, non-executable, incompatible, or stalled binary as unavailable to smart parsing. Short-lived probe results are cached by executable state, so a changed file time or permission mode is checked again.
The worker provisions the package-local executable when an install did not run dependency lifecycle scripts. It runs the installer in a staging area, verifies the downloaded release against a platform-specific SHA-256 value, checks that the staged executable answers its version command, and moves it into place only after those checks pass. If the release build is verified but cannot run because the host has an older glibc, provisioning can use a pinned glibc-compatible conda-forge build for supported Linux targets. The fallback verifies both the archive and extracted executable before it replaces the package-local file. If no verified build can run, smart parsing stays unavailable and the file-read gate does not route the agent to tools that would fail.
Grammar libraries are built in the application data area when possible. A library is rebuilt after its grammar source changes, and the parser falls back to direct grammar loading when a reusable library cannot be used. Batch parsing groups files by language and runs one query over each group, while a single-file parse uses the same language-specific symbol model. Query output is collected in a temporary owned file so large batches do not lose valid captures to a pipe limit.
References: src/…/tree-sitter-bin-path.ts• L16-45 src/…/tree-sitter-bin-path.ts• L47-72 src/…/tree-sitter-bin-path.ts• L93-117 src/…/tree-sitter-bin-path.ts• L186-237 src/…/tree-sitter-cli-provision.ts• L41-102 src/…/tree-sitter-cli-provision.ts• L151-195 src/…/tree-sitter-cli-provision.ts• L197-317 src/…/tree-sitter-cli-checksums.ts• L1-31 src/…/tree-sitter-cli-checksums.ts• L33-74 src/…/tree-sitter-cli-conda.ts• L1-6 src/…/tree-sitter-cli-conda.ts• L61-101 src/…/tree-sitter-cli-provisioning.ts• L15-36 src/…/parser.ts• L368-521 src/…/parser.ts• L1118-1168
Workspace access boundary
Workspace access is a shared boundary for all smart file tools. A caller path is resolved against the project and checked after symlink resolution, so a path that points outside the project is rejected even when it appears inside the project lexically. Missing targets retain a normal not-found result only after the lexical path passes the boundary check.
This boundary applies before search, outline, and unfold read source content. It also gives the file-read gate the same answer as the smart tools, which prevents the gate from denying a normal read and then directing the agent to a path that the smart tools would refuse.
References: src/…/workspace-path.ts• L13-50 src/…/mcp-server.ts• L823-824 src/…/mcp-server.ts• L853-855 src/…/mcp-server.ts• L894-896
File-read gate
The file-read gate adds prior file observations as context and controls when a whole-file read should be replaced by targeted source inspection. It runs at the synchronous pre-read hook, so it can deny a qualifying read and explain the replacement path to the agent. It fails open when the worker, file check, workspace check, parser availability check, or other required state cannot provide a safe decision.
The gate denies only when all of these conditions hold:
- The request is a qualifying Claude Code main-session read.
- The file is large enough and has prior, still-valid observations.
- The file language is supported for smart inspection.
- The path stays inside the workspace.
- The request is a whole-file read rather than a targeted line read.
- The Tree-sitter CLI is present and loadable.
When the gate denies a read, the response points to smart_outline for structure, smart_unfold for selected source, get_observations for past work, and a targeted Read for exact lines. Small files, unsupported formats, modified files, subagent reads, and targeted reads continue through the normal path.
flowchart TD
A[Agent Read request] --> B[Find prior file observations]
B --> C{Qualifying whole-file read?}
C -- No --> D[Allow read and add context when valid]
C -- Yes --> E{Inside workspace and smart tools loadable?}
E -- No --> D
E -- Yes --> F[Deny whole-file read]
F --> G[Outline symbols]
F --> H[Unfold selected source]
F --> I[Retrieve past observations]
F --> J[Read exact lines]
The hook records which observation timeline it has already injected for a session and file. This avoids repeating unchanged history while allowing a newer observation to appear later. The record is local scratch state and does not leave the device through synchronization.
References: src/…/file-context.ts• L20-45 src/…/file-context.ts• L249-310 src/…/file-context.ts• L313-415 src/…/file-context-dedupe.ts• L1-20 src/…/file-context-dedupe.ts• L127-159 src/…/tree-sitter-bin-path.ts• L186-217
Local runtime relationships
The smart file-read system is a child capability of the local runtime. The MCP server exposes the inspection tools, the hook layer applies the read policy, and the worker provides the runtime support that keeps Tree-sitter usable. The HTTP and job layers are adjacent runtime concerns rather than alternate parsing paths.
Related pages:
- Local runtime and API service
- HTTP API and authentication
- Server jobs and process supervision
- Observation generation and provider runtime
- Interactive viewer client
References: src/…/mcp-server.ts• L797-909 src/…/tree-sitter-cli-provisioning.ts• L15-36 src/…/file-context.ts• L313-415
Change checklist
When these files or sections are changed, remember to consider these:
- Keep the language map, grammar availability, and hook gate's ungated-language policy aligned. A mismatch can route an agent to a tool that cannot produce a view.
- Keep Tree-sitter executable naming, load checks, checksum entries, provisioning, conda fallback metadata, and runtime lookup aligned for every supported platform and package version.
- Keep both the downloaded archive digest and extracted executable digest aligned with the conda-forge asset. A mismatch must stop installation before extraction or execution.
- Keep package-local fallback and
PATHlookup behavior aligned with the parser's actual process launch behavior. A present but unusable local file must not hide a working system executable. - Keep workspace containment checks identical for MCP tools and the pre-read gate. A difference can deny a read that the replacement tool cannot serve.
- Keep parser symbol names, owner qualification rules, and search qualification rules aligned. Search results must remain valid
smart_unfoldtargets, including nested TypeScript namespaces and modules. - Keep grammar-library freshness checks and direct grammar fallback aligned. A stale library can return the wrong symbol view, while a failed build must not disable parsing when direct loading works.
- Keep gate denial guidance aligned with the MCP tool names and arguments. The guidance is the agent's recovery path after a denied read.
- Keep the local injection claim durable and local. Changing its storage or sync boundary can cause repeated context or leak device-only session state.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Tree-sitter binary is missing, incompatible, or fails its version check | Smart views cannot parse and a whole-file read is allowed instead of being stranded behind the gate. | Probe the resolved executable at gate time and provision only a verified working binary. |
A working executable on PATH is hidden by an unusable package-local file | Parsing stays off even though the host has a usable parser. | Check the local file's loadability and fall back to an executable on PATH. |
| Downloaded executable has an unpinned or wrong digest | The executable is not run or installed. | Require a platform/version checksum before the final rename. |
| Conda fallback archive or extracted executable has a wrong digest | The compatibility fallback is rejected before it can replace the package-local binary. | Verify the archive and executable against separate pinned SHA-256 values. |
| A staged executable fails its version check | The package keeps its prior state and smart parsing remains unavailable. | Verify the staged file before moving it into the package path. |
| A verified release build cannot run on the host and no compatible fallback is available | Provisioning reports the host problem and smart parsing remains off. | Use the pinned glibc-compatible build when supported, then fail without installing an unusable file. |
| A grammar library is stale or cannot bind to its language | Symbol extraction can be wrong or fail for a language batch. | Compare the library with grammar source times and fall back to direct grammar loading after a build or bind failure. |
| Source path resolves outside the workspace through a symlink | Smart tools reject the path and the gate does not deny the normal read. | Realpath both workspace and target before containment comparison. |
| File is changed after its latest observation | Stale history is not injected and the read is not gated. | Compare file modification time with the newest observation. |
| A targeted read includes an offset or a limiting window | The agent can satisfy an edit or exact-line need without a full-file denial. | Gate only whole-file reads. |
| Gate database cannot open or concurrent hooks race | Context may be injected again, but the read remains usable. | Fail open and use one atomic local claim. |
| A TypeScript namespace or module is not captured as an owner | Its members appear at file level and qualified search or unfold cannot identify the nested symbol. | Match internal_module and module nodes as namespace symbols and nest their members under them. |
Key files
| File | Why to read it |
|---|---|
| src/…/mcp-server.ts | Defines the agent-facing search, outline, and unfold tools. |
| src/…/parser.ts | Describes the shared source model, language queries, symbol ownership, grammar loading, batch parsing, relevance scoring, and focused unfolding. |
| src/…/search.ts | Shows workspace scanning, relevance ranking, parser-independent path results, and the no-parser search path. |
| src/…/file-context.ts | Defines the pre-read context injection and whole-file denial policy. |
| src/…/workspace-path.ts | Defines the workspace access boundary shared by the tools and gate. |
| src/…/tree-sitter-bin-path.ts | Defines executable lookup, load checks, and fallback to PATH. |
| src/…/tree-sitter-cli-provision.ts | Defines safe native parser provisioning, executable probes, and compatibility fallback. |
| src/…/tree-sitter-cli-conda.ts | Shows how the supported conda-forge package yields a compatible executable. |
| src/…/tree-sitter-cli-provisioning.ts | Shows when the long-lived worker prepares Tree-sitter for the plugin root. |
| src/…/tree-sitter-cli-checksums.ts | Defines the trusted release and compatibility-build digests used during provisioning. |
| src/…/file-context-dedupe.ts | Defines durable, local deduplication for injected file history. |