Oxide scanner and native bindings
The Oxide scanner finds project content and extracts the candidate strings that drive Tailwind CSS generation. Rust manages source walks, content preparation, candidate and CSS-variable extraction, and scan state. A Node-API binding makes this work available to JavaScript consumers and provides candidate positions in JavaScript string units.
System role
The scanner connects source selection with CSS generation. It receives the content allowed by source rules and returns unique candidates for the compiler. It also exposes file and glob information for later watch updates. See Content discovery and scanning for the wider flow, Source resolution and globbing for source boundaries, and Node API and compilation for the JavaScript build interface.
The scanner supports both a first scan and later updates. Its state keeps known files and candidates so consumers can scan changed inputs without rebuilding the full candidate set.
Scan lifecycle
The scan lifecycle turns source entries into prepared content, then into candidates and watcher data. Full scans discover files, while update scans accept changed files or content already held in memory.
flowchart LR
A[Source entries] --> B[Selected files]
B --> C[Prepared content]
C --> D[Candidates and CSS variables]
D --> E[Unique candidate set]
E --> F[JavaScript compiler]
G[Changed files or raw content] --> C
B --> H[Files and watcher globs]
Full scans
Scanner::scan walks the configured sources and reads files that pass source and ignore rules. CSS follows a separate path for used custom properties. Other files pass through preparation and candidate extraction. The result is a sorted set of candidates, and the scanner records the files changed during that scan (crates/…/mod.rs• L123-135, crates/…/mod.rs• L375-519).
The scanner keeps candidates across calls. After the first scan, file modification times let later scans skip unchanged files. This supports watch mode without removing candidates that were found earlier (crates/…/mod.rs• L78-93, crates/…/mod.rs• L459-496).
References: crates/…/mod.rs• L78-93 crates/…/mod.rs• L123-135 crates/…/mod.rs• L375-519
Changed content scans
Scanner::scan_content accepts a file path with its extension or content supplied in memory. Known files can be processed at once. New file paths must also appear in the configured source walk, so update events use the same boundary as full scans (crates/…/mod.rs• L138-231).
This path returns only candidates that were not already seen by the scanner. It lets a rebuild process changed inputs while preserving the cumulative candidate set used by downstream CSS generation (crates/…/mod.rs• L228-262).
References: crates/…/mod.rs• L138-231 crates/…/mod.rs• L234-262
Source and watcher data
The scanner also reports the files it knows, the files read during the latest full scan, normalized sources, and optimized globs. These outputs let JavaScript consumers connect source discovery to file watchers without repeating the source walk (crates/…/mod.rs• L264-331).
References: crates/…/mod.rs• L264-331
Content extraction
Extraction is a lexical step. It finds text with the shape of Tailwind candidates and does not decide whether a candidate has a matching theme entry or generate CSS. Those decisions remain downstream.
Candidate classification
The extractor separates valid candidates from CSS variables. Candidate results cover utility and variant forms, arbitrary values, and arbitrary properties when their structure and boundaries are valid. It can inspect nested template expressions without requiring a full parser (crates/…/mod.rs• L24-36, crates/…/candidate_machine.rs• L38-206).
Invalid or incomplete forms are rejected. When a larger expression is not valid, the extractor can recover valid inner bracketed candidates, then removes overlapping results so the outer valid candidate takes precedence (crates/…/mod.rs• L74-145, crates/…/mod.rs• L184-275).
The extractor also limits repeated work on long lines. It reuses positions where candidate scanning is in a clean state and remembers ranges with no valid boundary. This preserves candidate results while keeping minified or generated content practical to scan (crates/…/candidate_machine.rs• L38-206, crates/…/mod.rs• L184-255).
References: crates/…/mod.rs• L24-36 crates/…/mod.rs• L74-145 crates/…/mod.rs• L184-275 crates/…/candidate_machine.rs• L38-206
CSS variable extraction
CSS files provide a signal for custom properties used by generated styles. The scanner does not treat CSS as template content. Instead, it extracts variables used through var(...); definitions alone do not become candidates (crates/…/mod.rs• L234-248, crates/…/mod.rs• L143-175).
This split keeps authored CSS out of ordinary candidate discovery while still passing variable dependencies to the compiler.
References: crates/…/mod.rs• L234-248 crates/…/mod.rs• L143-175
Template preparation
Before extraction, the scanner prepares syntax that can hide class-like text behind template markers. The selected preparation depends on the file extension and covers formats such as Markdown, Vue, Svelte, Ruby, and server-side templates (crates/…/mod.rs• L498-510, crates/…/mod.rs• L542-560).
Haml preparation removes syntax that should not be read as candidate text while keeping class-like text inside Ruby expressions and Haml attributes available to the common extractor. It accounts for Haml comments, multiline Ruby, embedded %w arrays, and class notation with dots. This keeps extraction aligned with the rendered view rather than with the surface syntax alone (crates/…/haml.rs• L12-164, crates/…/haml.rs• L206-279, crates/…/haml.rs• L314-485). The full-file fixture verifies that the prepared result preserves candidates across a realistic Haml view (crates/…/haml.rs• L644-651, crates/oxide/src/extractor/pre_processors/test-fixtures/haml/dst-17051.haml#L1-L262).
Preparation exposes the text needed by the common extractor. It does not change the scanner’s role: the output is still candidate data for downstream CSS generation.
References: crates/…/mod.rs• L498-510 crates/…/mod.rs• L542-560 crates/…/mod.rs• L1-31 crates/…/haml.rs• L12-164 crates/…/haml.rs• L206-279 crates/…/haml.rs• L314-485 crates/…/dst-17051.haml• L1-262
Candidate positions
The scanner can return each candidate with its location in one input. Rust measures byte offsets while reading UTF-8. The Node binding converts those offsets to UTF-16 indexes so JavaScript tools use normal string coordinates (crates/…/mod.rs• L334-372, crates/…/lib.rs• L135-161).
This position query is separate from the cumulative scan result. It reports candidates for the requested file or in-memory content and preserves a compatibility form used during candidate migration (crates/…/mod.rs• L334-372).
References: crates/…/mod.rs• L334-372 crates/…/lib.rs• L135-161 crates/…/utf16.rs• L1-47
Node binding
The Node binding is the boundary between JavaScript consumers and the Rust scanner. It converts JavaScript source and change records into native scanner inputs, owns a scanner instance, and exposes scan results, source data, and candidate positions through Node-API methods (crates/…/lib.rs• L88-191).
JavaScript scanner interface
JavaScript can create a scanner with source entries, request a full scan, submit changed files or raw content, and read file and glob data. The binding keeps these records small: a source describes a base path, pattern, and negation state, while changed content carries either a path or text plus an extension (crates/…/lib.rs• L8-42, crates/…/lib.rs• L111-191).
This interface lets JavaScript coordinate the first build and later file-watch updates without duplicating Rust source and extraction rules.
References: crates/…/lib.rs• L8-42 crates/…/lib.rs• L44-84 crates/…/lib.rs• L88-121 crates/…/lib.rs• L123-191
Native library lifetime
The binding keeps its native library available until process exit when native Rayon worker threads can outlive a Node worker. This protects asynchronous scan work from unloading the code that the workers still need (crates/…/lib.rs• L8-13). WebAssembly builds do not use this native-only retention path.
References: crates/…/lib.rs• L8-13 crates/…/Cargo.toml• L9-17
Native package delivery
The binding is built as a Node native module and depends on the Rust Oxide crate. The crate produces a dynamic library for Node-API and uses the Node-API v4 feature set. Rayon supplies the native parallel work that requires the lifetime guard above (crates/…/Cargo.toml• L1-17). The main package declares optional platform packages for Android, macOS, FreeBSD, Linux, Windows, and WASI. Package-manager platform matching selects a native artifact for the consumer’s operating system, CPU, and Linux C library (crates/…/package.json• L9-32, crates/…/package.json• L63-75, crates/…/package.json• L9-17, crates/…/package.json• L9-29, crates/…/package.json• L1-32).
The build script uses the Node-API setup for normal targets. For WASI, it first points Rust at the sysroot compiler and checks that crt1-reactor.o exists. This build contract prevents a WASM module that compiles but hangs when it loads (crates/…/build.rs• L5-11, crates/…/build.rs• L13-50).
The platform packages publish only the matching native file. The WASI package publishes the WASM file and its Node and browser loader files, with its runtime support bundled as dependencies (crates/…/package.json• L4-10, crates/…/package.json• L25-40).
References: crates/…/build.rs• L5-11 crates/…/build.rs• L13-50 crates/…/Cargo.toml• L1-17 crates/…/package.json• L9-32 crates/…/package.json• L35-61 crates/…/package.json• L63-75 crates/…/package.json• L9-26 crates/…/package.json• L9-26 crates/…/package.json• L9-26 crates/…/package.json• L9-26 crates/…/package.json• L9-26 crates/…/package.json• L9-26 crates/…/package.json• L9-30 crates/…/package.json• L9-30 crates/…/package.json• L9-30 crates/…/package.json• L9-30 crates/…/package.json• L9-26 crates/…/package.json• L9-26 crates/…/package.json• L1-40
Change checklist
When these files or sections are changed, remember to consider these:
- Changes to
ChangedContent, source entries, or glob entries must stay aligned across Rust values, Node-API objects, conversion code, and generated JavaScript types. - Changes to candidate or CSS-variable extraction must preserve the split between template candidates and used CSS variables.
- Changes to extension-based preparation must cover both source selection and the matching template syntax.
- Changes to Haml preparation must preserve candidate text inside Ruby expressions and Haml attributes while excluding comments and control syntax.
- Changes to candidate position units must update the Rust byte-offset contract and the Node UTF-16 conversion together.
- Changes to cumulative scan or modification-time behavior must remain compatible with watcher consumers that process rebuild updates.
- Changes to candidate boundary checks or nested recovery must preserve results while avoiding repeated work on long lines.
- Changes to the main package target list or optional dependencies must stay aligned with every platform package name and its runtime selector.
- Changes to a platform package’s artifact name or runtime constraints must stay aligned with the native file produced for that target.
- Changes to the WASI build path must preserve the sysroot compiler selection, loader files, and
crt1-reactor.ocheck required for a loadable module. - Changes to native worker use must preserve the library-retention rule for Rayon threads.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A changed file is outside the configured walk or is not found | The update is ignored and produces no candidates | Keep change events within the same source rules as the full scan (crates/…/mod.rs• L165-225) |
| A file cannot be read or its candidate bytes are not valid UTF-8 | Its candidate strings are absent from the result | Use readable files and valid UTF-8 when candidate text must be preserved (crates/…/mod.rs• L523-540, crates/…/mod.rs• L591-621) |
| CSS is sent through ordinary template extraction | CSS text can add unrelated candidate-like values | Keep CSS on the used-variable path (crates/…/mod.rs• L485-489, crates/…/mod.rs• L143-175) |
| Haml Ruby or comment boundaries are prepared incorrectly | Classes in view expressions can be missed or Ruby text can be reported as candidates | Keep Haml preparation aligned with its line, indentation, and delimiter rules (crates/…/haml.rs• L118-164, crates/…/haml.rs• L206-279, crates/…/haml.rs• L314-459) |
| Candidate recovery loses clean-state or boundary reuse | Long generated lines can take excessive time to scan | Preserve checkpoint reuse and the remembered no-boundary range (crates/…/candidate_machine.rs• L38-102, crates/…/mod.rs• L199-255) |
| A JavaScript consumer uses a Rust byte offset as a JavaScript index | Positions after non-ASCII text point to the wrong location | Use the binding’s UTF-16 position result (crates/…/lib.rs• L150-159) |
| Native worker code can unload before Rayon threads finish | A scan can access unloaded native code | Retain the native library for the process lifetime on non-WASM targets (crates/…/lib.rs• L8-13) |
| The main package omits a target package or a platform package names the wrong artifact | Installation cannot resolve or load the native binding on that target | Keep optional dependency names, platform selectors, and main artifact names aligned (crates/…/package.json• L63-75, crates/…/package.json• L9-18) |
The WASI build lacks crt1-reactor.o or uses the wrong Rust compiler | The WASM artifact can hang when it loads | Build with the sysroot compiler and install the target runtime files (crates/…/build.rs• L19-50) |
Key files
| File | Why to read it |
|---|---|
| crates/…/mod.rs | Main scan lifecycle, source walk, changed-content path, extraction handoff, and scanner outputs |
| crates/…/mod.rs | Boundary between candidates, CSS variables, prepared content, and nested candidate recovery |
| crates/…/candidate_machine.rs | Candidate validity, boundary rules, and long-line scan reuse |
| crates/…/mod.rs | Template preparation options used before extraction |
| crates/…/haml.rs | Haml syntax preparation and candidate-preserving rules for Ruby and Haml views |
| crates/…/lib.rs | JavaScript-facing scanner methods, native lifetime handling, and Rust-to-JavaScript conversions |
| crates/…/utf16.rs | Position conversion for JavaScript string coordinates |
| crates/…/Cargo.toml | Node-API crate type, Oxide dependency, and native parallel-work dependency |
| crates/…/build.rs | Native and WASI build setup for the Node binding |
| crates/…/package.json | Runtime targets and optional platform package delivery contract |
| crates/…/package.json | WASI artifact and loader delivery contract |