Diagram verification suite
The diagram verification suite checks that shipped diagrams preserve their intended meaning, remain readable, and stay safe to import. It combines diagram-family checks with shared checks for geometry, accessibility, motion, screenshots, imports, and documentation routes. See Quality assurance for the wider quality model and CI and Pages automation for the checks that run in automation.
Verification model
The suite treats each diagram as a visual contract. A check passes only when the source declares enough information for the checker to compare the intended relationship with the mark that a reader sees.
Contract layers
The checks answer four questions:
- Family checks ask whether the diagram uses its defining visual encoding. A box plot must place its statistics correctly. A unit grid must keep its cells and totals in agreement. A schedule network must keep its task calculations and precedence routes in agreement.
- Cross-cutting checks ask whether shared risks change the reading. These risks include label overlap, unclear connector routes, weak contrast, unsafe SVG content, stale screenshots, and unusable motion.
- Import checks ask whether Mermaid, draw.io, and Excalidraw input becomes a useful and safe description. Imported labels remain data. They do not become instructions or active links.
- Repository checks ask whether examples, references, commands, package files, and support files still reach one another.
Most checks read source markup. Render checks are the exception. They use a pinned browser on the Linux CI leg to check layout that source arithmetic cannot prove. The regression scripts use adversarial inputs so a checker fails when it cannot read a diagram instead of passing an unchecked file.
The following flow shows how a diagram moves through the suite.
flowchart LR
Source[Diagram source or import input] --> Family[Family contract]
Family --> Shared[Geometry, contrast, accessibility, motion, and import checks]
Shared --> Evidence[Rendered layout and screenshot evidence]
Evidence --> Result[Local or CI result]
References: scripts/verify-architecture-containment.py• L623-877 scripts/verify-heatmap.py• L339-532 scripts/verify-mermaid-import.py• L1652-1687 scripts/verify-schedule-network.py• L264-353
Adversarial regression fixtures
The regression scripts test the checkers as contracts. Each script runs valid shipped examples and then applies focused changes that should either remain valid or produce a named finding. This guards against false passes and rules that reject valid diagram forms.
- The beeswarm tests cover value-scale recovery, legal dodging, overlap, radius and ink rules, labels, ticks, transforms, budgets, scope, sibling skips, and fixture isolation.
- The box-plot tests cover shipped light, dark, and full examples. Their cases protect quartile and whisker geometry, medians, Tukey fences, outliers, row alignment, labels, focal limits, CSS and SVG movement, hidden or non-rendered marks, selector resolution, paper masks, checker scope, sibling skips, and fixture isolation. They also check inherited styles, hidden attributes, and alternate color forms that can change browser-visible paint.
- The exploded tests cover projected silhouettes, level spacing, leader routes, labels, focal parts, traces, motion lift, transform rejection, CSS declarations, and numeric path forms.
- The geometry tests cover label masks, translated panels, definition containers, connector shapes, ports, forks, trunks, curves, and fail-closed path parsing.
- The documentation-sync tests cover shared font links, title fallback, type ramps, size presets, routing links, packaged support paths, type discovery, export wording, gallery parity, asset citations, and boundary values.
- The heatmap tests cover grid completeness, monotone opacity, focal-cell rules, contrast, transforms, quoted markup, large finite values, and invalid values.
- The service-blueprint tests cover layer order, the visibility line, stage columns, cell census, focal treatment, printed labels, and geometry-changing CSS.
- The event-modeling tests cover ordered stages, typed roles, visible event labels, link endpoints, arrow geometry, lane order, and stage columns. They reject hidden content, unsupported movement, covering marks, and malformed CSS selectors.
- The requirement-traceability tests cover source references, requirement rows, typed links, explicit coverage gaps, visible labels, cell placement, page layout, and source citations. They reject hidden or clipped evidence and stale references.
- The schedule-network tests cover an independent critical-path calculation, task fields, critical status, precedence links, visible task boxes, route ports, graph validity, rank layout, summaries, and shipped light, dark, and full examples. They also reject hidden marks, malformed paths, unsupported links, and changes that make browser-visible text disagree with the calculated schedule.
- The SBGN tests cover typed glyphs and arcs, compartment ownership, route geometry, crossings, budgets, shipped variants, safe parsing, discovery, and documentation values. They preserve legal exceptions such as legends and bridged crossings.
- The steps tests cover rail continuity, equal stage sizing, stage census, focal treatment, centered medallions, printed numbers, documented six- and seven-stage layouts, transforms, hidden marks, stylesheet effects, and rendered overlays.
- The unit-grid tests cover cell order, totals, printed summaries, color tokens, geometry, and shipped variants. They reject duplicate or missing cells, hidden or moved content, stylesheet paint changes, overlays, and invalid boundary values.
- The packaged self-check tests cover complete documents, markup balance, safe references, view-box bounds, motion controls, and offline font behavior.
References: scripts/test-verify-beeswarm.py• L225-328 scripts/test-verify-box-plot.py• L279-362 scripts/test-verify-event-modeling.py• L20-207 scripts/test-verify-requirement-traceability.py• L25-226 scripts/test-verify-schedule-network.py• L137-402 scripts/test-verify-sbgn.py• L50-547 scripts/test-verify-steps.py• L49-293 scripts/test-verify-unit-grid.py• L196-302 scripts/test-self-check.py• L55-399
Diagram family verification
Each family checker protects a failure that can look correct on screen while changing the reader’s conclusion. A path argument supports focused work on one asset. An all-files mode checks the shipped examples for that family.
Quantitative family contracts
Quantitative checkers compare declared values with drawn size, position, thickness, or fill. They fail closed when the family has too few readable marks or uses a construct the checker cannot understand.
| Family | Meaning protected | Main checks |
|---|---|---|
| Treemap | Area represents share of a whole. | Cell area agrees with declared share. Labels and markers stay inside their cells. |
| Marimekko | Width, height, and area represent related shares. | Columns tile the plot. Segment shares, order, labels, keys, and markers agree. |
| Dumbbell | A pair comparison uses a fair value range and visible marks. | The domain remains valid for positive, negative, zero-touching, and all-zero data. |
| Slopegraph | A slope compares two states on one scale. | Axes share a scale. Endpoints, names, and captions stay bound to their data. |
| Sankey | Ribbon width represents conserved volume. | Node flow balances. Ribbon width and stage totals agree. |
| Ridgeline | Ridge height supports comparison across rows. | Ridges share amplitude, bins, baseline pitch, labels, and ranges. |
| Bubble | Position and area encode values. | Shared scales, area, paint order, labels, and ticks agree. |
| Bump chart | Vertical position represents ordinal rank. | Vertices use one rank grid. Each snapshot is a complete rank permutation. |
| Beeswarm | One axis carries value. The other packs dots. | Values share a scale. Dots do not overprint or add hidden encodings. |
| Box plot | Quartiles, whiskers, medians, and outliers describe a distribution. | Declared statistics agree with marks, rows, labels, focal treatment, and visible structure. |
| Heatmap | Fill opacity gives a monotone reading over a categorical grid. | Every row-column pair appears once. Opacity order, focal treatment, and text contrast remain valid. |
| Architecture containment | Nested boundaries show ownership and system scope. | Parents, node kinds, targets, peerings, axes, and grid placement agree. |
| Schedule network | Task durations and precedence links determine project finish and float. | A recomputed critical-path model agrees with task fields, critical flags, ranks, summaries, boxes, and visible routes. |
These contracts stay separate because generic geometry cannot tell which mark carries meaning. A heatmap uses opacity over categories, while a box plot uses distribution statistics. A schedule network adds a graph calculation: its displayed early and late dates, total float, critical tasks, and critical links must agree with the precedence graph.
References: scripts/verify-box-plot.py• L1017-1364 scripts/verify-heatmap.py• L367-532 scripts/verify-architecture-containment.py• L645-876 scripts/verify-schedule-network.py• L73-130 scripts/verify-schedule-network.py• L369-612 scripts/test-verify-box-plot.py• L279-1294 scripts/test-verify-unit-grid.py• L209-285
Structured diagram contracts
Structured checkers protect meaning that depends on typed relationships and ordered regions. Event modeling checks stage order, command-event-read-model lanes, visible labels, and typed links. Requirement traceability checks source citations, requirement rows, design links, and explicit gaps. SBGN checks species, processes, compartments, arc direction, route contact, and containment. These rules make the declared relationship agree with the visible route.
Steps and unit grids protect layout as data. Steps keep a continuous and equal rail with one focal stage and matching medallions. Unit grids keep a complete reading-order matrix with matching filled totals and printed summaries. Both reject source or CSS changes that let the browser draw something different from the declared state.
References: scripts/test-verify-event-modeling.py• L20-207 scripts/test-verify-requirement-traceability.py• L25-226 scripts/test-verify-sbgn.py• L71-476 scripts/test-verify-steps.py• L59-245 scripts/test-verify-unit-grid.py• L209-285
Specialized diagram contracts
Some diagrams need a focused contract because their meaning is more than a simple value-to-position mapping. Polar checks protect machine-readable radial metadata and variant parity. Exploded checks protect projected parts, levels, gaps, leaders, traces, labels, and motion lift. Arc checks protect node order and the declared span between endpoints. OAuth checks protect one shipped authentication story.
References: scripts/verify-arc-diagram.py• L281-296 scripts/verify-exploded.py• L218-445 scripts/verify-sequence-oauth.py• L297-545
Cross-cutting verification
Shared checks run beside family checks because a diagram can satisfy its data contract and still hide a label, fail contrast, expose unsafe markup, show old evidence, or fail for users who do not run its script.
Label geometry
The geometry check finds label masks that a later-painted node can cover. It also checks connector routes for clear segments, safe ports, readable corners, and distinct trunks. It resolves simple translations into shared canvas space and fails closed when an arrowed path cannot be parsed.
Use python3 scripts/verify-geometry.py path/to/diagram.html for one asset. Use --all for the shipped asset set. This check does not prove a family’s data encoding.
References: scripts/verify-geometry.py• L316-519 scripts/verify-geometry.py• L542-661 scripts/test-verify-geometry.py• L129-378
Color contrast
The contrast check measures semantic color roles against the required paper in both skins. Missing or unknown roles fail closed so a new token cannot avoid measurement. Family checks add rules when a non-text mark carries chart-specific meaning.
References: scripts/verify-dumbbell.py• L227-243 scripts/verify-service-blueprint.py• L240-276 scripts/test-verify-sbgn.py• L508-529
Accessible SVG and asset safety
The skin checks keep exported diagrams self-contained and usable. They require an informative SVG role, title, description, view box, and accessible-name links. They reject unsafe executable attributes and unapproved remote assets while allowing reviewed font and motion cases. The packaged self-check applies the same safety goal to complete output documents.
References: scripts/verify-semantic-motion.py• L498-551 scripts/self_check.py• L350-580 scripts/test-verify-event-modeling.py• L29-69 scripts/test-verify-requirement-traceability.py• L36-95
Screenshot freshness
Canonical screenshots provide evidence for the gallery. The freshness check ties each image to its source, renderer settings, digest, order, and dimensions. A changed example therefore needs matching screenshot evidence and a manifest update.
References: scripts/verify-screenshot-freshness.py• L21-100
Motion behavior
Motion checks protect progressive enhancement. The first frame remains useful without script. Interactive modes expose the required controls and status. Reduced-motion, print, screenshot, and static paths keep the same explanation available.
References: scripts/verify-semantic-motion.py• L304-551
Import behavior
Import checks make external diagrams useful without treating their content as trusted instructions. They check extraction, documentation and command wiring, supported grammar, selection behavior, output limits, UTF-8 output, and error paths. The Mermaid import page explains the user-facing import flow.
Mermaid selects blocks before parsing them. Draw.io keeps nested geometry and page selection stable. Excalidraw preserves scene structure such as bound text, frames, arrow direction, and discarded-content counts. Adversarial fixtures keep links, scripts, prompt-like labels, and binary payloads outside the output trust boundary.
References: scripts/verify-mermaid-import.py• L632-759 scripts/verify-mermaid-import.py• L1150-1467 scripts/verify-drawio-import.py• L205-763 scripts/verify-excalidraw-import.py• L118-400
Repository wiring
The documentation check keeps the user-facing skill, gallery, plugin surfaces, import routes, references, examples, and packaged support files in agreement. The local doctor checks whether the current installation can run the suite and reports warnings separately from strict failures.
References: scripts/verify-docs-sync.py• L562-614 scripts/verify-docs-sync.py• L1296-1814 scripts/verify-doctor.py• L24-190
Verification lifecycle
The suite supports focused local checks and a broad CI sweep. This split gives fast feedback during editing while keeping the merge gate broad enough to catch drift between examples, references, and evidence.
Local checks
When a diagram changes, start with the checker that owns its visual meaning. Add the shared checker when the change touches labels, contrast, accessibility, motion, screenshots, specialized metadata, or import behavior. Family scripts accept a file path for focused work. Their all-files modes check the shipped set.
The schedule-network verifier accepts a file path or --all for its light, dark, and full shipped examples. It recomputes the critical-path model with exact decimal values, then checks the visible task fields, flags, summaries, and routes against that model. Its regression script is useful when either the model or the rendering contract changes.
The regression scripts are useful when a verifier rule changes. They run standalone and check both shipped examples and focused valid and invalid cases. The usual result codes are 0 for a clean result, 1 for findings, and 2 for bad usage or unreadable input.
References: scripts/verify-architecture-containment.py• L934-962 scripts/verify-schedule-network.py• L616-643 scripts/test-verify-schedule-network.py• L137-402 scripts/test-verify-box-plot.py• L271-289 scripts/test-verify-sbgn.py• L531-547 scripts/verify-mermaid-import.py• L1652-1687
CI sweep
The CI validation job runs family checks, focused verifier tests, import checks, motion checks, screenshot freshness, geometry, skin checks, and render checks across a Python and operating-system matrix. Browser-dependent rendering runs on the pinned Linux leg. Pages deployment is separate. It publishes gallery assets after a push to main, while CI decides whether the source and evidence are fit to merge.
References: .github/…/ci.yml• L86-351 .github/…/pages.yml• L1-41
Change checklist
When these files or sections are changed, remember to consider these:
- A new or renamed visual type must update its family scope, screenshot catalog, documentation route, and CI summary so the example is not skipped.
- A geometry or accessible-label change may require the family check, geometry check, skin check, render check, and screenshot manifest to move together.
- Event-modeling changes must keep stage order, lane order, typed roles, visible labels, link endpoints, and arrow routes aligned.
- Requirement-traceability changes must keep source text, source references, row IDs, typed links, visible gap labels, cell positions, and printed citations aligned.
- SBGN changes must keep glyph namespaces, compartments, arc directions, endpoint contact, route crossings, budgets, all variants, and the budget documentation aligned.
- Steps changes must keep rail continuity, equal widths, stage indices, medallion centers, printed numbers, focal treatment, and the documented width formula aligned.
- Unit-grid changes must keep cell count, reading order, filled prefix, totals, printed labels, token colors, and all variants aligned.
- A box-plot change must keep quartiles, medians, whiskers, Tukey fences, outliers, row labels, focal treatment, rendered visibility, CSS movement rules, and sibling scope aligned across variants.
- Schedule-network changes must keep the independent CPM result, task fields, critical flags, rank positions, route endpoints, summary text, and all shipped variants aligned.
- An import grammar or extractor change must update its command, prompt, reference, fixture, worked example, UTF-8 behavior, limits, and adversarial checks as one contract.
- A motion change must preserve static and reduced-motion fallbacks, semantic-motion references, visual-type records, and screenshot hooks.
- A contrast, font, size preset, taxonomy, manifest, or split-reference change must update every surface checked by verify-docs-sync.py.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A checker cannot parse the marks that define its family. | The file could look verified while the checker read nothing useful. | Family checks fail closed when required marks, values, or variants are missing. |
| A schedule task duration, precedence link, or project finish changes without updating the other schedule fields. | Critical tasks, float, or the displayed finish can report a false plan. | verify-schedule-network.py recomputes exact forward and backward passes and compares the result with fields, flags, links, and summaries. |
| A schedule route misses a task port, crosses a task box, shares a stacked segment, or uses an unsupported path. | Precedence can point to the wrong task or become unreadable. | The schedule verifier restricts path forms and checks endpoints, clear routes, crossings, borders, and port spacing. |
| A box plot places a median, whisker, or outlier against the wrong statistic. | The distribution summary reports a false center, range, or extreme. | verify-box-plot.py checks the shared scale, declared statistics, fences, rows, visibility, and structure. |
| Event stages or trace rows are hidden while their metadata remains present. | The source claims evidence that a reader cannot see. | The event-modeling and requirement-traceability tests mutate hidden, faint, clipped, and covered content. |
| An event or trace link names an unknown endpoint or uses the wrong type. | A visible relationship points to the wrong business fact. | The family verifiers check IDs, roles, relationship types, and endpoints. |
| An SBGN arc crosses another route, misses its glyph, or uses the wrong direction. | A process network reads as a different reaction or junction. | test-verify-sbgn.py covers endpoint contact, crossings, route budgets, and arc direction. |
| A steps rail or unit grid keeps valid metadata but browser CSS moves or repaints it. | The visual count or progress reading becomes false. | Their tests reject transforms, hidden marks, style changes, overlays, and inherited paint. |
| A label mask overlaps a later-painted node or two routes share a trunk. | Text is clipped or the reader cannot trace a connection. | verify-geometry.py checks paint order, ports, routes, and stacked connector runs. |
| A canonical example changes without a new screenshot or manifest digest. | The gallery shows evidence for an older diagram. | verify-screenshot-freshness.py compares source and image digests, renderer settings, order, and dimensions. |
| Imported labels contain links, scripts, or prompt-like instructions. | Untrusted input can affect output or steer the redraw. | Import verifiers keep labels inert and reject unsafe references and oversized input. |
| A verifier regression test covers only a passing example or only a failing mutation. | A checker can reject valid markup or miss the defect it was meant to catch. | Regression scripts pair shipped positive cases with named adversarial cases and expected findings. |
Key files
| File | Why to read it |
|---|---|
| .github/…/ci.yml | Shows the verification matrix, browser leg, and CI summary. |
| scripts/verify-box-plot.py | Defines the distribution and visible-mark contract for box plots. |
| scripts/test-verify-box-plot.py | Shows positive, negative, scope, style, visibility, and fixture-isolation cases for box plots. |
| scripts/verify-schedule-network.py | Defines the schedule model, task fields, precedence graph, visible routes, and shipped-example contract. |
| scripts/test-verify-schedule-network.py | Shows independent CPM oracles, valid skins, graph edge cases, visibility failures, route mutations, and sibling scope. |
| scripts/test-verify-event-modeling.py | Shows the contract for ordered stages, typed roles, visible labels, and event links. |
| scripts/test-verify-requirement-traceability.py | Shows the contract for sources, requirements, design links, gaps, and visible citations. |
| scripts/test-verify-sbgn.py | Shows SBGN semantics, route safety, budgets, variants, and documentation agreement. |
| scripts/test-verify-steps.py | Shows stage-rail, census, typography, CSS, and rendered-layout checks. |
| scripts/test-verify-unit-grid.py | Shows unit order, progress totals, token, geometry, and browser-paint checks. |
| scripts/verify-geometry.py | Defines shared label-mask and connector-routing checks. |
| scripts/verify-mermaid-import.py | Shows Mermaid grammar coverage, selection, trust boundaries, limits, and Unicode handling. |
| scripts/verify-drawio-import.py | Shows draw.io parsing, page selection, nested geometry, hidden content, and output safety. |
| scripts/verify-excalidraw-import.py | Shows Excalidraw bindings, graph meaning, discarded-content records, and safe output. |
| scripts/self_check.py | Defines packaged output checks for completeness, safe assets, bounds, motion, and offline use. |
| scripts/verify-screenshot-freshness.py | Defines the relationship between canonical sources, screenshots, renderer settings, and the manifest. |
| scripts/verify-semantic-motion.py | Defines semantic motion patterns and fallback behavior. |
| scripts/verify-docs-sync.py | Defines the cross-file contract for routes, gallery reachability, packaged support, manifests, presets, and fonts. |
| scripts/screenshot_catalog.py | Provides the mapping between canonical types, source examples, and screenshots. |