Introducing a new way to review your PRs.

Read the announcement

usestrix / strix

Open-source AI penetration testing tool to find and fix your app’s vulnerabilities.

View on GitHub

26 documents · Updated Nov 20, 2026

docs / telemetry-event-delivery

Telemetry event delivery

Telemetry event delivery gives Strix one small, privacy-aware event contract and two independent ways to send it. The operational telemetry system uses this contract to report scan use, aggregate results, and failure categories without sending scan content or delaying the user for long network waits.

Shared telemetry contract

Event identity and metadata

The shared event contract describes a process with a short-lived session ID and coarse system context. Each event can also carry scan context such as the model route, scan mode, scan type, run phase, and whether the process is a first run. The same event meaning is sent to both delivery services, while each service uses the shape required by its own endpoint.

The main scan lifecycle has three event types:

  • scan_started records how the scan was launched.
  • finding_reported records aggregate finding labels such as severity and CWE category.
  • scan_ended records the exit reason, duration, aggregate vulnerability counts, model usage totals, per-provider model usage, and loaded skills.

The session ID is generated in memory for the process. It is not saved as a user identity. The first-run marker is local and only supports a coarse adoption signal.

References: strix/…/_common.py• L14-20 strix/…/_common.py• L33-55 strix/…/posthog.py• L59-135 strix/…/scarf.py• L60-141

Error event privacy

Error events explain where a failure occurred without exposing the failure text. An error contains an application-defined error category and the current coarse scan phase. When an exception is available, telemetry adds only its class name and, when present, the class name of its cause or context.

Exception messages, arguments, tracebacks, prompts, responses, targets, paths, and finding details stay on the machine. An exception group contributes its first exception class so the event remains small and predictable. The public error helper sends the same error to both backends, and a failed delivery does not become a scan failure.

References: strix/…/_common.py• L59-96 strix/…/__init__.py• L5-11 tests/test_telemetry_errors.py• L23-109

Log context

Per-scan logging adds task-local scan and agent context to each Strix log record so operators can connect messages to a scan and, when present, to an agent. The context follows the current async task and its later child tasks, while an unset value appears as -.

The logging boundary writes detailed records to the scan's strix.log file and sends selected records to stderr for both Strix and agent activity. Debug mode expands stderr output, while dependency filters keep noisy library output from hiding scan messages. Logging setup also returns an idempotent cleanup action so scan shutdown can remove only the handlers it added.

References: strix/…/logging.py• L20-46 strix/…/logging.py• L50-116 strix/…/logging.py• L132-212

Delivery paths

PostHog delivery

PostHog is the analytics path for event capture and product-use analysis. It sends events as JSON with the in-memory session ID as the distinct ID and includes the shared metadata plus the event-specific properties. Its scan-end event also carries per-provider model usage in the structured event properties. PostHog receives viewer interaction events, which are outside the paired scan event stream.

PostHog checks the resolved telemetry setting before sending. A send uses short connect and read limits, logs delivery problems at debug level, and returns control to Strix whether the service is reachable or not. Scan-end delivery is guarded by state so one report cannot create duplicate end events.

References: strix/…/posthog.py• L25-55 strix/…/posthog.py• L59-193 strix/…/_common.py• L16-20

Scarf delivery

Scarf is the package-usage path for the same core scan lifecycle and error events. It encodes the event name and Strix version in the request path, then sends the remaining event properties as query data. It includes the session ID explicitly so Scarf can relate events from one process. Per-provider model usage is encoded as JSON because the query data is flat.

Scarf follows the same opt-out check, short timeout, and best-effort rule as PostHog. It does not mirror PostHog-only viewer events. Its scan-end guard is separate from PostHog's guard, so a problem in one service does not prevent the other service from receiving its event.

References: strix/…/scarf.py• L28-57 strix/…/scarf.py• L60-153 strix/…/_common.py• L14-20

Initialization and shutdown boundaries

Scan initialization

Scan telemetry starts only after Strix has completed the work needed to prepare a scan. That boundary gives scan_started a resolved model, authentication mode, scan mode, scan type, interaction mode, and instruction-presence flag. Failures during model preflight or preparation can report an error, but they do not create a normal started event before the scan is ready.

The command-line and TUI launch paths both call the same telemetry start step. Viewer-only, authentication, completion, and cloud commands leave through separate command paths and do not enter this scan initialization boundary.

flowchart TD
    A[Launch scan] --> B[Preflight and prepare run]
    B -->|failure| C[Report error]
    B -->|ready| D[Start PostHog and Scarf events]
    D --> E[Run scan]

References: strix/…/main.py• L397-419 strix/…/main.py• L434-480 strix/…/runtime.py• L160-224 strix/…/scan_setup.py• L225-236

Scan completion

Scan-end telemetry is best effort and can run from two completion boundaries. A completed report can emit its end event when the report state saves final results. The top-level command cleanup also attempts delivery for normal stops, interrupts, and errors, which covers exits that do not reach normal report completion.

Each backend keeps its own sent flag in the report state. The first recorded exit reason remains authoritative, so later cleanup does not replace a more precise reason such as a budget limit. Network failure does not block cleanup because each send has a short timeout and suppresses delivery exceptions at the shutdown boundary.

sequenceDiagram
    participant S as Scan process
    participant R as Report state
    participant P as PostHog
    participant C as Scarf
    S->>R: Record result or exit reason
    R->>P: Send scan_ended when applicable
    R->>C: Send scan_ended when applicable
    S->>R: Best-effort final cleanup
    R->>P: Send only if not already sent
    R->>C: Send only if not already sent

References: strix/…/state.py• L227-229 strix/…/state.py• L560-582 strix/…/main.py• L500-519 strix/…/posthog.py• L94-135 strix/…/scarf.py• L97-141

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep the event fields and event names aligned between posthog.py and scarf.py when a shared scan or error event changes.
  • Keep the per-provider model usage data aligned between the structured PostHog properties and the JSON value sent through Scarf query data.
  • Keep the error-class-only rule intact when adding failure context. Do not pass exception text or trace data into event properties.
  • Keep scan and agent log context attached when changing scan setup or task boundaries. Log records rely on task-local values and use - when no value is set.
  • Keep both initialization paths after scan preparation. Moving telemetry earlier can report scan metadata that was not resolved or report a scan that never started.
  • Keep the separate PostHog and Scarf sent flags when changing scan-end delivery. One backend may succeed while the other fails.
  • Keep the recorded exit reason ahead of later cleanup defaults so an early, specific stop reason is not overwritten.

Important failure modes

TriggerConsequenceGuard
Telemetry is disabled in settingsBoth delivery paths skip the eventEach backend checks the resolved telemetry setting before network work
A telemetry endpoint is blocked or slowThe event is lost, but scan exit remains independentShort connect and read limits plus exception handling make sends best effort
Error reporting receives an exception with private textThe text must not enter telemetryThe shared exception mapper sends class names and selected cause classes only
Final reporting and top-level cleanup both runA scan-end event could be duplicatedReport state stores one sent flag per backend
A later cleanup path supplies a less precise exit reasonThe original failure or stop reason could be hiddenReport state keeps the first recorded exit reason

Key files

FileWhy to read it
strix/…/_common.pyDefines the session identity, shared metadata, timeout rule, scan phase, and private exception mapping.
strix/…/__init__.pyShows the public error fan-out to both delivery paths.
strix/…/logging.pyDefines scan and agent log context, console and per-scan log boundaries, dependency log filtering, and cleanup.
strix/…/posthog.pyDescribes the analytics payloads, viewer events, per-provider usage data, and PostHog send boundary.
strix/…/scarf.pyDescribes the package-usage request shape, per-provider usage encoding, and Scarf send boundary.
strix/…/scan_setup.pyShows the shared scan-start boundary used by command-line and TUI launches.
strix/…/main.pyShows preflight ordering and final best-effort cleanup.
strix/…/state.pyShows per-backend duplicate protection and the recorded exit reason.
tests/test_telemetry_errors.pyStates the privacy and fan-out guarantees for error events.
tests/test_telemetry_resume.pyStates the scan-end usage and duration behavior for resumed runs.