Introducing a new way to review your PRs.

Read the announcement

usestrix / strix

Open-source AI penetration testing tool to find and fix your app’s vulnerabilities.

View on GitHub

26 documents · Updated Nov 20, 2026

docs / operational-telemetry

Operational telemetry

Strix records small, anonymous signals about scan use, findings, model use, viewer activity, and failures. When telemetry is enabled, it sends these signals as best-effort beacons to PostHog for product analytics and Scarf for package-usage data. Scan content, targets, prompts, responses, and vulnerability details stay out of this path.

Telemetry scope

The telemetry system captures the operational facts needed to understand a run without collecting its private work. It uses one short-lived random session identifier and adds broad system and scan context to supported events.

The main event groups are:

Event groupPurposeTypical data
Scan startShow how Strix is usedModel and auth mode, scan mode and type, interaction mode, first-run state
FindingMeasure scan output at an aggregate levelSeverity, CWE category, and CVE flag
Scan endDescribe the run result and resource useExit reason, duration, finding totals by severity, model request and token totals, cost, providers, and loaded skills
ErrorGroup operational failuresError category, scan phase, exception class, and optional cause class
Viewer activityMeasure use of the local report viewerOpen, feedback, call-to-action, email-flow, and agent-steering actions

The data policy excludes usernames, paths, URLs, domains, code, finding text, LLM prompts, and LLM responses. A model name can be included as usage context, but the model interaction itself is not collected. The source policy is documented in strix/…/README.md.

Telemetry lifecycle

Telemetry follows a scan from setup through completion or failure. The scan records a coarse phase before major work, and the final event uses the stop reason and usage accumulated by the current process.

This diagram shows how scan activity becomes optional outbound telemetry while local scan work remains the primary path.

flowchart LR
    A[Scan setup] --> B[scan_started]
    B --> C[Scan phases]
    C --> D[finding events]
    C --> E[error events]
    C --> F[Scan stop]
    F --> G[scan_ended]
    B --> H{Telemetry enabled?}
    D --> H
    E --> H
    G --> H
    H -- No --> I[No outbound beacon]
    H -- Yes --> J[PostHog]
    H -- Yes --> K[Scarf]

The start event is emitted after scan preparation. Finding events are emitted when Strix adds findings to the report. The end event is attempted during normal cleanup and failure cleanup, with separate guards that prevent duplicate end events for each destination. A failed or slow telemetry request does not become a scan failure because delivery is best effort and uses short time limits.

Privacy and opt-out

Telemetry is controlled by the telemetry setting resolved through Strix configuration. Users can disable all outbound telemetry before a run with STRIX_TELEMETRY=0; disabled destinations skip event delivery.

The shared event context contains operating system, CPU architecture, Python version, Strix version, and a process-local random session ID. Error events add only a named category and coarse scan phase. If an exception is present, Strix sends its class name and at most the class name of its cause. It does not send messages, arguments, or tracebacks.

Analytics destinations

PostHog and Scarf receive the same operational event intent through separate adapters. PostHog sends structured event properties for product analysis. Scarf sends package-oriented request data with the event and Strix version in the request path.

The adapters keep the shared event meaning while matching each service's data shape. PostHog keeps provider totals as structured properties. Scarf represents those totals as JSON because its query data is flat, and it represents the loaded skills as a comma-separated value.

Neither destination controls scan progress. Each destination checks the same telemetry setting, sends with bounded connect and read timeouts, and records delivery problems only in local debug logs. This separation lets one service fail without blocking the other or changing the scan result.

The detailed event payload rules and delivery contracts are described in Telemetry event delivery.

Usage and cost signals

Strix keeps model usage in the run state so the final telemetry event can report work done during the current process. This includes requests, input tokens, output tokens, total tokens, provider totals, and a cost value when one is available.

Resumed scans report only the usage added by the current process. Subscription-backed runs still report token activity but keep cost at zero because they do not have a metered per-token charge. These signals describe resource use and do not include model content.

Operational logs

Operational telemetry and scan logs serve different purposes. Telemetry sends small aggregate signals to external services. Scan logging keeps detailed local diagnostics from Strix and the agent runtime in the per-scan run directory and on the console, with scan and agent context on each record so an engineer can trace a local run without putting that detail into telemetry. The local file is append-only for a scan rerun. Console output stays focused on important messages unless debug mode is enabled, and dependency noise is reduced so it does not hide scan activity.

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep event names and shared property meanings aligned between the PostHog and Scarf adapters.
  • Preserve the destination-specific end-event guards and the recorded exit reason when changing scan cleanup or resume behavior.
  • Keep the process-usage baseline aligned with resumed-run handling so an end event does not report the full historical run twice.
  • Keep the privacy boundary intact when adding error fields or scan context. Do not pass messages, traces, targets, or report content to telemetry.
  • Keep the telemetry setting and its environment alias aligned with the opt-out command shown to users.
  • Preserve the destination data shape when adding structured provider or skill data. PostHog properties and Scarf query values do not have the same encoding rules.
  • Keep scan and agent context available on both local file and console records when changing logging setup. Do not let third-party debug output hide the primary scan messages.

Important failure modes

TriggerConsequenceGuard
Telemetry is disabledNo PostHog or Scarf beacon is sentEach destination checks the shared telemetry setting before sending
A destination is blocked or unavailableThat destination loses the event, but the scan continuesDelivery is best effort and uses short bounded timeouts
Cleanup runs more than onceA duplicate end event could distort run totalsEach destination records whether its end event was sent
A scan fails before its final eventThe final event can still describe the failure when report state existsCleanup sends the recorded exit reason and failure handlers report an error category
An exception contains private diagnostic textPrivate text could leak if serializedError payloads use exception class names and cause class names only
A resumed run reuses historical usageResource totals could be overstatedThe final event uses usage accumulated since the process baseline
Provider totals or skill names use the wrong destination encodingScarf data can become hard to parse or PostHog properties can lose structureKeep provider data structured for PostHog and JSON-encode it for Scarf; join Scarf skill names as query-safe text
Local logging is configured without its context or filteringEngineers lose scan and agent trace context, or dependency output hides useful recordsShared log filters add context, and dependency and console filters limit noise

Key files

FileWhy to read it
strix/…/README.mdDefines the user-facing data policy, collected fields, excluded data, and opt-out behavior.
strix/…/__init__.pyShows the public fan-out for shared error reporting and scan-phase updates.
strix/…/_common.pyDefines shared session context, phase state, privacy-safe exception fields, and delivery limits.
strix/…/posthog.pyDescribes structured analytics events and their PostHog destination.
strix/…/scarf.pyDescribes package-usage events, flat query encoding, and the Scarf destination.
strix/…/scan_setup.pyShows when scan-start telemetry begins after scan preparation.
strix/…/main.pyShows failure reporting and final best-effort delivery during scan cleanup.
strix/…/state.pyConnects findings and process-level model usage to telemetry end events.
strix/…/usage.pyDefines how model token use and cost are aggregated, including subscription runs.
strix/…/logging.pyDefines the separate local scan-log path for Strix and agent-runtime events, console filtering, dependency noise controls, and scan and agent context.