Introducing a new way to review your PRs.

Read the announcement

usestrix / strix

Open-source AI penetration testing tool to find and fix your app’s vulnerabilities.

View on GitHub

26 documents · Updated Nov 20, 2026

docs / context-budget-and-compaction

Context budgets and compaction

Context budgeting keeps each model request within the model's usable input and output limits. Compaction replaces older conversation turns with a factual checkpoint while keeping recent work available, so a long security scan can continue without losing findings, credentials, or the next action. This page is a child of Model context management.

Model context capacity

The model context capacity is the number of input tokens that a selected model can accept. The system first uses model metadata from LiteLLM, then uses a configured fallback when the model is not mapped. Token counts also use the model tokenizer when available and fall back to a conservative UTF-8 byte count when they do not. This keeps budget checks safe across providers and model names. strix/…/context_budget.py• L16-89

The budget treats the request as more than stored conversation. Agent instructions and tool descriptions also consume prompt space, while the model needs room for its response. The main context settings are:

SettingRole
Automatic compactionAllows proactive history reduction before a request.
Compaction bufferReserves room around the usable request budget.
Recent-history allowanceSets the target size for turns kept verbatim.
Fallback context capacitySupplies input capacity for an unmapped model.
Summary output limitCaps the checkpoint response and respects the model output limit.

The default values are conservative controls rather than a promise that every provider has the same limits. They can be changed through the context settings. strix/…/settings.py• L85-102

References: strix/…/context_budget.py• L30-89 strix/…/settings.py• L85-102

Compaction decision

Compaction is a history rewrite that runs before a model turn when the estimated request is over its usable budget. The estimate includes the current instructions, tool text, and serialized session items. The system keeps at least the configured recent-history allowance and reserves the larger of the configured buffer or the model's output limit. A short session is not compacted because there is not enough history to reduce. strix/…/compaction.py• L339-368

The execution loop also uses compaction as recovery. If a provider reports a context-window overflow, the loop recognizes typed overflow errors and known provider message forms. It excludes rate-limit and similar errors, then forces one compaction attempt before retrying the turn. Proactive and recovery compaction share the same history rewrite path. strix/…/execution.py• L643-687 strix/…/execution.py• L755-773

When a scan turn reaches the context limit, the system follows this sequence:

flowchart TD
    A[Prepare session] --> B[Estimate request size]
    B -->|Within budget| C[Run model turn]
    B -->|Over budget| D[Select older history]
    D --> E[Summarize older history]
    E -->|Summary succeeds| F[Write checkpoint and recent turns]
    F --> C
    E -->|No room or summary failure| G[Keep session unchanged]
    C -->|Provider reports overflow| D

References: strix/…/compaction.py• L339-368 strix/…/execution.py• L643-687 strix/…/execution.py• L755-773

Checkpoint history

The checkpoint is a user-scoped history item that marks earlier conversation as established context rather than a new instruction. The summary asks the model to preserve the security objective, every distinct finding, exact secrets and proof values, system details, completed and active work, dead ends, next moves, and relevant files. This shape protects scan state that would otherwise disappear with old turns. strix/…/compaction.py• L96-143 strix/…/compaction.py• L278-285

The summary request has its own budget. The system subtracts the summary instructions, any earlier checkpoint, the allowed summary output, and a small wrapper margin from the model context capacity. If the older conversation is still too large, it keeps the start and end of that text and marks the omitted middle. A summary is not attempted when no input room remains. strix/…/compaction.py• L233-261 strix/…/compaction.py• L380-387

Later compactions update the prior checkpoint instead of creating an unrelated chain of summaries. The new checkpoint then combines with the recent turns that were kept verbatim. This gives the active agent one current record of older state plus the latest conversation. strix/…/compaction.py• L224-230 strix/…/compaction.py• L368-390

References: strix/…/compaction.py• L96-143 strix/…/compaction.py• L224-230 strix/…/compaction.py• L233-285 strix/…/compaction.py• L368-390

Tool-call continuity

Recent history is trimmed only at a safe boundary. The boundary must not leave a tool result without its matching tool call, including when several calls run in parallel. This preserves valid provider input while retaining as much recent work as the token allowance permits. strix/…/compaction.py• L190-221

References: strix/…/compaction.py• L190-221

Session rewrite safety

Compaction changes persisted session history through a serialized read-modify-write operation. The write lock prevents competing out-of-band changes from interleaving with the rewrite. Before replacing the session, the system checks that the item count is still the same as when compaction began. If another writer added turns, the rewrite is skipped; if storage fails during replacement, the original items are restored. strix/…/sessions.py• L89-156

This protection matters because summary generation is a model call and can take time. A slow summary must not overwrite newer scan progress. A failed summary therefore leaves the original session available for the next attempt. strix/…/compaction.py• L302-336 strix/…/compaction.py• L386-400

References: strix/…/sessions.py• L89-156 strix/…/compaction.py• L302-336 strix/…/compaction.py• L386-400

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep model metadata lookup, token counting, and fallback capacity aligned. A mismatch can make a request appear safe when the provider rejects it.
  • Keep the usable request budget consistent with the reserved output allowance and the summary request budget. Changing one reserve without the other can make compaction itself exceed the model window.
  • Preserve the checkpoint section contract when changing the summary prompt. Downstream agent behavior depends on exact findings, secrets, work state, dead ends, and next steps remaining visible.
  • Keep the tool-call boundary rule coupled to the session rewrite. A smaller recent tail must never create an unmatched tool result.
  • Keep overflow classification separate from rate-limit classification. A false overflow can rewrite history during a provider throttle.
  • Keep the expected-length check and restore-on-failure behavior with any new session rewrite path. A slow compaction must not remove newer turns.

Important failure modes

TriggerConsequenceGuard
LiteLLM has no model metadata or tokenizerThe exact model capacity or token count is unavailable.Use the configured context fallback and UTF-8 byte length as a conservative estimate.
Instructions, tools, and history exceed the usable budgetThe next model request can exceed the provider context window.Count all three inputs and reserve response space before the turn.
Provider returns an untyped context errorThe run can fail without reducing its history.Match known overflow message markers while excluding rate-limit and service errors.
Summary input has no room inside the model windowCompaction cannot produce a valid checkpoint.Skip the rewrite and leave the session unchanged.
Summary model call fails or returns no textOlder history cannot be represented safely.Return without rewriting the session.
Compaction cuts between a tool call and its resultThe provider can reject the rebuilt history.Move the split to a boundary with no open tool calls.
A new turn arrives during summary generationA completed summary can erase newer scan state.Compare the expected item count under the session write lock.

Key files

FileWhy to read it
strix/…/context_budget.pyDefines how model capacity and token use are estimated.
strix/…/compaction.pyDefines the compaction trigger, checkpoint content, safe split, and summary budget.
strix/…/execution.pyShows where proactive compaction and overflow recovery sit in the model-turn loop.
strix/…/sessions.pyDefines safe persisted session replacement and concurrency protection.
strix/…/settings.pyDefines the context controls that shape capacity, reserves, and summary size.
docs/model-context-management.mdParent page for the broader model context lifecycle, including related protections.