Desktop application environments
Desktop application environments turn a Linux desktop application into a recorded, inspectable, and repeatable computer environment. The system captures a live session, explains the state that changed, helps define a task, provisions a build computer, and exports a local bundle that can start the application with known data.
This page is part of artifact and application analysis. Binary diagnostics and comparison, application reconstruction, JavaScript and desktop reconstruction, and firmware and embedded analysis cover related analysis work.
Environment lifecycle
The lifecycle keeps observed facts separate from derived products. A live session produces checkpoints and retained content. State inspection explains those checkpoints. Task planning turns a useful change into checks. Export then combines the capture, state view, and tasks into a computer app bundle.
The main flow is:
flowchart LR
A[Installed Linux application] --> B[Capture session]
B --> C[Checkpoints and retained content]
C --> D[State inspection]
D --> E[Task draft and checks]
C --> F[Export plan]
D --> F
E --> F
F --> G[Local computer app bundle]
G --> H[Repeatable desktop environment]
H --> I[Task witness]
H --> J[Fresh task measurements]
The capture and state view describe what was observed or inferred. The bundle is a derived product. It keeps the capture link and its limits so an engineer can judge how closely it represents the source session.
Application capture
Capture session
The capture session starts one Linux desktop application in a controlled run and records its launch facts, processes, windows, opened files, and file changes. It checks the host, display, window manager, and output folder before launch. It refuses to start beside a matching application window because that window could add old or unsaved state to the baseline.
References: src/…/DesktopApplicationCapture.ts• L202-249 src/…/DesktopApplicationCapture.test.ts• L180-249 src/…/DesktopApplicationSession.ts• L111-131
The session records a baseline and then samples the application while it runs. A caller can request named checkpoints. Stop, application exit, a time limit, and cancellation end the run with the checkpoints already taken. The session owns the launched process group and records when cleanup cannot be verified. A failed publication removes the partial capture rather than presenting it as complete.
References: src/…/DesktopApplicationCapture.ts• L254-353 src/…/DesktopApplicationCapture.ts• L356-431 src/…/DesktopApplicationSession.ts• L214-243 src/…/DesktopApplicationSession.ts• L375-451
Capture scope
The capture scope covers application state directories, selected paths, and paths the application opens. Each checkpoint records file facts and retains file bytes within entry, file, and total content limits. Exclusions and broad home-directory selections stop the capture from becoming an unbounded copy of the host.
References: src/…/DesktopApplicationCapture.ts• L316-347 src/…/HomeStateWalker.ts• L103-167 src/…/DesktopContentStore.ts• L42-63
The retained content store uses file digests so repeated bytes are kept once. A path can be marked as changed while its bytes are omitted because it is too large, unreadable, changed during the read, or outside the content budget. Capture limitations keep these facts visible to later inspection and export.
References: src/…/DesktopContentStore.ts• L119-173 src/…/DesktopContentStore.ts• L176-248 src/…/DesktopRetainedContent.ts• L39-99
Capture content is evidence, not a disk image. A change that occurs and is undone between checkpoints is not visible. A walk can also be incomplete when it reaches an entry limit, an unreadable directory, or another filesystem. Later task drafting must treat these limits as unknown state rather than as proof that nothing changed.
References: src/…/DesktopApplicationCapture.ts• L115-169 src/…/HomeStateWalker.ts• L88-107 src/…/DesktopTaskDraftNotes.test.ts• L151-239
State inspection
State views
State inspection reads a valid capture and creates a time-based view of the paths it observed. It identifies likely application state and documents from paths and application signals. Safe readers expose text, JSON, key files, XML, zip archives, SQLite data, dconf data, and compressed documents. Binary content remains a content record without an invented structured view.
References: src/…/DesktopApplicationState.ts• L111-156 src/…/DesktopApplicationState.ts• L390-413 src/…/stateRoles.ts• L1-20
The view compares each path across checkpoints. It reports creation, deletion, content change, format change, and unreadable state. Structured views can also report added, removed, and changed values. Dconf is read from a staged database rather than the live user service. SQLite views use the recorded sidecar files when they are needed to describe committed state. Reader failures keep their command and exit details in the state evidence.
References: src/…/DesktopApplicationState.ts• L265-384 src/…/DesktopApplicationState.ts• L425-532 src/…/DesktopDconfDiagnostics.test.ts• L66-97 src/…/DconfDatabaseReader.ts• L21-61
View limits
View policy bounds the amount of text, structured data, archive data, and database data that inspection reads. A bounded view is useful for analysis but cannot prove that an omitted value is absent. Task evaluation therefore treats a needed unreadable or cut view as undecided.
References: src/…/DesktopApplicationState.ts• L535-544 src/…/DesktopApplicationState.ts• L589-627 src/…/DesktopZipView.ts• L86-137
Inspection also preserves uncertainty from the capture. It does not turn an incomplete directory walk, missing retained bytes, or an unreadable reader result into a claim that the state is absent.
References: src/…/DesktopApplicationState.ts• L455-484 src/…/DesktopApplicationState.ts• L535-557 src/…/DesktopTaskDraftNotes.test.ts• L359-400
Task planning
Task drafts
Task planning compares a seed checkpoint with an end checkpoint and proposes a portable task. A task is a user prompt plus declarative checks over home-relative application files. The draft suggests checks for file presence, lines, named values, XML text, archive entries, counts, and database rows. It also records notes when the observed change is incomplete, unowned, unreadable, or not covered by a check.
References: src/…/DesktopTaskDraft.ts• L58-181 src/…/DesktopTaskDraftNotes.test.ts• L104-149 src/…/DesktopTaskDraftNotes.test.ts• L242-356 src/…/appTaskDraft.ts• L1-20
The draft is an aid for an engineer, not a complete user request. The author must replace its prompt and keep checks that express the intended result rather than volatile values. A task can add a reader that runs the application's own headless command on a private copy when the built-in state view cannot read the document.
References: src/…/DesktopTaskDraft.ts• L36-41 src/…/DesktopTaskReaders.ts• L54-117 src/…/DesktopTaskExport.test.ts• L481-601
Task checks
Task checks describe a result without depending on model judgment for file-backed state. They can test presence, lines, values, text, rows, member counts, or a screen question. A screen check is reserved for a result that no owned file can show. Readers provide a controlled view for binary or application-specific documents.
References: src/…/DesktopTaskReaders.ts• L21-31 src/…/DesktopBuildWitness.ts• L276-324 src/…/DesktopBuildWitnessBundle.ts• L114-167
Task admission
Export can include several tasks, each with its own end checkpoint. It accepts a task only when its checks apply to owned paths, fail on the shipped seed, and pass on the selected captured end state. If a check depends on a cut or unreadable view, admission refuses to claim that the task is proved. This prevents a task from being complete before the agent starts or from describing a result the capture did not prove.
References: src/…/DesktopApplicationExport.ts• L389-413 src/…/DesktopTaskExport.ts• L549-635 src/…/DesktopTaskExport.test.ts• L214-341
An accepted task becomes an agent procedure, a verifier, and an evaluation entry in the bundle. File-backed checks use exported state views and admitted readers. Screen checks use the bundle's screen judge and require checkpoint screenshots as examples. A task can use a task-specific end checkpoint, which lets one capture support several changes.
References: src/…/DesktopTaskExport.ts• L655-719 src/…/DesktopExportReaders.ts• L62-152 src/…/DesktopTaskExport.test.ts• L344-409
Build-computer provisioning
Build sources
The build computer is a temporary agentenv computer for installing and operating a desktop application before export. A request can install from an Ubuntu archive, a package file, an AppImage, or a tarball. An existing bundle can also be started for review. A request-built computer has the application and REA runtime but no captured seed or data hooks while it is used for building.
References: src/…/appBuild.ts• L75-157 src/…/appBuild.ts• L243-339 src/…/appBuild.ts• L425-477 src/…/DesktopApplicationExport.test.ts• L201-223
Build computer lifecycle
The build computer owner starts a computer, forwards desktop actions, keeps an action log, and serializes actions and build steps per computer. It also supports task measurement on fresh computers. This gives an engineer a clean place to discover launch behavior and task actions before those facts become an exported environment.
References: src/…/DesktopBuildComputers.ts• L408-487 src/…/DesktopBuildComputers.ts• L601-705
The owner keeps a computer addressable through startup or close failures until teardown is confirmed. Server shutdown cancels work and closes every held computer. The screen judge, long-running commands, remote readers, and computer are released as one lifecycle so a failed helper does not leave a hidden build resource.
References: src/…/DesktopBuildComputers.ts• L237-334 src/…/DesktopBuildComputers.ts• L721-798 src/…/DesktopBuildComputers.cleanup.test.ts• L18-61
Build capture
Build capture runs inside a request-based build computer. It first checks that the installed REA runtime supports the export options needed for synthetic seeds and task-specific ends. It then records seed and end checkpoints, keeps optional screenshots for screen tasks, summarizes the application and task draft, and exports the bundle back to the host. A computer started from an existing bundle is for review and cannot create another build capture.
References: src/…/DesktopBuildCapture.ts• L142-224 src/…/DesktopBuildCapture.ts• L319-375 src/…/DesktopBuildCapture.ts• L620-707
Build capture can mark a sign-in window and named task checkpoints. Export uses these boundaries to separate state that can be supplied at deploy time from state that can be placed in the seed. The two sign-in boundaries are required as a pair because one boundary cannot show which files the sign-in wrote.
References: src/…/DesktopBuildCapture.ts• L479-600 src/…/DesktopBuildCapture.ts• L620-670
Build secrets
Build sessions can hold sign-in or other sensitive settings outside the exported seed. Secret values are supplied by handle and are redacted from action results, logs, and witness records. Withheld files are kept as secrets outside the bundle unless the person gives explicit consent to seed them.
References: src/…/DesktopBuildSecrets.ts• L39-46 src/…/DesktopBuildSecrets.ts• L265-319 src/…/DesktopExportWithheld.ts• L114-182
Task measurement
Task measurement runs an exported task in fresh computers and records whether its solve rate falls within the expected difficulty band. It preserves the task digest with the solve results, cost, turns, time, model, and runner versions. The result is written beside the bundle's task documentation so later measurements do not replace records that cannot be read.
References: src/…/DesktopBuildSolves.ts• L116-170 src/…/DesktopBuildSolves.ts• L257-286 src/…/DesktopBuildSolves.ts• L288-376
Measurement stops when the result is decided or when the budget cannot cover another solve. Completed solves remain attached to a failure caused by cancellation, output limits, or cleanup. This makes a partial measurement useful without presenting it as a complete difficulty result.
References: src/…/DesktopBuildSolves.ts• L301-429 src/…/DesktopBuildSolves.test.ts• L256-294 src/…/DesktopBuildCommands.ts• L144-175
Task witness
Witness stages
The task witness checks that an exported task describes a real and repeatable change. It grades the live application in four ordered stages: the seed must fail some check, the edit must pass all checks, a near miss must fail again, and reset must restore the seed result.
References: src/…/DesktopBuildWitness.ts• L174-177 src/…/buildWitness.ts• L1-24
The witness accepts a stage only when the matching desktop actions occurred. An edit or near miss needs a new applied input. The reset stage needs a reset as its last change. An unknown reset result blocks a passing witness instead of treating the reset as absent. A witness also checks that its task steps and verifier still match the bundle that started the computer.
References: src/…/DesktopBuildWitness.ts• L327-389 src/…/DesktopBuildWitness.ts• L497-657 src/…/DesktopBuildWitnessBundle.ts• L68-111 src/…/DesktopBuildComputers.ts• L662-686
Witness record
When the final stage ends, the witness writes an Evidence record with the task digest, stage results, action log, and an optional desktop recording. Recording segments are checked against their manifest and can be fetched again after a partial transfer. The record is written as a complete output tree so a failed unpublished write leaves no partial result.
References: src/…/DesktopBuildWitness.ts• L391-437 src/…/DesktopBuildWitnessRecord.ts• L99-136 src/…/DesktopBuildWitnessRecord.ts• L138-196
The witness is controlled replay, not a second run of the generated verifier. It uses the same application state views and check evaluator as the exported task. Screen tasks use the bundle's own judge on a fresh screenshot. Its record reports when the action log dropped old actions, the app state did not settle, or the desktop recording is missing.
References: src/…/DesktopBuildWitnessRecord.ts• L198-245 src/…/DesktopBuildWitnessBundle.ts• L114-167
Environment export
Export plan
Export combines the capture and state view into an agentenv computer app for Ubuntu 24.04 X11. It selects seed and end checkpoints, identifies the application's owned home paths, records package and library needs, preserves the launch command, and copies retained seed files into a new local bundle.
References: src/…/DesktopApplicationExport.ts• L203-303 src/…/DesktopApplicationExport.ts• L355-430 src/…/appExport.ts• L52-123
The exported app launches at boot and becomes ready when its application window appears. Its data hooks expose only owned paths and typed views of those files. data/add adds task inputs under owned paths and data/reset restores the captured file seed. Application settings stored in dconf are loaded separately and are not part of the file reset contract.
References: src/…/appExport.ts• L527-659 src/…/appExport.ts• L750-781 src/…/DesktopBuildComputers.ts• L174-226
Seed identity
The seed can represent a user's real state at a selected checkpoint or a synthetic state made for a build task. Text state and dconf values can replace the capture home, user, and host with the new computer's identity. Binary state and unsupported text encodings remain as captured. Export reports seeded, unseeded, scrubbed, and unscrubbed files so the bundle's privacy limits are visible.
References: src/…/DesktopApplicationExport.ts• L142-184 src/…/DesktopApplicationExport.ts• L225-287 src/…/DesktopExportFiles.ts• L107-125
Keep bundles local until their state has been reviewed. A bundle can contain personal data and can still contain identity in binary files, file names, or content that export cannot rewrite. Withheld sign-in or licence files remain outside the seed unless consent allows them in.
References: src/…/appExport.ts• L609-626 src/…/DesktopExportFiles.ts• L192-263 src/…/DesktopExportWithheld.ts• L163-182
Launch state
Export preserves the captured working directory and rewrites home-relative launch arguments for the new computer. It uses selected paths and captured documents to retarget arguments that point to the captured host. Unrelated outside-home arguments remain unchanged. When the seed contains one document and the desktop entry accepts a file, the exported app opens that document at launch. Otherwise the bundle reports why it cannot promise a document launch.
References: src/…/DesktopExportLaunch.ts• L82-207 src/…/DesktopExportLaunch.test.ts• L26-126 src/…/seedDocument.ts• L55-101
Environment interface
The exported computer presents one stable boundary for an agent. The application is ready at boot, desktop actions operate on its visible window, data/get reads owned files and typed views, data/add adds task inputs under owned paths, and data/reset restores the file seed. This boundary lets task evaluation and witness replay use the same state contract. Task readers run on private copies of owned files when a task needs an application-specific view.
References: src/…/appExport.ts• L540-559 src/…/DesktopExportReaders.ts• L62-113 src/…/DesktopBuildWitness.ts• L276-324
Change checklist
When these files or sections are changed, remember to consider these:
- Keep capture checkpoint names, state versions, task end selection, build summaries, and export seed selection aligned when lifecycle rules change.
- Preserve the link between retained content hashes, state views, seed files, and task views. Missing bytes must remain visible as unseeded or unreadable state.
- Keep build source contracts, runtime capability checks, build capture commands, package or download metadata, and export result contracts aligned when provisioning changes.
- Keep task checks, readers, generated steps, generated verifiers, task evaluation entries, and witness comparison aligned when task semantics change.
- Recheck owned home paths, selected-path aliases, launch argument rewriting, identity scrubbing, withheld files, and dconf handling together when state mapping changes.
- Keep action ordering, reset semantics, settle results, screen examples, recording segments, and witness stage admission aligned when replay behavior changes.
- Keep guest data-size limits consistent across
data/get, state readers, task readers, command output, and recording transfer when resource limits change. - Keep task digests, solve records, bundle README updates, and per-bundle write ordering aligned when task measurement changes.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A matching application window is already open | Old or unsaved state can enter the baseline | Capture refuses the session before launch |
| A changed or seeded file has no retained bytes | State inspection cannot read it and export cannot seed it | The capture records the omission and export reports it as unseeded |
| A view is truncated or unreadable where a task needs it | The task result cannot prove that the check passes or fails | Evaluation marks the check undecided and admission refuses proof based on it |
| A task passes on the seed or fails on its selected end | The task does not prove a useful user change | Task admission refuses the task |
| A task names an unowned, withheld, or identity-bearing path | The exported app cannot evaluate the check consistently | Admission checks owned paths, seeded files, withheld files, and identity-safe values |
| A screen task lacks seed or end screenshots | Its judge has no examples for the false and true states | Export requires captured checkpoint screenshots |
| A witness stage has no required action or has an unknown reset | Replay cannot prove the intended edit or reset | Stage admission requires ordered actions and refuses unknown reset state |
| A bundle task or verifier differs from the task held by the witness | The record could describe a different task than the exported bundle | Witness admission compares the task files before recording stages |
| A build computer, command, reader, or output cleanup is not verified | A temporary computer, process, recording, or output handle can remain active | Ownership stays addressable and cleanup can be retried |
| Seed files retain personal data or host identity | Sharing the bundle can disclose capture data | Export reports scrubbed and unscrubbed files; review local bundles |
| A task measurement stops after partial solves or cannot read prior records | The difficulty result is incomplete or a prior record could be lost | Measurement retains completed solves and refuses to replace unreadable records |
Key files
| File | Why to read it |
|---|---|
| src/…/DesktopApplicationCapture.ts | Main capture flow from host admission through session recording and evidence publication |
| src/…/DesktopApplicationSession.ts | Session lifetime, checkpoint timing, observation loop, and process cleanup |
| src/…/HomeStateWalker.ts | Bounded file-state walks and their completeness limits |
| src/…/DesktopContentStore.ts | Content retention, deduplication, and omission reasons |
| src/…/DesktopApplicationState.ts | Conversion from captured bytes to time-based state views and changes |
| src/…/DconfDatabaseReader.ts | Isolated reading of captured dconf state |
| src/…/DesktopTaskDraft.ts | Task suggestions derived from observed state changes |
| src/…/DesktopTaskReaders.ts | Controlled readers for application-specific task files |
| src/…/DesktopTaskExport.ts | Task admission and generation of procedures, verifiers, and evaluation entries |
| src/…/DesktopApplicationExport.ts | Export planning, task admission, seed handling, and bundle publication |
| src/…/DesktopExportLaunch.ts | Reproducible launch paths, document arguments, and home identity mapping |
| src/…/DesktopExportReaders.ts | Task views from shipped seed files, captured checkpoints, readers, and screenshots |
| src/…/appExport.ts | Exported app launch, owned paths, data hooks, seed, and README contract |
| src/…/DesktopBuildComputers.ts | Lifecycle owner for temporary build computers, actions, secrets, and teardown |
| src/…/DesktopBuildCapture.ts | Build capture, runtime checks, checkpoints, screenshots, consent, and build export flow |
| src/…/DesktopBuildCommands.ts | Bounded and redacted command results exchanged with the build computer |
| src/…/DesktopBuildSolves.ts | Fresh-computer task measurement and difficulty record publication |
| src/…/DesktopBuildWitness.ts | Ordered task replay checks and witness stage handling |
| src/…/DesktopBuildWitnessBundle.ts | Bundle-task matching and screen-task judging |
| src/…/DesktopBuildWitnessRecord.ts | Recording transfer and witness Evidence publication |
| src/…/appBuild.ts | Supported build sources and build-computer bundle plan |
| src/…/buildWitness.ts | Witness stage order, action requirements, and reset comparison |
| src/…/AgentEnvComputer.ts | Computer readiness, desktop driver, app data, recording, file transfer, and teardown boundary |