Introducing a new way to review your PRs.

Read the announcement

morluto / rea

Reverse engineer anything with agents, from app behavior down to native binaries.

View on GitHub

27 documents · Updated Nov 20, 2026

docs / desktop-application-environments

Desktop application environments

Desktop application environments turn a Linux desktop application into a recorded, inspectable, and repeatable computer environment. The system captures a live session, explains the state that changed, helps define a task, provisions a build computer, and exports a local bundle that can start the application with known data.

This page is part of artifact and application analysis. Binary diagnostics and comparison, application reconstruction, JavaScript and desktop reconstruction, and firmware and embedded analysis cover related analysis work.

Environment lifecycle

The lifecycle keeps observed facts separate from derived products. A live session produces checkpoints and retained content. State inspection explains those checkpoints. Task planning turns a useful change into checks. Export then combines the capture, state view, and tasks into a computer app bundle.

The main flow is:

flowchart LR
    A[Installed Linux application] --> B[Capture session]
    B --> C[Checkpoints and retained content]
    C --> D[State inspection]
    D --> E[Task draft and checks]
    C --> F[Export plan]
    D --> F
    E --> F
    F --> G[Local computer app bundle]
    G --> H[Repeatable desktop environment]
    H --> I[Task witness]
    H --> J[Fresh task measurements]

The capture and state view describe what was observed or inferred. The bundle is a derived product. It keeps the capture link and its limits so an engineer can judge how closely it represents the source session.

Application capture

Capture session

The capture session starts one Linux desktop application in a controlled run and records its launch facts, processes, windows, opened files, and file changes. It checks the host, display, window manager, and output folder before launch. It refuses to start beside a matching application window because that window could add old or unsaved state to the baseline.

References: src/…/DesktopApplicationCapture.ts• L202-249 src/…/DesktopApplicationCapture.test.ts• L180-249 src/…/DesktopApplicationSession.ts• L111-131

The session records a baseline and then samples the application while it runs. A caller can request named checkpoints. Stop, application exit, a time limit, and cancellation end the run with the checkpoints already taken. The session owns the launched process group and records when cleanup cannot be verified. A failed publication removes the partial capture rather than presenting it as complete.

References: src/…/DesktopApplicationCapture.ts• L254-353 src/…/DesktopApplicationCapture.ts• L356-431 src/…/DesktopApplicationSession.ts• L214-243 src/…/DesktopApplicationSession.ts• L375-451

Capture scope

The capture scope covers application state directories, selected paths, and paths the application opens. Each checkpoint records file facts and retains file bytes within entry, file, and total content limits. Exclusions and broad home-directory selections stop the capture from becoming an unbounded copy of the host.

References: src/…/DesktopApplicationCapture.ts• L316-347 src/…/HomeStateWalker.ts• L103-167 src/…/DesktopContentStore.ts• L42-63

The retained content store uses file digests so repeated bytes are kept once. A path can be marked as changed while its bytes are omitted because it is too large, unreadable, changed during the read, or outside the content budget. Capture limitations keep these facts visible to later inspection and export.

References: src/…/DesktopContentStore.ts• L119-173 src/…/DesktopContentStore.ts• L176-248 src/…/DesktopRetainedContent.ts• L39-99

Capture content is evidence, not a disk image. A change that occurs and is undone between checkpoints is not visible. A walk can also be incomplete when it reaches an entry limit, an unreadable directory, or another filesystem. Later task drafting must treat these limits as unknown state rather than as proof that nothing changed.

References: src/…/DesktopApplicationCapture.ts• L115-169 src/…/HomeStateWalker.ts• L88-107 src/…/DesktopTaskDraftNotes.test.ts• L151-239

State inspection

State views

State inspection reads a valid capture and creates a time-based view of the paths it observed. It identifies likely application state and documents from paths and application signals. Safe readers expose text, JSON, key files, XML, zip archives, SQLite data, dconf data, and compressed documents. Binary content remains a content record without an invented structured view.

References: src/…/DesktopApplicationState.ts• L111-156 src/…/DesktopApplicationState.ts• L390-413 src/…/stateRoles.ts• L1-20

The view compares each path across checkpoints. It reports creation, deletion, content change, format change, and unreadable state. Structured views can also report added, removed, and changed values. Dconf is read from a staged database rather than the live user service. SQLite views use the recorded sidecar files when they are needed to describe committed state. Reader failures keep their command and exit details in the state evidence.

References: src/…/DesktopApplicationState.ts• L265-384 src/…/DesktopApplicationState.ts• L425-532 src/…/DesktopDconfDiagnostics.test.ts• L66-97 src/…/DconfDatabaseReader.ts• L21-61

View limits

View policy bounds the amount of text, structured data, archive data, and database data that inspection reads. A bounded view is useful for analysis but cannot prove that an omitted value is absent. Task evaluation therefore treats a needed unreadable or cut view as undecided.

References: src/…/DesktopApplicationState.ts• L535-544 src/…/DesktopApplicationState.ts• L589-627 src/…/DesktopZipView.ts• L86-137

Inspection also preserves uncertainty from the capture. It does not turn an incomplete directory walk, missing retained bytes, or an unreadable reader result into a claim that the state is absent.

References: src/…/DesktopApplicationState.ts• L455-484 src/…/DesktopApplicationState.ts• L535-557 src/…/DesktopTaskDraftNotes.test.ts• L359-400

Task planning

Task drafts

Task planning compares a seed checkpoint with an end checkpoint and proposes a portable task. A task is a user prompt plus declarative checks over home-relative application files. The draft suggests checks for file presence, lines, named values, XML text, archive entries, counts, and database rows. It also records notes when the observed change is incomplete, unowned, unreadable, or not covered by a check.

References: src/…/DesktopTaskDraft.ts• L58-181 src/…/DesktopTaskDraftNotes.test.ts• L104-149 src/…/DesktopTaskDraftNotes.test.ts• L242-356 src/…/appTaskDraft.ts• L1-20

The draft is an aid for an engineer, not a complete user request. The author must replace its prompt and keep checks that express the intended result rather than volatile values. A task can add a reader that runs the application's own headless command on a private copy when the built-in state view cannot read the document.

References: src/…/DesktopTaskDraft.ts• L36-41 src/…/DesktopTaskReaders.ts• L54-117 src/…/DesktopTaskExport.test.ts• L481-601

Task checks

Task checks describe a result without depending on model judgment for file-backed state. They can test presence, lines, values, text, rows, member counts, or a screen question. A screen check is reserved for a result that no owned file can show. Readers provide a controlled view for binary or application-specific documents.

References: src/…/DesktopTaskReaders.ts• L21-31 src/…/DesktopBuildWitness.ts• L276-324 src/…/DesktopBuildWitnessBundle.ts• L114-167

Task admission

Export can include several tasks, each with its own end checkpoint. It accepts a task only when its checks apply to owned paths, fail on the shipped seed, and pass on the selected captured end state. If a check depends on a cut or unreadable view, admission refuses to claim that the task is proved. This prevents a task from being complete before the agent starts or from describing a result the capture did not prove.

References: src/…/DesktopApplicationExport.ts• L389-413 src/…/DesktopTaskExport.ts• L549-635 src/…/DesktopTaskExport.test.ts• L214-341

An accepted task becomes an agent procedure, a verifier, and an evaluation entry in the bundle. File-backed checks use exported state views and admitted readers. Screen checks use the bundle's screen judge and require checkpoint screenshots as examples. A task can use a task-specific end checkpoint, which lets one capture support several changes.

References: src/…/DesktopTaskExport.ts• L655-719 src/…/DesktopExportReaders.ts• L62-152 src/…/DesktopTaskExport.test.ts• L344-409

Build-computer provisioning

Build sources

The build computer is a temporary agentenv computer for installing and operating a desktop application before export. A request can install from an Ubuntu archive, a package file, an AppImage, or a tarball. An existing bundle can also be started for review. A request-built computer has the application and REA runtime but no captured seed or data hooks while it is used for building.

References: src/…/appBuild.ts• L75-157 src/…/appBuild.ts• L243-339 src/…/appBuild.ts• L425-477 src/…/DesktopApplicationExport.test.ts• L201-223

Build computer lifecycle

The build computer owner starts a computer, forwards desktop actions, keeps an action log, and serializes actions and build steps per computer. It also supports task measurement on fresh computers. This gives an engineer a clean place to discover launch behavior and task actions before those facts become an exported environment.

References: src/…/DesktopBuildComputers.ts• L408-487 src/…/DesktopBuildComputers.ts• L601-705

The owner keeps a computer addressable through startup or close failures until teardown is confirmed. Server shutdown cancels work and closes every held computer. The screen judge, long-running commands, remote readers, and computer are released as one lifecycle so a failed helper does not leave a hidden build resource.

References: src/…/DesktopBuildComputers.ts• L237-334 src/…/DesktopBuildComputers.ts• L721-798 src/…/DesktopBuildComputers.cleanup.test.ts• L18-61

Build capture

Build capture runs inside a request-based build computer. It first checks that the installed REA runtime supports the export options needed for synthetic seeds and task-specific ends. It then records seed and end checkpoints, keeps optional screenshots for screen tasks, summarizes the application and task draft, and exports the bundle back to the host. A computer started from an existing bundle is for review and cannot create another build capture.

References: src/…/DesktopBuildCapture.ts• L142-224 src/…/DesktopBuildCapture.ts• L319-375 src/…/DesktopBuildCapture.ts• L620-707

Build capture can mark a sign-in window and named task checkpoints. Export uses these boundaries to separate state that can be supplied at deploy time from state that can be placed in the seed. The two sign-in boundaries are required as a pair because one boundary cannot show which files the sign-in wrote.

References: src/…/DesktopBuildCapture.ts• L479-600 src/…/DesktopBuildCapture.ts• L620-670

Build secrets

Build sessions can hold sign-in or other sensitive settings outside the exported seed. Secret values are supplied by handle and are redacted from action results, logs, and witness records. Withheld files are kept as secrets outside the bundle unless the person gives explicit consent to seed them.

References: src/…/DesktopBuildSecrets.ts• L39-46 src/…/DesktopBuildSecrets.ts• L265-319 src/…/DesktopExportWithheld.ts• L114-182

Task measurement

Task measurement runs an exported task in fresh computers and records whether its solve rate falls within the expected difficulty band. It preserves the task digest with the solve results, cost, turns, time, model, and runner versions. The result is written beside the bundle's task documentation so later measurements do not replace records that cannot be read.

References: src/…/DesktopBuildSolves.ts• L116-170 src/…/DesktopBuildSolves.ts• L257-286 src/…/DesktopBuildSolves.ts• L288-376

Measurement stops when the result is decided or when the budget cannot cover another solve. Completed solves remain attached to a failure caused by cancellation, output limits, or cleanup. This makes a partial measurement useful without presenting it as a complete difficulty result.

References: src/…/DesktopBuildSolves.ts• L301-429 src/…/DesktopBuildSolves.test.ts• L256-294 src/…/DesktopBuildCommands.ts• L144-175

Task witness

Witness stages

The task witness checks that an exported task describes a real and repeatable change. It grades the live application in four ordered stages: the seed must fail some check, the edit must pass all checks, a near miss must fail again, and reset must restore the seed result.

References: src/…/DesktopBuildWitness.ts• L174-177 src/…/buildWitness.ts• L1-24

The witness accepts a stage only when the matching desktop actions occurred. An edit or near miss needs a new applied input. The reset stage needs a reset as its last change. An unknown reset result blocks a passing witness instead of treating the reset as absent. A witness also checks that its task steps and verifier still match the bundle that started the computer.

References: src/…/DesktopBuildWitness.ts• L327-389 src/…/DesktopBuildWitness.ts• L497-657 src/…/DesktopBuildWitnessBundle.ts• L68-111 src/…/DesktopBuildComputers.ts• L662-686

Witness record

When the final stage ends, the witness writes an Evidence record with the task digest, stage results, action log, and an optional desktop recording. Recording segments are checked against their manifest and can be fetched again after a partial transfer. The record is written as a complete output tree so a failed unpublished write leaves no partial result.

References: src/…/DesktopBuildWitness.ts• L391-437 src/…/DesktopBuildWitnessRecord.ts• L99-136 src/…/DesktopBuildWitnessRecord.ts• L138-196

The witness is controlled replay, not a second run of the generated verifier. It uses the same application state views and check evaluator as the exported task. Screen tasks use the bundle's own judge on a fresh screenshot. Its record reports when the action log dropped old actions, the app state did not settle, or the desktop recording is missing.

References: src/…/DesktopBuildWitnessRecord.ts• L198-245 src/…/DesktopBuildWitnessBundle.ts• L114-167

Environment export

Export plan

Export combines the capture and state view into an agentenv computer app for Ubuntu 24.04 X11. It selects seed and end checkpoints, identifies the application's owned home paths, records package and library needs, preserves the launch command, and copies retained seed files into a new local bundle.

References: src/…/DesktopApplicationExport.ts• L203-303 src/…/DesktopApplicationExport.ts• L355-430 src/…/appExport.ts• L52-123

The exported app launches at boot and becomes ready when its application window appears. Its data hooks expose only owned paths and typed views of those files. data/add adds task inputs under owned paths and data/reset restores the captured file seed. Application settings stored in dconf are loaded separately and are not part of the file reset contract.

References: src/…/appExport.ts• L527-659 src/…/appExport.ts• L750-781 src/…/DesktopBuildComputers.ts• L174-226

Seed identity

The seed can represent a user's real state at a selected checkpoint or a synthetic state made for a build task. Text state and dconf values can replace the capture home, user, and host with the new computer's identity. Binary state and unsupported text encodings remain as captured. Export reports seeded, unseeded, scrubbed, and unscrubbed files so the bundle's privacy limits are visible.

References: src/…/DesktopApplicationExport.ts• L142-184 src/…/DesktopApplicationExport.ts• L225-287 src/…/DesktopExportFiles.ts• L107-125

Keep bundles local until their state has been reviewed. A bundle can contain personal data and can still contain identity in binary files, file names, or content that export cannot rewrite. Withheld sign-in or licence files remain outside the seed unless consent allows them in.

References: src/…/appExport.ts• L609-626 src/…/DesktopExportFiles.ts• L192-263 src/…/DesktopExportWithheld.ts• L163-182

Launch state

Export preserves the captured working directory and rewrites home-relative launch arguments for the new computer. It uses selected paths and captured documents to retarget arguments that point to the captured host. Unrelated outside-home arguments remain unchanged. When the seed contains one document and the desktop entry accepts a file, the exported app opens that document at launch. Otherwise the bundle reports why it cannot promise a document launch.

References: src/…/DesktopExportLaunch.ts• L82-207 src/…/DesktopExportLaunch.test.ts• L26-126 src/…/seedDocument.ts• L55-101

Environment interface

The exported computer presents one stable boundary for an agent. The application is ready at boot, desktop actions operate on its visible window, data/get reads owned files and typed views, data/add adds task inputs under owned paths, and data/reset restores the file seed. This boundary lets task evaluation and witness replay use the same state contract. Task readers run on private copies of owned files when a task needs an application-specific view.

References: src/…/appExport.ts• L540-559 src/…/DesktopExportReaders.ts• L62-113 src/…/DesktopBuildWitness.ts• L276-324

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep capture checkpoint names, state versions, task end selection, build summaries, and export seed selection aligned when lifecycle rules change.
  • Preserve the link between retained content hashes, state views, seed files, and task views. Missing bytes must remain visible as unseeded or unreadable state.
  • Keep build source contracts, runtime capability checks, build capture commands, package or download metadata, and export result contracts aligned when provisioning changes.
  • Keep task checks, readers, generated steps, generated verifiers, task evaluation entries, and witness comparison aligned when task semantics change.
  • Recheck owned home paths, selected-path aliases, launch argument rewriting, identity scrubbing, withheld files, and dconf handling together when state mapping changes.
  • Keep action ordering, reset semantics, settle results, screen examples, recording segments, and witness stage admission aligned when replay behavior changes.
  • Keep guest data-size limits consistent across data/get, state readers, task readers, command output, and recording transfer when resource limits change.
  • Keep task digests, solve records, bundle README updates, and per-bundle write ordering aligned when task measurement changes.

Important failure modes

TriggerConsequenceGuard
A matching application window is already openOld or unsaved state can enter the baselineCapture refuses the session before launch
A changed or seeded file has no retained bytesState inspection cannot read it and export cannot seed itThe capture records the omission and export reports it as unseeded
A view is truncated or unreadable where a task needs itThe task result cannot prove that the check passes or failsEvaluation marks the check undecided and admission refuses proof based on it
A task passes on the seed or fails on its selected endThe task does not prove a useful user changeTask admission refuses the task
A task names an unowned, withheld, or identity-bearing pathThe exported app cannot evaluate the check consistentlyAdmission checks owned paths, seeded files, withheld files, and identity-safe values
A screen task lacks seed or end screenshotsIts judge has no examples for the false and true statesExport requires captured checkpoint screenshots
A witness stage has no required action or has an unknown resetReplay cannot prove the intended edit or resetStage admission requires ordered actions and refuses unknown reset state
A bundle task or verifier differs from the task held by the witnessThe record could describe a different task than the exported bundleWitness admission compares the task files before recording stages
A build computer, command, reader, or output cleanup is not verifiedA temporary computer, process, recording, or output handle can remain activeOwnership stays addressable and cleanup can be retried
Seed files retain personal data or host identitySharing the bundle can disclose capture dataExport reports scrubbed and unscrubbed files; review local bundles
A task measurement stops after partial solves or cannot read prior recordsThe difficulty result is incomplete or a prior record could be lostMeasurement retains completed solves and refuses to replace unreadable records

Key files

FileWhy to read it
src/…/DesktopApplicationCapture.tsMain capture flow from host admission through session recording and evidence publication
src/…/DesktopApplicationSession.tsSession lifetime, checkpoint timing, observation loop, and process cleanup
src/…/HomeStateWalker.tsBounded file-state walks and their completeness limits
src/…/DesktopContentStore.tsContent retention, deduplication, and omission reasons
src/…/DesktopApplicationState.tsConversion from captured bytes to time-based state views and changes
src/…/DconfDatabaseReader.tsIsolated reading of captured dconf state
src/…/DesktopTaskDraft.tsTask suggestions derived from observed state changes
src/…/DesktopTaskReaders.tsControlled readers for application-specific task files
src/…/DesktopTaskExport.tsTask admission and generation of procedures, verifiers, and evaluation entries
src/…/DesktopApplicationExport.tsExport planning, task admission, seed handling, and bundle publication
src/…/DesktopExportLaunch.tsReproducible launch paths, document arguments, and home identity mapping
src/…/DesktopExportReaders.tsTask views from shipped seed files, captured checkpoints, readers, and screenshots
src/…/appExport.tsExported app launch, owned paths, data hooks, seed, and README contract
src/…/DesktopBuildComputers.tsLifecycle owner for temporary build computers, actions, secrets, and teardown
src/…/DesktopBuildCapture.tsBuild capture, runtime checks, checkpoints, screenshots, consent, and build export flow
src/…/DesktopBuildCommands.tsBounded and redacted command results exchanged with the build computer
src/…/DesktopBuildSolves.tsFresh-computer task measurement and difficulty record publication
src/…/DesktopBuildWitness.tsOrdered task replay checks and witness stage handling
src/…/DesktopBuildWitnessBundle.tsBundle-task matching and screen-task judging
src/…/DesktopBuildWitnessRecord.tsRecording transfer and witness Evidence publication
src/…/appBuild.tsSupported build sources and build-computer bundle plan
src/…/buildWitness.tsWitness stage order, action requirements, and reset comparison
src/…/AgentEnvComputer.tsComputer readiness, desktop driver, app data, recording, file transfer, and teardown boundary