Introducing a new way to review your PRs.

Read the announcement

earendil-works / pi

AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI

View on GitHub

13 documents · Updated Nov 19, 2026

docs / env

Remote execution environment package (packages/env/**)

Overview

Path routing / coverage: route changes under packages/env/** to this document. This includes the public TypeScript API in packages/env/src/**, the Rust daemon in packages/env/daemon/**, package docs in packages/env/docs/**, package metadata and README/changelog, and tests under packages/env/test/**.

packages/env provides @earendil-works/pi-env: a Durable ExecutionEnv whose operations run on another machine through a small Rust daemon. The public exports are centralized in packages/…/index.ts: Connection, RemoteExecutionEnv, SSH bootstrap helpers, LocalExecutionEnv, openTerminal, openProcess, and RemoteWatcher.

The client/daemon boundary is Connection in packages/…/connection.ts and the Rust server in packages/…/main.rs. Connection lazily starts a daemon command, appends serve --token <hex>, waits for the sync line PI-ENV <token>, sends length-prefixed frames, tracks request ids, dispatches event frames, sends cancels on abort, and restarts on the next request after a lost session. The frame format and operation contract are documented in packages/…/protocol.md and implemented on the daemon side by packages/…/frame.rs.

RemoteExecutionEnv in packages/…/remote-env.ts is the Durable environment adapter. It resolves paths using the remote machine's hello info (home, cwd, Windows driveCwds) and Node path rules, converts daemon RemoteErrors into Durable FileError/ExecutionError shapes via packages/…/errors.ts, and implements file operations by sending daemon ops such as lstat, open, pread, write, opendir, readdir, rm, and exec. It keeps Node compatibility in the client for read chunking, readFile size limits and decoding, abort checkpoints, directory sort behavior, environment merging, exec timeout validation, and command cleanup tracking.

The daemon entry point packages/…/main.rs owns remote state for one connection: open file/directory handles, command controls, process groups, PTYs, and long-running process requests. Short file operations run on a fixed worker pool; exec, watch, pty, and process get dedicated threads. packages/…/output.rs is the single stdout writer: control frames, pings, results, and watcher events take priority over bulk command output, while command readers wait when unsent bulk output exceeds the daemon's limit. On EOF, silence, or Unix exit signals, main.rs kills process groups it started before exiting.

The file-system implementation is split between client semantics and daemon syscalls. packages/…/fs.rs implements Node-like file behavior where Rust differs: recursive mkdir, writes that create parents, Node-style rm, mkdtemp, regular-file reader validation, nonblocking positional reads, directory entries with lossy UTF-8 names plus raw bytes, and Node-shaped metadata. OS-specific behavior lives in packages/…/unix.rs and packages/…/windows.rs; those files map OS errors to libuv/Node codes, choose shell behavior, set up child process groups or Windows job objects, implement pipe reads/writes, Windows argument quoting, Windows PATH lookup, and platform-specific path identity.

Command execution has three surfaces:

  • RemoteExecutionEnv.exec() in packages/…/remote-env.ts calls daemon op exec, implemented by packages/…/exec.rs. String commands run through the remote shell; argv commands run directly. Output is decoded text, optionally windowed/spilled, and failures map to Durable execution errors.
  • openTerminal() in packages/…/terminal.ts calls daemon op pty, implemented by packages/…/pty.rs. It runs an interactive program in a pseudo-terminal, sends raw terminal bytes, supports input, resize, foreground-job lookup, and is Unix-only; Windows returns ENOTSUP.
  • openProcess() in packages/…/process.ts calls daemon op process, implemented by packages/…/process.rs. It starts a long-lived argv program with plain stdin/stdout/stderr pipes for protocol-speaking servers, sends raw byte output events, queues stdin writes in order, and kills the process group/job object when the program exits or is closed.

File watching is remote but Durable-compatible. RemoteExecutionEnv.watch() resolves targets and opens RemoteWatcher in packages/…/watch.ts; the daemon implementation in packages/…/watch.rs ports Durable's NodeFileWatcher close to the files. Native events trigger debounced rescans; Windows and unreliable network/FUSE filesystems poll; a lost client connection makes the TypeScript watcher reconnect and report overflow because changes while disconnected were missed.

Tree listing is a daemon feature separate from Durable's ordinary directory readers. packages/…/list-files.ts checks hello.features for listFiles, sends the bounded listing request, decodes payload records into { path, kind }, and also defines LocalExecutionEnv, which is NodeExecutionEnv plus listFiles through a local daemon. The daemon code in packages/…/list.rs uses the ignore crate to list files, directories, and unfollowed symlinks below a root, applies gitignore rules inside work trees, excludes .git and caller-provided names, and stops with truncated at entry, byte, or timeout limits.

SSH bootstrap and deployment are in packages/…/ssh.ts. sshArguments() builds a locked-down ssh invocation with batch mode, no forwarding, no control master, no remote commands from config, no locale forwarding, strict host-key checking against the application's own knownHostsFile, and a fixed HostKeyAlias. scanHostKey(), acceptHostKey(), and forgetHostKey() manage trusted host keys; changed keys are refused until forgotten. detectPlatform() probes POSIX via uname and Windows via PowerShell, including Git Bash-on-Windows detection and Termux warnings. deployDaemon() chooses a packaged daemon path, verifies or uploads it under the remote home as a SHA-256-named binary, validates the upload before renaming into place, and removes older daemon files. connectSsh() performs detection/deployment eagerly; sshConnection() wraps the same flow in a lazy Connection command provider that retries failed starts.

The package docs are part of the module contract. packages/…/README.md gives usage and supported systems. packages/…/protocol.md is the wire and operation contract for TypeScript/Rust compatibility. packages/…/semantics.md records which Node/Durable semantics are implemented on the client versus daemon. packages/…/build.rs injects the npm package version into the daemon so hello.version reports the release it shipped with.

The tests under packages/env/test/** exercise the package at several layers: conformance.test.ts runs Durable's environment conformance suite against RemoteExecutionEnv and LocalExecutionEnv; differential.test.ts compares random file operation sequences and Durable tools against NodeExecutionEnv; ssh.test.ts spins up a disposable sshd to verify host-key trust, deployment, lazy SSH starts, login-shell behavior, and listFiles over SSH; remote.test.ts covers output windows, aborts, stale handles, chunked file transfers, and old-daemon feature fallback; process.test.ts and terminal.test.ts cover the long-running process and PTY surfaces; remote-abort.test.ts covers hosts that never answer or stop answering.

Coupling

Risks

  • packages/…/exec.rs, packages/…/process.rs, packages/…/pty.rs, and packages/…/output.rs are fragile around process output and backpressure. packages/…/CHANGELOG.md records a fix where exec, pty, and process could lose up to a pipe buffer when a program exited while the link was backed up; if output draining or Gate::held() behavior changes, large-output callers can miss trailing bytes.
  • packages/…/windows.rs has a known cleanup gap for Windows job objects: packages/…/protocol.md notes that a terminated daemon cannot clean up processes started outside the job-object window, and packages/…/windows.rs documents that a child started before assignment can remain outside the per-process job. Changes to Windows spawning or job assignment can leave remote helper processes running after close(), lost connections, or openProcess() exits.
  • packages/…/connection.ts abort handling is timing-sensitive. packages/…/CHANGELOG.md records a fix for requests hanging when a remote host stopped answering, and packages/…/remote-abort.test.ts asserts both startup aborts and sent-request abort grace. If ABORT_GRACE_MS, keepalive timers, or request teardown change, callers may hang past command timeouts or lose spill/error information from a late daemon response.
  • packages/…/ssh.ts is security-sensitive. Host keys are stored under a fixed alias and acceptHostKey() refuses changed keys; sshArguments() disables forwarding and shared connections. Weakening those options, auto-accepting changed keys, or changing knownHostsFile handling can make remote execution trust the wrong host or inherit unsafe user SSH config behavior.
  • packages/…/remote-env.ts stale handle protection depends on daemon session ids from packages/…/connection.ts. packages/…/remote.test.ts covers a killed daemon where a new daemon reuses numeric handle ids; if session checks are removed or bypassed, reads/writes against an old handle can operate on a different file in a new daemon session.