Backup and restore workflows
Backup and restore protect the local SQLite data that supports the worker. The system creates verified snapshots, keeps a bounded set of them, and restores a chosen snapshot through a restart so the worker can replace data before it begins normal use. See Persistence and synchronization for the wider data lifecycle, Storage backends and data models for local storage, and Sync application protocol for cloud data exchange.
Snapshot lifecycle
The snapshot lifecycle turns the live database into a standalone file without blocking the worker for the full copy. It checks disk capacity first, writes to a temporary name, validates the result, and publishes the file only after validation succeeds. Automatic runs start after worker boot and repeat at a configured interval when local backups are enabled (src/…/DatabaseManager.ts• L72-76, src/…/BackupManager.ts• L42-67).
The lifecycle has four main stages:
- Check that the data store has enough free space for a full copy.
- Create a consistent database snapshot and check its integrity.
- Rename the verified temporary file to its time-based snapshot name.
- Remove snapshots older than the configured retention count.
The copy work runs outside the worker's main request path. A single-flight guard prevents a manual run and a scheduled run from producing overlapping full copies. A failed run records an error and leaves no file that can be mistaken for a usable snapshot (src/…/BackupManager.ts• L136-171, src/…/BackupManager.ts• L174-227, src/…/sqlite-file-job.ts• L14-20).
References: src/…/BackupManager.ts• L42-67 src/…/BackupManager.ts• L136-227 src/…/sqlite-file-job.ts• L14-20
Backup storage
Backup storage keeps automatic snapshots under the local data directory in backups/auto. Snapshot names use a sanitized time value, so name order also gives newest-first order. Temporary files and SQLite sidecars are not treated as snapshots (src/…/paths.ts• L130-150, src/…/snapshot-files.ts• L12-27).
Retention is part of each completed backup cycle. The manager keeps the newest configured number of snapshot files and removes older files and any related sidecars. Status reads the directory, so the last snapshot and count remain visible after a worker restart (src/…/BackupManager.ts• L119-133, src/…/BackupManager.ts• L174-194).
Restore validation and swap
Restore replaces the live database only after the chosen snapshot is copied to a staging file and made standalone. The worker first protects the current database, then performs one final file replacement. This gives the system a usable pre-restore copy and avoids leaving the new database beside sidecars from the old one (src/…/restore-swap.ts• L129-151).
The swap follows this order:
- Stage and validate the selected snapshot.
- Preserve the live database and its write-ahead log when needed.
- Checkpoint the live database, retrying briefly when the configured restore window allows it.
- Save the current database as a pre-restore snapshot.
- Carry forward the live row-id counters so later writes do not reuse old IDs.
- Move old sidecars aside and atomically publish the staged database.
If validation, checkpoint, or publication fails before the final rename, the live database remains in place and temporary files are removed. A database that cannot be opened can still be replaced because its original bytes and sidecar are saved before SQLite examines them. After a successful restore, the semantic index is marked for a rebuild because its entries describe the replaced database (src/…/restore-swap.ts• L153-245, src/…/restore-swap.ts• L247-259).
References: src/…/restore-swap.ts• L129-151 src/…/restore-swap.ts• L153-245 src/…/restore-swap.ts• L247-259
Pending restore recovery
Pending restore recovery separates the user request from the database swap. The NPX command validates that the requested file is a snapshot inside the backup directory, writes a small pending marker, and restarts the worker. The next worker boot reads that marker before opening the database (src/…/restore.ts• L1-10, src/…/restore.ts• L25-49).
The marker is cleared as soon as the worker reads it. The worker records either a successful swap or a failure in last-restore.json, so a failed request does not repeat on every boot. A failed restore leaves the database unchanged. A successful restore records the pre-restore copy and starts semantic-index recovery (src/…/restore-swap.ts• L262-309, src/…/snapshot-files.ts• L30-69).
This flow also limits the restore target to a snapshot name directly inside the backup directory. Absolute paths, nested paths, traversal paths, and other file names are rejected before the database is touched (src/…/snapshot-files.ts• L17-27).
References: src/…/restore.ts• L1-49 src/…/snapshot-files.ts• L17-69 src/…/restore-swap.ts• L262-309
NPX and runtime interfaces
The NPX interface is a client of the worker. npx claude-mem backup status reports backup and last-restore state, backup run requests one snapshot, and backup list shows available files. npx claude-mem restore <file> only stages the request; the worker owns validation and replacement (src/…/backup.ts• L20-37, src/…/backup.ts• L66-147, src/…/restore.ts• L25-49).
The runtime exposes three local endpoints:
| Interface | Purpose |
|---|---|
GET /api/backup/status | Reports configuration, snapshot state, and the last restore result. |
POST /api/backup/run | Starts one snapshot and retention cycle. |
GET /api/backup/list | Lists valid snapshot files newest first. |
The run endpoint rejects disabled backups, overlapping runs, and writes from a different localhost browser origin. There is no restore endpoint because restore must occur during the next worker boot, before the database is opened (src/…/BackupRoutes.ts• L17-32, src/…/BackupRoutes.ts• L52-99).
References: src/…/backup.ts• L20-37 src/…/backup.ts• L66-147 src/…/restore.ts• L25-49 src/…/BackupRoutes.ts• L17-32 src/…/BackupRoutes.ts• L52-99
Change checklist
When these files or sections are changed, remember to consider these:
- Keep snapshot naming, file filtering, path validation, list ordering, and retention rules aligned.
- Keep the pending marker, boot-time restore order, and last-restore result contract aligned between the NPX command and worker.
- Keep database swap cleanup safe for both the main file and SQLite sidecars on every failure path.
- Keep successful restore handling aligned with semantic-index rebuild state and row-ID continuity.
- Keep the runtime endpoint responses aligned with the NPX output and the disabled-backup behavior.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Snapshot creation fails or disk space is too low | No usable new snapshot is published | Preflight checks, temporary names, validation, and error status prevent partial files from being listed. |
| A second snapshot starts while one is running | Full database copies could overlap | The single-flight guard returns a conflict. |
| The selected snapshot is corrupt or outside the backup directory | Restore could damage the live data or read an unintended file | Path checks and staged standalone validation happen before the live swap. |
| Another process holds the live database | Restore cannot safely checkpoint or replace it | The swap retries within its window or fails without changing the live database. |
| The worker cannot apply a pending restore | A repeated boot could retry the same bad request | The marker is cleared and the result is written to last-restore.json. |
| The semantic index still reflects the replaced database | Search can miss restored rows | A successful restore marks all projects for index rebuild. |
Key files
| File | Why to read it |
|---|---|
| src/…/BackupManager.ts | Defines scheduled and manual snapshot behavior, status, and retention. |
| src/…/restore-swap.ts | Defines validation, safe database replacement, rollback cleanup, and pending restore application. |
| src/…/snapshot-files.ts | Defines snapshot names, restore markers, result records, and path safety. |
| src/…/sqlite-file-job.ts | Defines the isolated SQLite file operations used for snapshot and restore validation. |
| src/…/BackupRoutes.ts | Defines the runtime backup status, run, and list interfaces. |
| src/…/backup.ts | Defines the user-facing backup commands. |
| src/…/restore.ts | Defines the staged restore command and restart handoff. |
| src/…/DatabaseManager.ts | Shows when backup protection is enabled and attached to worker lifecycle. |