Introducing a new way to review your PRs.

Read the announcement

thedotmack / claude-mem

Persistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More

View on GitHub

47 documents · Updated Nov 20, 2026

docs / sync-api-service

Sync API service

The Sync API service is the long-running client endpoint for cloud-synchronized clients. In hub mode it authenticates client identity, stores one ordered canonical operation log per user, helps clients catch up, and sends new-change hints while a separate projector updates canonical application data. In forward mode it keeps the client endpoint stable, streams sync requests to an upstream service, and returns the upstream result. It is the service described by the cloud sync platform, with processing and control details in sync hub worker processing and sync hub control plane.

Service role

Client synchronization boundary

In hub mode, the service is the durable boundary between a client device and the rest of the sync platform. Clients send signed-by-content operations to the hub, read changes from the hub, and use the current sequence and projection state to decide whether local data is caught up. In forward mode, the upstream service owns this boundary and the Sync API preserves the client-facing route.

The hub keeps the client path separate from projection work. A successful write is durable in the sync log before the service reports the result, while the response also states whether the projector has reached the new log position. Forward mode does not open Postgres or perform local authentication. It streams the sync request to the configured upstream and keeps the health check local.

flowchart LR
    C[Client device] -->|Bearer token + user and device identity| A[Sync API service]
    A -->|Verify token in hub mode| V[Pro identity service]
    A -->|Store and read ordered operations| D[(Postgres sync store)]
    A -->|Projection pages| P[Projection service]
    A -->|Change hints| C
    P --> D
    A -.->|Forward sync requests| U[Configured upstream]

References: services/…/index.ts• L1-23 services/…/index.ts• L381-418 services/…/forward.ts• L1-15 services/…/forward.ts• L91-117 services/…/store.ts• L191-203

User and device identity

In hub mode, every client sync route requires a bearer token and an X-User-Id header. The service asks the Pro verification endpoint for the canonical user ID and rejects a request when that ID does not match the presented value. This binding prevents a valid token from being used to address another user's log. In forward mode, the bearer and identity headers pass to the upstream service, which performs authentication.

The optional X-Device-Id and X-Device-Name headers identify the client device. Device identity enables per-device cursors and socket fan-out. A user can have at most 64 registered devices.

Positive verification results use a bounded in-memory cache. Cache failure does not grant access or turn a valid request into an authentication failure; the service can verify the token again.

References: services/…/auth.ts• L175-263 services/…/forward.ts• L18-25 services/…/forward.ts• L73-97 services/…/store.ts• L247-261 services/…/store.ts• L760-815

Canonical content

Canonical operation model

The sync log stores canonical operations rather than arbitrary client JSON. A canonical operation has a stable content or mutation identity, an entity revision, a canonical JSON body, and hashes that let the service detect changes to the same revision.

Content operations represent observations, summaries, prompts, or tombstones. Mutation operations carry controlled changes such as title updates, prompt-session links, or project remaps. Content IDs derive from content kind, origin device, and origin local ID, so the same local item has the same identity when it reaches the service again.

The service accepts only canonical JSON with known schema versions and bounded payloads. It represents sequence and revision values as unsigned decimal strings so values remain exact across clients and services.

References: services/…/canonical-content.ts• L1-40 services/…/canonical-content.ts• L136-160 services/…/canonical-content.ts• L223-268

Durable user log

Each user has an epoch, a dense ordered head sequence, a projection checkpoint, and the latest revision for each entity. Postgres transactions and a per-user lock make a push one ordered decision: a new revision gets the next sequence, an identical revision is acknowledged again, and a stale or conflicting revision is refused.

The log keeps the latest entity head separate from the ordered operation history. This lets clients replay changes in sequence order while the service still checks revision order for each entity. Device records track the last acknowledged sequence and last seen time.

References: services/…/store.ts• L187-224 services/…/store.ts• L357-471 services/…/schema.ts• L12-64

Client protocols

HTTP synchronization protocol

The HTTP API is the source of truth for client state. In hub mode it exposes three authenticated operations: push operations, read changes after a cursor, and read sync status. In forward mode, these sync routes use the same client contract while the upstream handles the request.

RouteClient purposeKey result
POST /v1/sync/opsAdd a batch of canonical operationsAcknowledged operations, the hub head, and projection state
GET /v1/sync/changes?since=<seq>Replay changes after a sequenceAn epoch, ordered operations, the head sequence, and a more flag
GET /v1/sync/statusInspect sync progressHead sequence, projected sequence, operation count, and device count

Push requests use protocol version 2 in hub mode. The service limits a request to 500 operations and an 8 MB body. A write can be durable while still returning a retryable projection error when the projector has not reached the new head, so clients must not treat a non-200 response as proof that the durable write failed. Forward mode streams the sync request to its upstream and returns its status, body, and safe response headers. Plain push bodies are compressed in transit when needed, while bodies already encoded by the client pass through unchanged.

References: services/…/index.ts• L25-31 services/…/index.ts• L151-212 services/…/index.ts• L449-495 services/…/forward.ts• L18-26 services/…/forward.ts• L61-117 services/…/forward.test.ts• L120-204 services/…/store.ts• L47-94

WebSocket change hints

In hub mode, the authenticated GET /v1/sync/ws route provides a live hint channel for connected devices. The service answers ping with pong and sends other devices either an op frame with changes or an advance frame that tells them to fetch the changes over HTTP. In forward mode, this legacy socket is retired and the route returns HTTP 410 with a polling hint that disables socket reconnects.

Socket frames are advisory. Clients must use the HTTP changes route to obtain authoritative data, recover from a missed frame, or handle a compact advance hint. A reset closes all sockets for the affected user so clients must start from the new epoch. Clients using forward mode must use HTTP polling because no local socket is available.

References: services/…/index.ts• L461-517 services/…/forward.ts• L119-131 services/…/forward.test.ts• L212-229 services/…/store.ts• L326-355 services/…/sockets.ts• L12-47

Projection protocol

Hub-to-projector handoff

The service forwards ordered log pages to the internal projector so durable sync content can become application data. The projector receives the user, epoch, sequence range, and operation bodies in projection protocol version 1, then returns the sequence through which it applied the page.

The service gives one worker at a time a short-lived per-user projection lease. It checks the returned epoch and sequence before advancing the checkpoint. A lease, bounded page, and request deadline keep a slow or split-brain projector from moving the checkpoint incorrectly.

sequenceDiagram
    participant Client
    participant API as Sync API
    participant Store as Postgres store
    participant Projector
    Client->>API: Push protocol-v2 operations
    API->>Store: Commit log and advance head
    API->>Store: Acquire projection lease
    API->>Projector: Send protocol-v1 operation page
    Projector-->>API: Confirm projected sequence
    API->>Store: Compare-and-set projection checkpoint
    API-->>Client: Ack with head and projected sequence

The normal push path drains up to eight pages while it serves the request. If projection is busy, not configured, unreachable, too slow, or returns a mismatched result, the service reports a retryable state with the durable head and current projected sequence. The internal repair route can drain one page at a time.

Projection requests use a fixed envelope and count the full serialized request toward a four-million-byte limit. This keeps the page contract stable for both the hub and the projector.

References: services/…/projection-protocol.ts• L1-73 services/…/projection.ts• L103-159 services/…/projection.ts• L165-331 services/…/store.ts• L659-818

Internal service boundary

Projection and control requests do not use client tokens. They use the configured internal projector secret and strict request shapes. In hub mode, the internal routes support projection repair, metadata reads, device renames, and a per-user reset of the hub state. Forward mode retires these routes because the upstream control plane reads the merged data directly.

Metadata reports the user epoch, log head, projection lag, sync health, and device connection state. A reset removes that user's log, entity heads, device records, and projection checkpoint, then creates a new epoch and closes the user's sockets.

References: services/…/index.ts• L117-133 services/…/index.ts• L259-379 services/…/forward.ts• L132-133 services/…/forward.test.ts• L231-237 services/…/store.ts• L280-303 services/…/store.ts• L603-657

Service operations

Configuration and startup

The service supports two runtime modes. Hub mode needs a Postgres URL and uses the configured Pro verification URL, internal projector URL, projector secret, host, port, and authentication-cache lifetime to connect its boundaries. Forward mode needs FORWARD_ORIGIN instead of Postgres and passes each /v1/sync/* request to that origin. Projection settings are optional in hub mode, but writes that need catch-up return projection_not_configured until the internal projector settings exist.

At startup, the selected mode starts its HTTP server. Hub mode applies its idempotent sync schema before it accepts traffic and opens the WebSocket path. Forward mode opens no Postgres connection. The deployed service runs on Fly with one minimum machine, HTTPS enforcement, and a /health liveness check.

SettingRole
DATABASE_URLRequired Postgres connection for the durable sync store in hub mode
TOKEN_VERIFY_URLPro endpoint that verifies the bearer token and returns the canonical user
INTERNAL_PROJECTOR_URLInternal endpoint that applies projection pages
CMEM_INTERNAL_PROJECTOR_SECRETCredential for projector and control requests
AUTH_CACHE_TTL_SECONDSPositive auth-cache lifetime bounded to 60–3,600 seconds
FORWARD_ORIGINOptional absolute HTTPS origin that selects forward mode and receives sync requests
HOST, PORTListener address and port

FORWARD_ORIGIN can include a path prefix but cannot include credentials, a query, or a fragment. Plain HTTP is accepted only for loopback development hosts. When it is set, DATABASE_URL is ignored and the service runs as a proxy.

References: services/…/env.ts• L1-20 services/…/env.ts• L24-72 services/…/index.ts• L381-418 services/…/index.ts• L531-543 services/…/fly.toml• L1-25

Health behavior

GET /health is an unauthenticated process check for the deployment platform. In hub mode it returns { "ok": true } with HTTP 200 when the process can serve requests. In forward mode it returns { "ok": true, "mode": "forward" } with HTTP 200. Neither form checks Postgres or the upstream sync service.

The health route does not prove that the database, token verification, projection, or forward upstream is available. Hub database trouble and forward upstream trouble appear as retryable request failures instead. Operators must use authenticated sync results and projection status to distinguish process liveness from sync-path readiness.

References: services/…/index.ts• L423-429 services/…/forward.ts• L100-107 services/…/forward.ts• L119-126 services/…/fly.toml• L19-32

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep the client protocol version, canonical body rules, operation hash, and revision checks aligned across push validation, stored rows, change pages, and projection pages.
  • Preserve the user and device identity binding across bearer verification, X-User-Id, X-Device-Id, X-Device-Name, device registration, and socket fan-out.
  • Keep head_seq, projected_seq, epoch checks, projection leases, and push response status aligned so a durable write is never reported as fully projected when it is not.
  • Keep WebSocket frames advisory when changing socket behavior; clients must still be able to recover through GET /v1/sync/changes.
  • Keep the projection page byte limit based on the complete wire envelope when changing projection fields or page limits.
  • Keep forward-mode route selection, allowed request headers, streamed bodies, push compression, response-header filtering, upstream error mapping, retired routes, and mode selection aligned when changing the client routes.
  • Update deployment health settings when changing the /health contract or the service mode used by the deployment.

Important failure modes

TriggerConsequenceGuard
Token verification returns a different canonical user IDThe request is rejected with HTTP 403 and no user's sync data is exposedCompare the verified ID with X-User-Id before serving the route
A device submits a stale revision or a different body for the same revisionThe batch is refused and the entity head does not moveValidate the entity revision and operation hash inside the per-user transaction
The projector is unavailable or cannot keep upThe log remains durable but the client receives a retryable projection resultReturn the head and projected sequences and let the client retry
A client misses or receives a large WebSocket frameThe client may not see each change notificationTreat sockets as hints and replay through the HTTP changes cursor
Forward mode cannot reach its upstreamThe client receives HTTP 503 with Retry-After: 5 while /health remains greenRetry the upstream request and keep /health limited to local process liveness
FORWARD_ORIGIN is malformed or uses an unsafe schemeThe service refuses to load its environment and does not start in forward modeRequire an absolute HTTPS origin, with HTTP allowed only for loopback hosts
A user exceeds the device capNew device registration is refused with device_limit_exceededEnforce the 64-device limit when a device is first touched
Hub database work times out or the pool is briefly unavailableThe affected request receives HTTP 503 with a retry hint while the process stays liveMap transient store failures to sync_hub_unavailable and let the client retry

Key files

FileWhy to read it
services/…/index.tsMaps public and internal routes to authentication, storage, projection, sockets, and health behavior
services/…/auth.tsDefines the token verification, canonical user binding, rejection contract, and cache behavior
services/…/canonical-content.tsDefines the canonical operation identity, content kinds, hashes, revisions, and validation rules
services/…/store.tsDefines the per-user ordered log, device cursors, idempotent writes, status, and projection checkpoints
services/…/projection.tsDescribes the lease-based handoff from the durable log to the projector
services/…/projection-protocol.tsDefines the stable Hub-to-projector request envelope and page limits
services/…/forward.tsDefines forward mode, streamed request pass-through, push compression, upstream failures, and retired routes
services/…/sockets.tsDefines the in-process registry for advisory per-user WebSocket notifications
services/…/schema.tsShows the durable state model used by the sync service
services/…/env.tsDefines hub and forward mode selection plus service settings and safe defaults
services/…/fly.tomlShows the deployed listener, health check, and runtime shape