Sync API service
The Sync API service is the long-running client endpoint for cloud-synchronized clients. In hub mode it authenticates client identity, stores one ordered canonical operation log per user, helps clients catch up, and sends new-change hints while a separate projector updates canonical application data. In forward mode it keeps the client endpoint stable, streams sync requests to an upstream service, and returns the upstream result. It is the service described by the cloud sync platform, with processing and control details in sync hub worker processing and sync hub control plane.
Service role
Client synchronization boundary
In hub mode, the service is the durable boundary between a client device and the rest of the sync platform. Clients send signed-by-content operations to the hub, read changes from the hub, and use the current sequence and projection state to decide whether local data is caught up. In forward mode, the upstream service owns this boundary and the Sync API preserves the client-facing route.
The hub keeps the client path separate from projection work. A successful write is durable in the sync log before the service reports the result, while the response also states whether the projector has reached the new log position. Forward mode does not open Postgres or perform local authentication. It streams the sync request to the configured upstream and keeps the health check local.
flowchart LR
C[Client device] -->|Bearer token + user and device identity| A[Sync API service]
A -->|Verify token in hub mode| V[Pro identity service]
A -->|Store and read ordered operations| D[(Postgres sync store)]
A -->|Projection pages| P[Projection service]
A -->|Change hints| C
P --> D
A -.->|Forward sync requests| U[Configured upstream]
References: services/…/index.ts• L1-23 services/…/index.ts• L381-418 services/…/forward.ts• L1-15 services/…/forward.ts• L91-117 services/…/store.ts• L191-203
User and device identity
In hub mode, every client sync route requires a bearer token and an X-User-Id header. The service asks the Pro verification endpoint for the canonical user ID and rejects a request when that ID does not match the presented value. This binding prevents a valid token from being used to address another user's log. In forward mode, the bearer and identity headers pass to the upstream service, which performs authentication.
The optional X-Device-Id and X-Device-Name headers identify the client device. Device identity enables per-device cursors and socket fan-out. A user can have at most 64 registered devices.
Positive verification results use a bounded in-memory cache. Cache failure does not grant access or turn a valid request into an authentication failure; the service can verify the token again.
References: services/…/auth.ts• L175-263 services/…/forward.ts• L18-25 services/…/forward.ts• L73-97 services/…/store.ts• L247-261 services/…/store.ts• L760-815
Canonical content
Canonical operation model
The sync log stores canonical operations rather than arbitrary client JSON. A canonical operation has a stable content or mutation identity, an entity revision, a canonical JSON body, and hashes that let the service detect changes to the same revision.
Content operations represent observations, summaries, prompts, or tombstones. Mutation operations carry controlled changes such as title updates, prompt-session links, or project remaps. Content IDs derive from content kind, origin device, and origin local ID, so the same local item has the same identity when it reaches the service again.
The service accepts only canonical JSON with known schema versions and bounded payloads. It represents sequence and revision values as unsigned decimal strings so values remain exact across clients and services.
References: services/…/canonical-content.ts• L1-40 services/…/canonical-content.ts• L136-160 services/…/canonical-content.ts• L223-268
Durable user log
Each user has an epoch, a dense ordered head sequence, a projection checkpoint, and the latest revision for each entity. Postgres transactions and a per-user lock make a push one ordered decision: a new revision gets the next sequence, an identical revision is acknowledged again, and a stale or conflicting revision is refused.
The log keeps the latest entity head separate from the ordered operation history. This lets clients replay changes in sequence order while the service still checks revision order for each entity. Device records track the last acknowledged sequence and last seen time.
References: services/…/store.ts• L187-224 services/…/store.ts• L357-471 services/…/schema.ts• L12-64
Client protocols
HTTP synchronization protocol
The HTTP API is the source of truth for client state. In hub mode it exposes three authenticated operations: push operations, read changes after a cursor, and read sync status. In forward mode, these sync routes use the same client contract while the upstream handles the request.
| Route | Client purpose | Key result |
|---|---|---|
POST /v1/sync/ops | Add a batch of canonical operations | Acknowledged operations, the hub head, and projection state |
GET /v1/sync/changes?since=<seq> | Replay changes after a sequence | An epoch, ordered operations, the head sequence, and a more flag |
GET /v1/sync/status | Inspect sync progress | Head sequence, projected sequence, operation count, and device count |
Push requests use protocol version 2 in hub mode. The service limits a request to 500 operations and an 8 MB body. A write can be durable while still returning a retryable projection error when the projector has not reached the new head, so clients must not treat a non-200 response as proof that the durable write failed. Forward mode streams the sync request to its upstream and returns its status, body, and safe response headers. Plain push bodies are compressed in transit when needed, while bodies already encoded by the client pass through unchanged.
References: services/…/index.ts• L25-31 services/…/index.ts• L151-212 services/…/index.ts• L449-495 services/…/forward.ts• L18-26 services/…/forward.ts• L61-117 services/…/forward.test.ts• L120-204 services/…/store.ts• L47-94
WebSocket change hints
In hub mode, the authenticated GET /v1/sync/ws route provides a live hint channel for connected devices. The service answers ping with pong and sends other devices either an op frame with changes or an advance frame that tells them to fetch the changes over HTTP. In forward mode, this legacy socket is retired and the route returns HTTP 410 with a polling hint that disables socket reconnects.
Socket frames are advisory. Clients must use the HTTP changes route to obtain authoritative data, recover from a missed frame, or handle a compact advance hint. A reset closes all sockets for the affected user so clients must start from the new epoch. Clients using forward mode must use HTTP polling because no local socket is available.
References: services/…/index.ts• L461-517 services/…/forward.ts• L119-131 services/…/forward.test.ts• L212-229 services/…/store.ts• L326-355 services/…/sockets.ts• L12-47
Projection protocol
Hub-to-projector handoff
The service forwards ordered log pages to the internal projector so durable sync content can become application data. The projector receives the user, epoch, sequence range, and operation bodies in projection protocol version 1, then returns the sequence through which it applied the page.
The service gives one worker at a time a short-lived per-user projection lease. It checks the returned epoch and sequence before advancing the checkpoint. A lease, bounded page, and request deadline keep a slow or split-brain projector from moving the checkpoint incorrectly.
sequenceDiagram
participant Client
participant API as Sync API
participant Store as Postgres store
participant Projector
Client->>API: Push protocol-v2 operations
API->>Store: Commit log and advance head
API->>Store: Acquire projection lease
API->>Projector: Send protocol-v1 operation page
Projector-->>API: Confirm projected sequence
API->>Store: Compare-and-set projection checkpoint
API-->>Client: Ack with head and projected sequence
The normal push path drains up to eight pages while it serves the request. If projection is busy, not configured, unreachable, too slow, or returns a mismatched result, the service reports a retryable state with the durable head and current projected sequence. The internal repair route can drain one page at a time.
Projection requests use a fixed envelope and count the full serialized request toward a four-million-byte limit. This keeps the page contract stable for both the hub and the projector.
References: services/…/projection-protocol.ts• L1-73 services/…/projection.ts• L103-159 services/…/projection.ts• L165-331 services/…/store.ts• L659-818
Internal service boundary
Projection and control requests do not use client tokens. They use the configured internal projector secret and strict request shapes. In hub mode, the internal routes support projection repair, metadata reads, device renames, and a per-user reset of the hub state. Forward mode retires these routes because the upstream control plane reads the merged data directly.
Metadata reports the user epoch, log head, projection lag, sync health, and device connection state. A reset removes that user's log, entity heads, device records, and projection checkpoint, then creates a new epoch and closes the user's sockets.
References: services/…/index.ts• L117-133 services/…/index.ts• L259-379 services/…/forward.ts• L132-133 services/…/forward.test.ts• L231-237 services/…/store.ts• L280-303 services/…/store.ts• L603-657
Service operations
Configuration and startup
The service supports two runtime modes. Hub mode needs a Postgres URL and uses the configured Pro verification URL, internal projector URL, projector secret, host, port, and authentication-cache lifetime to connect its boundaries. Forward mode needs FORWARD_ORIGIN instead of Postgres and passes each /v1/sync/* request to that origin. Projection settings are optional in hub mode, but writes that need catch-up return projection_not_configured until the internal projector settings exist.
At startup, the selected mode starts its HTTP server. Hub mode applies its idempotent sync schema before it accepts traffic and opens the WebSocket path. Forward mode opens no Postgres connection. The deployed service runs on Fly with one minimum machine, HTTPS enforcement, and a /health liveness check.
| Setting | Role |
|---|---|
DATABASE_URL | Required Postgres connection for the durable sync store in hub mode |
TOKEN_VERIFY_URL | Pro endpoint that verifies the bearer token and returns the canonical user |
INTERNAL_PROJECTOR_URL | Internal endpoint that applies projection pages |
CMEM_INTERNAL_PROJECTOR_SECRET | Credential for projector and control requests |
AUTH_CACHE_TTL_SECONDS | Positive auth-cache lifetime bounded to 60–3,600 seconds |
FORWARD_ORIGIN | Optional absolute HTTPS origin that selects forward mode and receives sync requests |
HOST, PORT | Listener address and port |
FORWARD_ORIGIN can include a path prefix but cannot include credentials, a query, or a fragment. Plain HTTP is accepted only for loopback development hosts. When it is set, DATABASE_URL is ignored and the service runs as a proxy.
References: services/…/env.ts• L1-20 services/…/env.ts• L24-72 services/…/index.ts• L381-418 services/…/index.ts• L531-543 services/…/fly.toml• L1-25
Health behavior
GET /health is an unauthenticated process check for the deployment platform. In hub mode it returns { "ok": true } with HTTP 200 when the process can serve requests. In forward mode it returns { "ok": true, "mode": "forward" } with HTTP 200. Neither form checks Postgres or the upstream sync service.
The health route does not prove that the database, token verification, projection, or forward upstream is available. Hub database trouble and forward upstream trouble appear as retryable request failures instead. Operators must use authenticated sync results and projection status to distinguish process liveness from sync-path readiness.
References: services/…/index.ts• L423-429 services/…/forward.ts• L100-107 services/…/forward.ts• L119-126 services/…/fly.toml• L19-32
Change checklist
When these files or sections are changed, remember to consider these:
- Keep the client protocol version, canonical body rules, operation hash, and revision checks aligned across push validation, stored rows, change pages, and projection pages.
- Preserve the user and device identity binding across bearer verification,
X-User-Id,X-Device-Id,X-Device-Name, device registration, and socket fan-out. - Keep
head_seq,projected_seq, epoch checks, projection leases, and push response status aligned so a durable write is never reported as fully projected when it is not. - Keep WebSocket frames advisory when changing socket behavior; clients must still be able to recover through
GET /v1/sync/changes. - Keep the projection page byte limit based on the complete wire envelope when changing projection fields or page limits.
- Keep forward-mode route selection, allowed request headers, streamed bodies, push compression, response-header filtering, upstream error mapping, retired routes, and mode selection aligned when changing the client routes.
- Update deployment health settings when changing the
/healthcontract or the service mode used by the deployment.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Token verification returns a different canonical user ID | The request is rejected with HTTP 403 and no user's sync data is exposed | Compare the verified ID with X-User-Id before serving the route |
| A device submits a stale revision or a different body for the same revision | The batch is refused and the entity head does not move | Validate the entity revision and operation hash inside the per-user transaction |
| The projector is unavailable or cannot keep up | The log remains durable but the client receives a retryable projection result | Return the head and projected sequences and let the client retry |
| A client misses or receives a large WebSocket frame | The client may not see each change notification | Treat sockets as hints and replay through the HTTP changes cursor |
| Forward mode cannot reach its upstream | The client receives HTTP 503 with Retry-After: 5 while /health remains green | Retry the upstream request and keep /health limited to local process liveness |
FORWARD_ORIGIN is malformed or uses an unsafe scheme | The service refuses to load its environment and does not start in forward mode | Require an absolute HTTPS origin, with HTTP allowed only for loopback hosts |
| A user exceeds the device cap | New device registration is refused with device_limit_exceeded | Enforce the 64-device limit when a device is first touched |
| Hub database work times out or the pool is briefly unavailable | The affected request receives HTTP 503 with a retry hint while the process stays live | Map transient store failures to sync_hub_unavailable and let the client retry |
Key files
| File | Why to read it |
|---|---|
| services/…/index.ts | Maps public and internal routes to authentication, storage, projection, sockets, and health behavior |
| services/…/auth.ts | Defines the token verification, canonical user binding, rejection contract, and cache behavior |
| services/…/canonical-content.ts | Defines the canonical operation identity, content kinds, hashes, revisions, and validation rules |
| services/…/store.ts | Defines the per-user ordered log, device cursors, idempotent writes, status, and projection checkpoints |
| services/…/projection.ts | Describes the lease-based handoff from the durable log to the projector |
| services/…/projection-protocol.ts | Defines the stable Hub-to-projector request envelope and page limits |
| services/…/forward.ts | Defines forward mode, streamed request pass-through, push compression, upstream failures, and retired routes |
| services/…/sockets.ts | Defines the in-process registry for advisory per-user WebSocket notifications |
| services/…/schema.ts | Shows the durable state model used by the sync service |
| services/…/env.ts | Defines hub and forward mode selection plus service settings and safe defaults |
| services/…/fly.toml | Shows the deployed listener, health check, and runtime shape |