Here is the whole protocol told as one story. A user saves a file on one device. We follow that save — through the engine, the blob store, the server, and back out across the network — until every other device has it. Each piece of the protocol enters the story at the moment the save needs it, and not before.
1. A save in Finder
The user is editing report.pdf in some app and hits ⌘S. The app writes the file. The operating system tells our OS integration layer that something changed — on macOS through a File Provider callback, on Windows through a CFAPI callback. The adapter turns that OS-specific signal into one plain message for the engine: this item was modified.
That is all the engine ever sees of the operating system. The deep mechanics — how placeholders register, how Finder enumeration works, how the OS decides when to call us — are the OS adapter layer's job, and the subject of chapter 5. For the journey, the only thing that matters is that a change has arrived.
2. The engine queues it
The engine does not call the network. The first thing it does is write the change down. It generates an op_id — a unique identifier for this one operation — and records a pending op in its local SQLite database describing the modification. Only after that record is durable does anything else happen.
The order is deliberate: the op_id is persisted before any network call. This is what makes the engine crash-safe.
op_id is written to pending_ops in SQLite before any network call. If the device crashes after the server commits but before the client receives the response, the engine retries on restart with the same op_id. The server looks it up in idempotency_keys and returns the original result — the mutation is not applied twice. Because the op_id never changes across retries, every retry is idempotent.
3. Blob first, always
Before the engine tells the server anything about the change, it deals with the bytes. It reads the file's content and hashes it. That content hash is the file's identity in the blob store — identical content always produces the same hash, so two devices that save the same bytes share one blob and pay for storage once.
The engine then uploads the blob: PUT /blobs/{content_hash}. The upload is idempotent, so a failed-and-retried upload costs nothing extra. Only once the bytes are safely stored does the engine move on to describing the change. The ordering is not an optimization — it is a protocol invariant.
PUT is idempotent.
4. The mutation
With the bytes in place, the engine submits the actual change as a typed mutation request — a proposal the server may accept or reject. For our save it is a ModifyFile: it names the item, carries the content hash, and includes the base_item_version — the version the device believed the file was at when it started. That field is the optimistic-concurrency precondition: it lets the server decide whether this change is built on current truth or on a stale view.
The mutation request types
enum MutationRequest {
CreateFile {
op_id, parent_item_id, item_id, name,
content_hash, size // blob must already be uploaded
},
CreateFolder { op_id, parent_item_id, item_id, name },
ModifyFile {
op_id, item_id,
base_item_version, // optimistic concurrency precondition
content_hash, size
},
Delete { op_id, item_id, base_item_version },
MoveRename {
op_id, item_id, base_item_version,
to_parent_item_id, new_name // atomic — never decomposed into delete + create
},
}What the server validates
| The server accepts only if… |
|---|
| Device credential is valid and not revoked. |
op_id is new, or exactly repeats a prior operation (idempotent replay). |
| Name passes cross-platform validation. |
| No live sibling with the same normalized name at the target parent. |
base_item_version matches the current server version for existing-item mutations. |
| Parent folder exists and is not deleted. |
| File content blob is already present in the blob store for file mutations. |
5. The server commits
The server checks the preconditions. If base_item_version still matches, it commits in a single transaction: it updates the item record, bumping item_version; it appends an entry to the change_log, which assigns the change a global seq; and it records the result against the op_id in idempotency_keys so a retry returns the same answer. Then it replies to the originating device, which marks its pending op complete.
Here is the whole local-to-cloud path — from the file change on disk to the committed seq on the server.
The rejection branch is the seed of the next chapter. When two devices race for the same file, only the first whose base_item_version still holds wins; the other gets StaleBaseItemVersion and preserves its content as a conflict copy. That story is Truth & Conflict. For now, our save was accepted: it is seq N.
6. The wake
The change is committed, but the other devices do not know yet. The moment the transaction commits, the server issues a Postgres NOTIFY on the vault's channel carrying nothing but latest_seq. The in-process Wake Hub is listening; it broadcasts to every device currently connected to that vault over its WebSocket. Each online device receives a single small message: something changed, up to seq N.
latest_seq — a hint that something changed, never the change itself. Devices never apply a wake payload directly. Correctness never depends on a hint arriving: a dropped or delayed wake is harmless, because the next poll or reconnect catches up. The WebSocket is a latency optimisation, not a correctness path.
7. Device B catches up
The wake tells Device B only that it is behind. To learn what changed, it asks the server for the log after its own cursor: GET /log?after=last_seq. That cursor — last_seq_processed — is owned by the client, not the server; the server simply returns every event since whatever value the device sends. Device B gets the events in strict seq order and applies them to its local tree: upsert the item in SQLite, fetch the blob by its content hash, write the bytes.
Each change event records the device_id that originated it. That is how a device suppresses its own echo — when an event comes back bearing its own device_id, it already has that change and does not re-apply it. An offline device, like Device C below, simply gets nothing now and catches up on reconnect by replaying from its own cursor.
8. Hydration on demand
On the production adapters — macOS File Provider and Windows CFAPI — Device B does not download the bytes just because the metadata arrived. Applying the event registers a placeholder: report.pdf appears in Finder or Explorer immediately, with the right name and size, but no content on disk. Only opened files consume disk space.
The bytes are fetched lazily — at the moment the user actually opens the file. The OS calls back asking for content, the engine downloads the blob by its content hash, and the placeholder becomes a real file. Until then, the file is present in every sense the user cares about, while costing nothing.
9. The first journey
Everything above assumes Device B already had a cursor — a last_seq_processed to ask after. But there is a moment before that exists: when a device first joins the vault, or when its local state is too stale for incremental replay. Then there is no cursor to catch up from, and the engine runs bootstrap instead.
Bootstrap is snapshot plus log replay. The device fetches a snapshot — the full set of items as of some seq N — and stores it, setting last_seq_processed = N. From that point it has a cursor, and it replays the log after N exactly as in the normal cycle. Bootstrap is how a device earns the cursor that every later journey relies on.
And that is the whole protocol. A save becomes a pending op, then a blob, then a mutation; the server commits it as a seq and wakes the vault; each device pulls the log after its own cursor and applies it, hydrating bytes only when asked. A new device starts from a snapshot and joins the same stream. Every device, eventually, holds the same truth.