TitanSync v0.15.0

Architecture

The engine is four layers. Reconcile detects that a file is out of date, encoding determines how to describe the difference, sourcing locates the missing bytes and the peers holding them, and the content substrate cuts and names bytes so they can be deduplicated and pulled from several peers at once.

Reconcile

• A manifest is every path a node tracks, each with its kind, a content fingerprint, size, mode, mtime and version. Two nodes reconcile by trading manifests and diffing them
• Vector clocks are a map of device to counter per file. Comparing two tells you whether one descends from the other or whether they are concurrent, which a timestamp or single counter cannot do, and that distinction is what turns two offline edits into an explicit conflict instead of one side vanishing
• Conflicts settle with keep-mine, keep-theirs or keep-both; deleted files stay recoverable in a trash
• Deletes travel as tombstones, so a delete is never read as an absence
• The dirty-drain is the push trigger. A watcher on fsnotify marks changed paths dirty and a drain loop debounces them, since editors save in bursts, with a max-staleness cap so a file under constant edits still syncs on a deadline
• The Merkle summary folds a digest over each directory up to one root. Peers compare roots first and stop there when they match, and descend only into the subtrees whose digests differ when they don't. Full manifest exchange is the fallback when too much has changed
• A periodic reconcile is the backstop: it converges regardless of dropped watcher events, crashes or partitions, and writes its plan to durable storage before executing, so an interrupted sync resumes instead of restarting

Encoding

• FULL sends the file verbatim. It applies when there is no prior version to diff against, when the file is under about 256 KiB, and it is the fallback for everything else
• DELTA is a librsync rolling-checksum diff against a signature of the receiver's previous copy, restricted to text under about 8 MiB, which is where that pays off
• BLOCK is a fixed-size block diff for binaries and large text, sending only the blocks that differ. Simpler and more predictable than a rolling diff on that content
• Any of the three bails to FULL if the patch would not be smaller than the file. Large payloads are zstd-compressed on the wire; deltas and block-patches are already dense, so they are not

Sourcing and content

• Content-defined chunking (FastCDC) puts boundaries where a rolling fingerprint hits a pattern, so a mid-file edit disturbs one chunk instead of shifting every boundary after it. The function is seeded identically everywhere, so every peer cuts a given file into exactly the same chunks
• Every chunk is named by the SHA-256 of its bytes, and that hash is its address. Two identical chunks anywhere in the system are the same object, so dedup needs no separate index
• Whole-file dedup runs first and uses no network: if the content already exists under another path, it is rebuilt locally. Renames and copies transfer no bytes
• Peers advertise which chunks they hold by hash, riding along with the manifest exchange. Above 128 MiB a swarm fetch fans requests across everyone holding a chunk, and a stalled source's share is reassigned
• Every optimization degrades: if a delta cannot be computed or too few peers hold the chunks, the transfer sends the whole file and still completes

Trust

• Each node holds its own ed25519 device key and its own signed record of who belongs to the workspace
• Noise/TLS over TCP or QUIC, with every connection authenticated by that device key
• An invite is a relay address and a network id. Reaching the relay with one gets you nothing until an admin approves the device, and that membership record is signed
• Offline enrollment by hand-passed payloads, plus a recovery account for losing every device
• Assembly is checked three times, because chunks arrive from other people's machines: each chunk against its expected hash before it is written, the assembled file against the hash in its manifest, then against the fingerprint reconcile asked for. Advertised availability is re-checked the same way, so a peer cannot cause another to accept the wrong bytes

Testing

• 41 plans, each provisioning real VMs and running against a real network
• NAT hole-punching, relay and Redis crashes, full disks, corrupted payloads, revoked peers, mass deletes, macOS/Linux case collisions
• Running all 41 end to end takes 9 to 10 hours

Measured

Lab suite on libvirt VMs at 1 vCPU and 1–1.5 GB RAM each.

Control is the same run with that optimization switched off.

Placing one file costs 0.092 ms in a 12k-entry tree and 0.119 ms at 30k, so the apply path is flat in tree size rather than linear.