In Fractal, I left a depth-22 view and came back. Depth is the canvas's zoom level, from 0 to 24. On a link slowed to 10 Mbps with a 40 ms round trip, the return took 757.4 ms and downloaded 867,735 bytes. The browser already had nearly all of those bytes.
The waste came from how the client, Fractal's code in the browser, follows the canvas. The canvas is divided into squares called tiles, and the client subscribes to the ones it needs, meaning it asks the server to send each tile's strokes and keep them updated. Say I came back in the same tab and nobody had edited anything. If the client had stopped following those tiles while I was away, it asked for them again, and the server sent everything again. The same went for the client's other cache, the cell images: PNGs the server draws so a zoomed-out view can show art drawn deeper in.
I traced that repeated traffic and set the rules for when the server may skip a reply; agents wrote the code, under the two-session limit I keep. After the change, the same return took 48.8 ms and 1,380 bytes.
Reusing a copy still downloaded it#
The fix came in two rounds: images first, then strokes. An earlier change had already stopped the browser from decoding the same PNG twice. If a reply matched an image it still held, it kept the old one, but the reply still came. On a depth-13 return at a device pixel ratio (DPR) of 2, that was 196 images, every one a copy of something already in memory.
Two cheaper answers fell short. The first was to skip the request whenever the client already has a copy. The first written plan for this fix ruled that out in one line: it "misses mutations while away." Someone else can draw, delete, or clear while this tab is looking somewhere else. The second was compression, and WebSocket compression was already on for large messages. It made the downloads smaller, but the client still had to read and load everything again.
So now the browser says what it already has, and the server answers "unchanged" when it can prove it. I'll call that validation.
Asking before sending#
Validation works the same way for both caches. The browser keeps tiles of strokes and cell images in the tab's memory, and each copy carries an identity, a short value saying which version it is. The request names those identities, and the server sends one short answer listing the copies that still match, then full replies for the rest. A match tells the client to keep drawing from the copy it has, so the server can only say "unchanged" when that copy is still exactly what it would have sent.
Here is the vector version, from the protocol notes. A tile's identity is called a token. The request lists tiles to check again under refresh, even if the client is already subscribed to them, and the tokens it holds under known. The answer lists the ones that still match, each with two counts.
{"type":"subscribe","tiles":[{"z":0,"x":0,"y":0}],
"requestId":"tv-12","refresh":[{"z":0,"x":0,"y":0}],
"known":[{"z":0,"x":0,"y":0,"token":"v-812"}]}
{"type":"tile_validation_result","requestId":"tv-12","accepted":true,
"matched":[{"index":0,"descendantStrokes":12,"maxDepth":7}]}
A missing match never means a tile is empty or deleted, only that a full reply follows.
What makes an image unchanged#
An image's identity is three values, taken when its PNG was drawn. The first is the cell's revision, which changes whenever a stroke that could show up in that cell is added or removed. The second is the mark, the latest paint order that has gone into the image so far, and the third is whether the image is empty.
The mark alone wouldn't be enough. A delete changes the pixels but leaves the mark where it was, because removing a stroke never lowers it. Undoing an erase and redoing it is the less obvious case. Redo brings the eraser back at its original paint order, so neither step moves the mark, but both change the pixels. So the server confirms a cell only when its current image matches all three values exactly. If the newest revision hasn't been drawn yet, it doesn't match.
With that check in place, I measured the image round on the same slowed link, with a fixed set of 1,792 strokes and three runs of each build:1
| Depth 13, DPR 2 return | Before | After |
|---|---|---|
| Ready | 2,905.1 ms | 403.2 ms |
| Downstream wire bytes | 3,308,361 | 334,275 |
| Upstream wire bytes | 287 | 1,508 |
| Cell-image JSON body bytes | 2,979,248 | 6,780 |
Upstream grew, because each request now carries the list of what the client holds. The strokes didn't change: on that depth-13 return they were 1,189,809 bytes of JSON in both builds. That left the deep return. The depth-22 view I measured is drawn from vectors alone, so coming back to it still downloaded every stroke. Those are the 867,735 bytes in the opening, measured later on a larger set of strokes.
A version for each tile#
The strokes needed an identity of their own, and it took two designs. It lives in the hub, the one loop per canvas that tracks every connection's subscriptions and applies changes in order.
The mark was out for the same reason as with images: a delete changes what's drawn without moving it. So the first plan proposed a revision per tile, tied to the current run of the server, plus leftover revisions for tiles emptied during that run. It asked how big that leftover record could grow.
The second round replaced all of that with a token: v- plus a counter. The hub keeps tokens for up to 4,096 tiles and forgets the least recently used one when full. A forgotten token can't match, so the cost is one full reply and a fresh token. Empty tiles get ordinary tokens, so there's no record of emptied tiles to grow.
Any stroke added or removed drops the token of the tile it touched, which covers drawing, delete, undo, redo, and both ends of a move. A clear empties the hub's list of tokens, but the counter keeps counting, so no token from before a clear can equal one from after it.
Unchanged here, new art deeper#
A token covers a tile's own strokes, but a tile's reply carries more than that. It also carries two numbers about everything below it: descendantStrokes and maxDepth, the two counts in the example match. The client reads them to decide whether to ask for cell images at all. The server only sends images for cells inside tiles with at least one stroke below them.
Those counts can change from far away. The client subscribes to every ancestor of the current view, each larger tile that contains it, up to depth 0. A stroke drawn at depth 20 changes no depth-5 tile's own strokes. It does change that tile's count, and the count of every ancestor up to depth 0.
So there are two ways to get "unchanged" wrong. Put the counts inside the token, and every stroke added or removed anywhere drops the tokens of all the tiles above it, including the depth-0 tile every view subscribes to. Leave the counts out and skip the whole reply, and the client keeps old counts. New art drawn deeper may then never get its image requested.
The design that shipped does both halves: the token covers only a tile's own strokes, and every match carries the current counts. The first plan gave the reason. One stroke doesn't change the version of every larger tile, and the live updates don't have to carry versions. The server-side check (trimmed):
func (ls *loopState) matchTileValidators(c *Client, msg protocol.SubscribeMsg) map[canvas.TileKey]bool {
// ...
for i, known := range msg.Known {
key := canvas.TileKey{Z: known.Z, X: known.X, Y: known.Y}
if ls.tileIdentities.current(key) != known.Token {
continue // a full tile_data follows
}
matched[key] = true
meta := ls.hub.canvas.GetMeta(key)
results = append(results, protocol.TileValidationMatch{
Index: i, DescendantStrokes: meta.DescendantStrokes, MaxDepth: meta.MaxDepth,
})
}
c.Send(protocol.MarshalTileValidationResult(msg.RequestID, true, results))
return matched
}
The counts would go stale if a change could land between reading the token and reading the counts. None can, because reading the token, reading the counts, and queuing the answer all happen in one turn of the hub's loop. A later change waits in the same queue and drops the token as usual. The client had its own way to end up with stale counts, caught in review. If a full copy of a tile arrived while its confirmation was still pending, the client ignored the confirmation's counts. Now it applies them either way.
When a saved copy doesn't count#
A token says the client's copy matches the server's. Sometimes the client can't promise that, and then it doesn't use one.
The first case is your own drawing. The client draws your stroke before the server confirms it. These early strokes sit in the same tile lists as the server's, with IDs starting local-. A token vouches for a complete copy from the server, and a list with an unconfirmed local stroke in it isn't one.
So the client saves a token, with the connection it came on and the tile's stroke revision, only after loading a full reply with no local strokes in it. Any change to that tile's list drops the token first: your own new stroke, a confirmation, someone else's stroke, pruning, or a clear.
The second case is a new connection. A token only counts on the connection that delivered it, so after a reconnect the client gets full replies again. Images follow the same rule, and for them it also covers a server restart. Image revisions start from the server's clock when it boots, so they keep rising across a restart. That fails if the restarted server's clock is behind the last revision the old one handed out. The new server could then hand out a revision the client already holds, for different content. Since a copy only counts on its own connection, those two are never compared.
The last case is the wait for an answer. While a confirmation is pending, the old image can still show as a stand-in, but it doesn't count as current. The ready signal and export both wait for the server's answer.
Forty returns, zero repeated points#
With both rounds in, I measured the return again, this time on a fixed set of 6,481 strokes: eight views, five returns each. Before is the build with image validation. After is the same build plus vector validation:2
| Link | Depth / DPR | Ready before | Ready after | Downstream bytes before | After |
|---|---|---|---|---|---|
| Loopback | 13 / 1 | 44.4 ms | 32.0 ms | 405,310 | 1,661 |
| Loopback | 22 / 1 | 76.0 ms | 2.0 ms | 867,245 | 1,345 |
| Loopback | 13 / 2 | 49.4 ms | 34.4 ms | 411,688 | 8,074 |
| Loopback | 22 / 2 | 79.5 ms | 2.0 ms | 867,245 | 1,345 |
| 10 Mbps / 40 ms | 13 / 1 | 413.5 ms | 71.8 ms | 405,520 | 1,731 |
| 10 Mbps / 40 ms | 22 / 1 | 757.4 ms | 48.8 ms | 867,735 | 1,380 |
| 10 Mbps / 40 ms | 13 / 2 | 401.2 ms | 99.4 ms | 411,898 | 8,109 |
| 10 Mbps / 40 ms | 22 / 2 | 765.7 ms | 57.8 ms | 867,735 | 1,380 |
The depth-22 row on the slowed link at DPR 1 is the opening: 757.4 ms down to 48.8 ms, and 867,735 bytes down to 1,380. Counting stroke points, the coordinates that make up each stroke, shows where the bytes went. Before, a depth-13 return carried 36,703 stroke points and a depth-22 return 78,194. Across all 40 returns after, the count was zero. Every run still drew the same artwork.
A cold visit gets nothing#
Validation only helps when the client has copies to name. A first visit has none, so every reply comes in full. With the image change, a fresh page load on the slowed link went from 3,019.4 to 3,045.2 ms. A reload is no better off: copies live in the tab's memory, so it starts from nothing too. Cold loads like these were what the coarse preview layer went after, and it didn't ship. Across the whole round of performance work that validation belonged to, a depth-22 cold load got slightly slower.3
Where the limits land#
Validation has costs of its own, so the requests and the hub's memory all have caps, and each one gives up some matches to stay small. A vector request lists at most 128 tokens and an image request at most 256, and either message is capped at 16 KiB. Anything that doesn't fit gets a full reply. The hub keeps its 4,096 tile tokens and no history per client. If the server rejects a request, validation turns off for that connection, and the client gets one delayed full refresh rather than a loop of retries.
Every limit falls back to the old behavior, a full reply, so hitting one costs a download of strokes the browser already had and nothing worse.
Footnotes
-
Wire bytes are what actually crossed the network in the stated direction, after WebSocket compression and including framing. JSON body bytes are message sizes before compression, so the two can't be added or compared directly. The image figures use 1,792 strokes; the vector figures use a later set of 6,481. ↩
-
Medians of five returns per view, on one local machine, with headless Chrome and a FIFO proxy shaping the 10 Mbps / 40 ms link. Ready means the app had the vectors and current images it needed, not a paint timestamp from the browser's compositor. ↩
-
At DPR 1, median source-ready time went from 93.1 to 96.4 ms, and median main-thread CPU from 58.088 to 63.245 ms. I can't explain the difference. It stays in the record, and I didn't keep re-running until it went away. The check was five alternating pairs, a fresh browser against a warm server. It compares the build before vector validation with the build after the round's last performance change. In between, the round also shipped binary PNG frames, stroke JSON serialized once for many viewers, and new loading feedback. So the difference can't be pinned on validation alone. CPU is browser main-thread task time. Earlier grouped samples of the vector change alone were also slower; the report treats this final check as replacing them. ↩