sg

← Writing

My Rate Limiter Caught Me First

· systems · reliability

The limits I built to keep abusers off my canvas caught me first. Within ten hours of turning them on, they had slowed my zooming and shut out my own stress tool. Both times I fixed it by raising a number. Months later, one of those numbers went so high that the check behind it could no longer say no.

Fractal is a shared drawing canvas I built in Go, with agents writing most of the code while I make the calls. Each browser keeps one connection to the server open the whole time (a WebSocket). Strokes are stored in tiles, and you can zoom in more or less forever. Before putting it on the public internet, one of the first rules in my plan was to check and rate-limit every message before the rest of the server sees it.

Day one, 16:42: the limits go in#

That rule became one commit at 16:42 on the first day, and it added three layers.

First, every message gets checked. Second, every client gets a token bucket for each kind of traffic: strokes, tile subscriptions, cursor moves, and raw bytes. A bucket works like an allowance: each message spends from it, and it refills at a fixed rate up to a cap called the burst. A client that goes over its limits more than 50 times in a minute gets disconnected. Third, a connection guard caps the server at 500 connections in total and 10 per IP address, and it does this check before the WebSocket connection is set up.

The guard is short, so here it is, trimmed. It checks the server-wide total before it looks at the count for one address:

go
func (g *ConnectionGuard) TryAcquire(ip string) bool {
	g.mu.Lock()
	defer g.mu.Unlock()

	if g.total.Load() >= int64(g.maxTotal) {
		return false
	}
	if g.perIP[ip] >= g.maxPerIP {
		return false
	}
	g.perIP[ip]++
	g.total.Add(1)
	return true
}

18:25: my own zooming hits the limit#

The first thing to hit the limits was my own zooming, less than two hours later. Every pan or zoom can bring a new set of tiles into view, and the browser has to ask the server for updates on each one. Fast panning and zooming at deep levels changed the view often enough to run through the tile bucket.

So at 18:25 I loosened it. The burst went from 5 to 20, and the refill from 10 per second to 30. The commit gives the reason in plain terms: the old limit was too easy to hit. The fastest real user was the one who wrote it.

02:34: my own stress tool gets locked out#

The second catch came overnight. At 02:30 the next morning I committed a stress tool. It pretends to be any number of clients at once, all drawing and moving cursors, and measures how long the server takes to respond.

The guard was always going to turn it away. By default the tool pointed at localhost, my own machine, so every fake client came from the same address. The example command in that commit asks for 500 clients. The guard would let in 10 and turn the rest away with a 429 error saying my IP had too many connections.

Four minutes later, at 02:34, the limits went to 5,000 connections in total and 100 per IP. The commit says the old ones blocked stress testing on my own machine entirely. It also states the assumption behind a per-IP cap: "each real user has a unique IP."

May: the per-IP check stops firing#

In May I raised the limits again, and this time one of them stopped doing anything. The commit's whole message is "agents", and it set both constants to 50,000:

diff · go
 const (
-	MaxTotalConnections  = 5000
-	MaxConnectionsPerIP  = 100
+	MaxTotalConnections  = 50000
+	MaxConnectionsPerIP  = 50000
 )

The per-IP cap now equals the total cap. For one address to reach 50,000, the server total has to be at least 50,000 too, and the guard checks the total first. So the per-IP check is still in the code, but it can never be the one that says no.

There's no reasoning behind 50,000 to report. I bumped it and forgot. A commit message that just says "agents" is what vibe-coded work looks like when nobody reviews the change. On this project, nobody is me.

One address is not one person#

The assumption in that 02:34 commit, one real user per IP, holds for one person at one desk. It fails when many people share one address: an office, a school, or a phone carrier that puts many customers behind a shared pool of addresses. To the guard, a school looks just like my stress tool did, with one address and lots of busy connections, so a limit tuned for one person per address punishes everyone behind it.

Since then, on September 30, the per-IP default went back to 100, so it can say no again. The total stayed at 50,000 and is now counted across the whole server instead of per canvas, and both limits can be set by environment variable at startup. That makes the check work again, but the counter still has no way to tell one busy person from a school.