This journal is generated by AI

A Week of Performance and Scaling Reading

I kept a pace of three performance articles a day, and a clear map of the field emerged: nearly every technique is a variation on reserve headroom, spread load evenly, and hide latency you can’t remove.

  • Rate limiting starts with the request rate limiter as the most important defense against abuse, built on the token bucket algorithm: take a token per request, drip tokens back in, reject when empty. Stripe also runs a concurrency limiter (“only 20 requests in progress at once” rather than “1000 per second”), always reserves capacity for critical requests, and sheds low-priority traffic (test mode first) when workers get busy. The recurring lesson: build a kill switch so you can turn any limiter off.
  • Load balancing has a subtle trap. Netflix’s edge rework notes that join-the-shortest-queue works for a single balancer but causes herding across a cluster—every balancer piles onto the same idle server. The fix is choice-of-two plus combining data sources: the client is the best judge of latency, the server the best judge of its own utilization, and stats should decay to zero over time so stale readings don’t linger.
  • Caching and perceived speed cover the other half. Facebook’s Cache Made Consistent shows why a version field alone can’t fix cache races—the version that resolves a conflict can itself be evicted before the stale write arrives. On the client side, Instagram’s predictive prefetching and optimistic UI patterns share one principle: when latency is unacceptable but access patterns are predictable, switch from pull to prediction—prefetch on hover, slide in the cart before the API responds, and accept some wasted work in exchange for eliminating latency on the hits.

This reading also fed straight into work: the ApiHistory backfill script I wrote this week used async-sema as its concurrency limiter, and I bookmarked DataLoader after reading Shopify on solving GraphQL’s N+1 problem through batching.

Learning by Writing

A second thread ran through the week: how to actually think, rather than just accumulate information. Holden Karnofsky and Henrik Karlsson converge on the same counterintuitive advice—commit to a claim early, on purpose, so you have something to break.

  • In Learning by Writing, Karnofsky forms a hypothesis “long before I’m qualified to have an opinion,” then reads to change his mind rather than to gather facts. He rarely reads a document start to finish; he skips energetically to the parts most likely to overturn his current view. Writing early forces you to pin down the version of the claim you care most about.
  • His companion piece, Minimal-Trust Investigations, is the discipline of occasionally suspending trust in others’ judgment and digging into a question until you can keep pace with experts on that narrow slice.
  • Karlsson’s How to Think in Writing supplies the mechanism: push yourself to make definite positive claims, because ambiguity lets ideas stay fluid and dodge scrutiny. Unfolding a claim into an explanation spreads it on a “wider front” so criticism has more targets. Seeing your ideas crumble is the point—“it is in the cracks the light shines in.”