Notes / 3 of 3

Eighteen requests a second:
a state broadcast that became an infinite loop.

A page refetched its data because an auth event fired. The event fired again because saving a profile re-emitted it. Four lines to fix, and how the log query found it.

loner10 Nov 20266 mindebugging

A one-line log query turned into one of the better bug hunts I have had. Same IP, same second, eighteen requests: eleven calls to /quota/summary, five to /user/me, repeated until the user left the screen.

The pairing was the clue. Those two endpoints were only ever called together, from one place:

const [summary, me] = await Promise.all([fetchSummary(), fetchMe()]);

So something was calling that block in a loop, and the loop had to be synchronous enough to fire before the previous response even finished.

01The chain

Written out, the cycle is obvious. Inside the codebase it was invisible, because every link looked reasonable on its own:

onAuthChange(state => {
  if (state === 'ready') this.loadData();     // 1
})

loadData() -> fetchMe()                        // 2

fetchMe() -> setProfile(me.profile)            // 3

setProfile() -> emit()                         // 4  broadcasts state again

emit() -> listeners get state === 'ready'      // 5  back to 1

Every step is defensible. The listener wants to load data once auth is ready. fetchMe wants to cache the profile. setProfile wants to notify everyone that the profile changed. And emit wants to tell listeners the current state.

The bug is at step 5. emit broadcasts the current state, not the change. So a profile update — which has nothing to do with auth — looks identical to a fresh login.

02Why nobody noticed

Three reasons, all worth remembering:

What made it findable was looking at distribution instead of totals: which endpoints, which second, which IP. Eleven calls to the same endpoint in the same second from one device is not a user pattern.

03The fix

Two changes, four lines.

// 1. only act on the transition into 'ready'
let prev = '';
onAuthChange(state => {
  const becameReady = state === 'ready' && prev !== 'ready';
  prev = state;
  if (becameReady) this.loadData();
});

// 2. do not broadcast when nothing changed
function setProfile(next) {
  const unchanged = profile && JSON.stringify(profile) === JSON.stringify(next);
  profile = next;
  if (!unchanged) emit();
}

The first breaks the cycle. The second removes most of the noise that was causing it. Either alone would have stopped the loop; together they stop the class of bug.

An event should say what changed. If it only says "something happened", every listener has to guess — and guessing is where the loop comes from.

04What I would do differently

The fix took four lines. Finding it took a log query that grouped by path and second. That ratio is normal, and it is the argument for keeping boring, queryable access logs.

Worth keeping

  1. Broadcast on change, not on emit — otherwise a profile update looks like a login.
  2. Guard handlers with a transition check, not a value check.
  3. Look at rate per endpoint when traffic looks wrong. Totals hide loops.
PreviousTen passport specs, one tableBackwork.loner-archer.asia