A team shipping weekly steers on a metric that takes three weeks to settle, and every reasonable correction overshoots because it responds to a decision three releases old
Three Ships Behind — Volume I, Understanding Humans Before Products · Systems · Failure & recovery · Practitioner. A team shipping weekly steers on a metric that takes three weeks to settle, and every reasonable correction overshoots because it responds to a decision three releases old
01 · Opening
A team shipping weekly steers on a metric that takes three weeks to settle, and every reasonable correction overshoots because it responds to a decision three releases old
02 · The setup
Six weeks into a push on activation. The team ships every Thursday and reads the number every Monday.
Dev: We have changed direction four times in six weeks and it is getting worse.
03 · The setup
Activation rate, weekly readings — Week 1: 22%; Week 2: 31%; Week 3: 18%; Week 4: 34%; Week 5: 16%
04 · The evidence
Priya: It swings twenty points a week. Nothing we ship is that powerful.
05 · The evidence
Maya goes looking for how the number is built rather than for what it says.
Maya thinks: Activation counts a user as activated within twenty-one days. Of course it swings.
06 · The evidence
How the reading is assembled — Activation window: 21 days; Release cadence: 7 days; Releases inside one window: 3; Cohort fully settled after: 21 days
07 · The evidence
Maya: Monday's number is mostly three old releases finishing. We have been grading the wrong ship.
08 · The evidence
Each correction was sensible on the day it was made. Together they are a hand on a shower tap that runs three weeks late.
Maya thinks: Nobody made a bad call. The sequence is what is wrong.
09 · The evidence
Sam: So look at it more carefully before reacting.
Maya: We were careful every week. Careful and three weeks late is still three weeks late.
10 · The evidence
Measure the delay before the metric — How long until a change shows fully in this number, and how many releases fit inside that. Three or more and you are steering blind by construction, however good the data is.
Find a faster proxy for steering — Keep the slow metric as the truth and steer on something that settles inside one cycle — first-session completion, day-two return. It does not have to be perfect, it has to be prompt.
Or slow the hand, not the eye — If no fast proxy exists, change direction on the cadence the metric can support. Shipping weekly and deciding every three weeks is stable. Shipping weekly and deciding weekly is not.
11 · The evidence
They keep the weekly ship and move steering onto first-session completion, which settles in a day, checking activation monthly.
Dev: Same release cadence. It just stopped fighting itself.
12 · You make the call
The story does not tell you first.
A. The team is over-reacting to noise
B. The feedback arrives later than the decisions are made
C. Activation is the wrong metric for this team
13 · What happened
Activation settles at 29 percent over the next two months, with no week-to-week drama in it at all.
Maya thinks: We did not read it better. We stopped asking it a question it could not answer that fast.
14 · Complete
Delay between action and feedback destabilises a loop no matter how carefully it is read. Next: what people do when you measure them, and why nobody has to cheat for it to go wrong.