Teams that use Boards stay three times longer. Maya has found the biggest number of her career.
←→ to move through the storySwipe or use the arrows to move through the story
The Feature That Caused Nothing — Volume I, Understanding Humans Before Products · Seeing Reality · Two paths · Core. Maya finds that teams using Boards retain three times better, pushes Boards onto everyone, and learns from a holdout that the feature never caused the retention — the kind of team that retains anyway is the kind that finds Boards
01 · Opening
Teams that use Boards stay three times longer. Maya has found the biggest number of her career.
02 · The setup
The retention chart splits clean down the middle. Teams that use Boards stay. Teams that never open it leave.
Maya thinks: Three times the retention. That's the biggest number I've ever found.
03 · The setup
The split is real. Nobody has miscounted anything.
90-day retention by Boards usage — Uses Boards weekly: 71%; Never opened Boards: 24%; Accounts measured: 8,940; Randomly assigned: no
04 · The evidence
Maya: So we add Boards to onboarding and retention goes up.
Sam: I can sell that number tomorrow.
05 · You make the call
The story does not tell you first.
A. 8,940 accounts is too few to trust a gap that size
B. Teams chose Boards themselves, so the gap may describe them rather than the feature
C. Ninety days is too short, retention should be measured over a year
06 · What happened
Dev: Or the teams that were always going to stay are the ones who go looking for Boards.
07 · What happened
Nobody assigned these accounts to a group. The users sorted themselves — and whatever made a team organised enough to adopt Boards in week one is the same thing that makes them still be there in week twelve.
The number is accurate and the action it suggests is unsupported. That gap is the whole lesson: correct data plus a self-selected group tells you who these people are, not what the feature did to them.
08 · What happened
Priya: Two ways forward. Ship it to everyone, or hold a slice back and find out.
09 · What happened
Who put them in the group? — If the user did, you have a description of a type of person. If you did, at random, you have a result.
Name one thing that could cause both — Team size, budget, how organised they already are. If you can name it in ten seconds, the correlation is probably it.
What would the sceptical version cost to run? — A holdout is usually cheaper than the roadmap you were about to spend on the wrong thing.
10 · What happened
They hold back a tenth of new accounts at random. No Boards in onboarding, no prompt, nothing.
Maya: Six weeks. Then we look once.
11 · What happened
Boards in onboarding tripled adoption. It moved retention by less than a point.
Holdout, six weeks — Boards adoption — treated: 63%; Boards adoption — held back: 19%; 90-day retention — treated: 38.1%; 90-day retention — held back: 37.6%
12 · What happened
Adoption moved. Retention did not. The feature was a symptom of a good account, never a cause of one.
Maya thinks: I nearly spent two quarters on this.
13 · What happened
Maya: The holdout cost us six weeks and saved us two quarters.
Priya: Keep it. Make it the habit.
14 · Complete
When users sort themselves into a group, that group describes them — it does not prove what you did to them. Next: when three angry emails should change the roadmap, and when they should not.