A helpful onboarding tip measurably helps the people who read it and lowers overall completion, because it competes with the single action the screen exists for
The Tip That Worked And Cost Us — Volume I, Understanding Humans Before Products · Attention · Two paths · Foundations. A helpful onboarding tip measurably helps the people who read it and lowers overall completion, because it competes with the single action the screen exists for
01 · Opening
A helpful onboarding tip measurably helps the people who read it and lowers overall completion, because it competes with the single action the screen exists for
02 · The setup
Priya has done the work. The tip is short, it is accurate, and the people who read it do better.
Priya: Users who read the tip make fewer mistakes on the next step. I have the numbers.
03 · The setup
Among users who read the tip — Errors on next step: down 34%; Support contacts: down 12%; Read the tip: 22%; Said it was useful: 88%
04 · The evidence
Two weeks after the banner ships, the number that pays for onboarding has moved the other way.
Dev: Setup completion is down four points. Nothing else shipped that week.
05 · The evidence
Priya: But it helps. I measured it.
Maya: You did, and it does. Both of those can be true.
06 · The evidence
The screen has one job. A field, a button, and now a second thing asking to be read first.
Maya thinks: Twenty-two percent read it. What did the other seventy-eight do with the pause?
07 · The evidence
The whole screen, rather than the people who read the banner.
Setup screen, all visitors — Completion before: 61%; Completion after: 57%; Abandoned without touching the field: up 5 pts; Time to first keystroke: up 3.1s
08 · The evidence
Maya: Your measurement asked what the tip does for readers. It could not ask what it costs everyone else.
09 · The evidence
A screen does not have unlimited room for helpful things. The user arrives with a fixed amount of attention and spends it on whatever is in front of them, so a second element does not sit alongside the first — it competes with it.
This is why measuring an addition among the people who engage with it will nearly always look good. Those are the people it suited. The cost lands on the ones who paused, read nothing, and left, and they are invisible in that cut.
The question to ask of anything you add is not whether it helps. It is what it is displacing, and whether the thing being displaced was the reason the screen exists.
10 · The evidence
Sam: So delete the banner.
Maya: Move it. The error it prevents happens on the next screen, and that screen has room.
11 · You make the call
The story does not tell you first.
A. The 34 percent — it was a biased sample
B. Neither. They measure different populations
C. The four points — two weeks is too short
12 · What happened
The tip moves to the screen where the error actually happens. Completion returns to 61 percent and the error reduction survives.
Maya thinks: Same words. The cost was where we put them.
13 · Complete
Measuring what an addition does for the people it suits will nearly always look good, because the cost lands on the people who ignored it. Next: the same number rising for two opposite reasons.