gamesblog

Base rates

A test catches 90% of cases and wrongly flags 9% of the rest; 1% have it. You test positive — what is the chance it is real? Count it out.

  • Probability
  • Untimed
  • 16 levels

How it plays

Each question is a test for something rare — a disease, spam, fraud, a banned item, a faulty part — with the share that have it, the share of real cases the test catches, and the share of the rest it flags by mistake. Work out what a result means.

It starts with the counts given — of 10,000, so many have it, so many are flagged — then gives only the rates, then asks about a clear result, then a case flagged twice. As you climb, the thing grows rarer — tenfold at a time, down to a thousandth — and the rates stop being round. The ladder ends there, because past that nothing gets harder. The answer is a percentage; options are well apart, and a typed answer counts within a stated margin.

The wrong options are the real slips: the hit rate given as the answer, the prevalence alone, one minus the false-alarm rate. The working always imagines a crowd and counts it. The frames are illustrative; no real test's figures are used.

Keys

Space starts and skips ahead · 1–9 pick · Enter submits · Esc stops. On a phone, tap.

What it trains

This makes reasoning from base rates in situations like these quicker and surer. It is not medical advice and claims nothing wider.

Learn

How to get better at base rates: the method, the techniques, what each new level asks of you, and the traps the wrong answers are made of.

Your approach to each question

  1. Read what is asked: the chance a flagged case is real, the chance a cleared case is real all the same, or the chance after two flags. Each counts a different pair of groups.
  2. Picture a crowd — 10,000 is the usual size here — and split it: how many have it, how many do not. The rare group is small; the other group is almost everyone.
  3. Apply the test to each group separately: the hit rate to those who have it, the false-alarm rate to those who do not. You now have four counts: real flags, false flags, missed cases, true clears.
  4. Answer with the right two counts: real flags ÷ all flags for a positive, missed cases ÷ all clears for a negative. Then check the size of your answer against the options.

Techniques and tricks

  • Count, do not multiply rates

    Turn every rate into a number of people (or emails, or bags) out of the crowd. The game always makes these counts whole.

    e.g. 1% have it, 90% caught, 9% false alarms. Of 10,000: 100 have it, 90 flagged; 9,900 do not, 891 flagged. 90 ÷ 981 = 9.2%

  • Odds times the ratio

    Write the base rate as odds — have it : not — and multiply the left side by the hit rate ÷ the false-alarm rate. That is the odds after one flag.

    e.g. 1 : 99, ratio 90 ÷ 9 = 10, so 10 : 99, which is 10 out of 109 = 9.2%

  • Rare things: the quick estimate

    When the base rate is tiny, nearly everyone is in the “do not” group, so false flags are about the false-alarm rate of the whole crowd. The answer is about (base rate × hit rate) ÷ (that + false-alarm rate).

    e.g. 0.1%, 95%, 5%: 0.095 ÷ (0.095 + 5) ≈ 1.9%

  • A second flag: square the ratio

    An independent second run with the same result multiplies the odds by the ratio again. The chance after the first flag becomes the new base rate.

    e.g. 1 : 99, ratio 10: one flag 10 : 99 (9.2%), two flags 100 : 99, about 50.3%

  • Clear results: the misses

    For a negative, the real cases left are the ones the test missed: base group × (100% − hit rate). Divide by everyone cleared, which is almost the whole crowd.

    e.g. 1%, 90%, 9% of 10,000: 10 missed among 10 + 9,009 cleared, about 0.11%

As the levels climb

  • First the counts are given and you only divide; then only the rates are given and you build the crowd yourself. Write the four counts down every time, even when it feels slow.
  • Then come the clear result and the case flagged twice. They are the same crowd with a different pair of counts: missed ÷ cleared, or twice-flagged real ÷ all twice-flagged.
  • As the ladder climbs the thing grows rarer, tenfold every few levels, down to a thousandth of where it started, and the crowd grows to keep the counts whole, into the millions. Answers shrink to fractions of a percent; read them to two significant figures.
  • A little higher the rates stop being round: any whole percentage, false alarms up to 25%. Use the quick estimate to know the answer’s size, then compute it. The ladder stops at its top, since nothing gets harder beyond it.

Traps to avoid

  • The hit rate given as the answer: 90% caught does not mean a flag is 90% likely to be real. That is the chance of a flag given a real case — the reverse of what is asked.
  • The base rate alone, as if the flag told you nothing. A flag always raises the chance; the question is by how much.
  • One minus the false-alarm rate (91% from 9%). That is how often a clean case is cleared, not what a flag means.
  • The share of the whole crowd that is both real and flagged — 90 of 10,000. Divide by everyone flagged, not by everyone.
  • After two flags, giving the one-flag answer, or squaring the hit rate. Run the crowd through the test twice and count again.

How to practise

  • Untick all but “Counted out” and “From the rates” under Kinds of question until building the crowd is automatic, then add “A clear result” and “Flagged twice”.
  • Set Answer by to “Typing the answer”. Within 1 point counts — or within a tenth of the true value when it is small — so you must really compute it, not pick the one that looks plausible.
  • Do each question both ways — the crowd count and the odds — and see that they agree. Once they always agree, the odds method alone will do.
  • Raise Options shown to see more of the classic wrong answers side by side; naming which slip each option is teaches you to stop making it.

Levels

It is untimed unless you turn the clock on. There are 16 levels with the default settings, and the counter stops at the real top. Your level is judged on your last ten answers, every ten: 90% moves you up, below 80% moves you down, which holds most people near four in five right.

All of it is in the settings beside Start — how many answers a judgement looks at and how often it recalibrates, the thresholds, the time, which questions, how many options, and whether you see each verdict at once or review them together at the judgement. You can also jump to any level.

More probability

All games →