gamesblog

The reversal

A treatment can win in every group and still lose overall. Read results by group and in total, and work out which comparison to trust.

  • Probability
  • Untimed
  • Endless levels

How it plays

Two treatments (or tutors, or hospitals) and their results, group by group. Say which did better within each group, which has the higher overall rate, and what the numbers support. Sometimes the answer flips between the groups and the total — and sometimes it does not, so check.

It flips when the better one got mostly the hard cases: the overall rate is a mix of the group rates, weighted by how many cases each group had. Levels add computing the overall rate, choosing the honest conclusion, three groups, and which comparison to trust given how cases were assigned.

Every table is built so the counts are whole and the rates exact; whether it reverses is computed, not assumed. The wrong options are the real traps: trusting the total, averaging the group rates, assuming every table reverses.

Keys

Space starts and skips ahead · 1–9 pick · Enter submits · Esc stops. On a phone, tap.

What it trains

This makes spotting and explaining Simpson's reversal in grouped results quicker and surer. It claims nothing wider.

Learn

How to get better at the reversal: the method, the techniques, what each new level asks of you, and the traps the wrong answers are made of.

Your approach to each question

  1. Turn every “x of n” into a rate, group by group, and write who wins each group. In this game one side wins every group, so a split answer means you misread a row.
  2. Look at the group sizes before anything else: who got most of the hard cases? If the group winner did, a reversal overall is likely.
  3. For an overall rate, add the successes and add the cases, then divide — never average the group rates.
  4. For “what the numbers show” and “which comparison to trust”, read the last line, which says how cases were shared out: the hard ones mostly to one side, or the same mix to each.

Techniques and tricks

  • Rates from counts in your head

    The groups come in multiples of 20, so each count turns into a rate with one division: 34 of 40 is 85%, 128 of 160 is 80%, 6 of 20 is 30%.

  • The overall rate is a weighted mix

    A side’s overall rate always lies between its group rates, close to the rate of the group where it had most of its cases. Most cases in the hard group pulls it down, however well it did there.

    e.g. A: mild 19 of 20 (95%), severe 56 of 160 (35%). Overall 75 of 180 = 41.7%, near the 35%

  • Add counts, not rates

    Overall rate = all successes ÷ all cases. The plain average of the group rates is right only when the groups are the same size.

    e.g. A above: the average of 95% and 35% is 65%, but the true overall rate is 41.7%

  • Spot the lopsided mix

    A reversal needs both sides to have very different mixes of groups. If both had the same number of cases in each group, the overall result agrees with the groups — no reversal.

    e.g. Routine 48 of 60 v 54 of 60, emergency 12 of 40 v 14 of 40: same mix, B wins every group and overall

  • Trust like with like

    When the group decides both which treatment a case got and how likely it was to go well, compare within groups. Only when both got the same mix can the totals be trusted too — and then they agree.

As the levels climb

  • You start with two questions: who wins within the groups, and who has the higher overall rate. Do both every time, even when only one is asked; the contrast is the point.
  • Then the overall rate as a number, to 1 dp; then which statement the numbers support; then three groups instead of two; last, which comparison to trust given how cases were assigned.
  • With three groups, the middle one is the same size for both sides here; the lopsided mix sits in the easiest and the hardest groups, so compare the sizes in those two rows first.
  • Between each new kind, and after the last, the clock tightens by about 7% a level, down to half the starting time. Under the tight clock, read the group sizes first — they tell you whether to expect a reversal before you work out a single rate.

Traps to avoid

  • Trusting the total: calling the overall leader better when the other side won every group. It leads only because it had more of the easier cases.
  • Averaging the group rates for the overall rate. Add the counts and divide.
  • Assuming every table reverses. Some are built with the same mix for both sides, and then the total agrees with every group — check it.
  • “Compare the overall rates” as the comparison to trust. When cases were chosen, the totals mix the treatment with the difficulty; when they were assigned alike, they agree with the groups anyway.
  • “Neither tells you anything.” The within-group comparison does: like is compared with like.

How to practise

  • Use Kinds of question to drill “The overall rate” or “Which comparison to trust” alone until they are quick, then switch the rest back on.
  • Set Answer by to “Typing the answer” for the overall-rate questions: within 0.05 points counts, so you must add the counts exactly. The other kinds stay as choices.
  • For every table, say one sentence before answering: “B won both groups but got most of the hard cases.” If you can say it, you will not fall for the total.
  • After a miss, read the working: it gives every group’s rates and both totals, and says whether the mix differed. Find the group sizes that did it.

Levels

It is untimed unless you turn the clock on. There is no last level: at least one dial turns without end. Your level is judged on your last ten answers, every ten: 90% moves you up, below 80% moves you down, which holds most people near four in five right.

All of it is in the settings beside Start — how many answers a judgement looks at and how often it recalibrates, the thresholds, the time, which questions, how many options, and whether you see each verdict at once or review them together at the judgement. You can also jump to any level.

More probability

All games →