Why Did Natural 20's Become Automatic Hits?

Here's an observation I just made, that came up from a friend's question about the 1E AD&D rules for a big project they're working on. Ultimately this highlights the less-than-optimal state of editing in those books. You may know that 1E AD&D did not have a "natural 20 always hits" rule, but it did show a repeating sequence of 20's in the to-hit tables (DMG p. 74-75):

A few pages later, there's a paragraph-long note highlighting the status of those repeating 20's (p. 82):

Of course, I've read and relied on this paragraph many times over the decades. But suddenly I found myself having trouble parsing the language re: "Should any DM find that this system offends his or her sensibilities, the following modification is suggested..." Specifically, what's the other (default) option for interpretation that is being modified here?

Here's the thing: This supposed "modification" is the mechanic you get anyway, based on an earlier core rule for how modifiers are applied to attack rolls. Note the end of this section alludes to the special "20" entries on the combat tables, and how this important rule is meant to interface with that (p. 70);

Playing by the rules as written, this establishes a key factor that since the attack tables are not entirely formed by a linear progression (due to the pattern-breaking flat spot of 20's), you really need to use those tables directly, and they can't be translated into elementary arithmetic formulas. You must apply modifiers to the row of the table, and not the die-roll, which would easily let you jump over the repeated 20s (to the benefit of either side depending on direction of modifier).

But that makes the "modification" on p. 82 redundant: Once you've applied all modifiers to the row of the table (per base rule p. 70), then the entry you look up is precisely the target you need on the die. There are no more bonuses to apply, and so a "20" means nothing except that you need to see a real, actual, natural 20 on the die. (Arguably we shouldn't even see scores of 21+ in the table, because by definition they're unhittable per this rule.)

So effectively the need to apply modifiers pre-die-roll has been presented to us twice -- Once on p. 70, and again on p. 82. It's just that the p. 82 text neglects the earlier rule, incorrectly takes an apply-modifiers-to-die-roll mechanic as the default, and so delivers the same rule again as a "modification".

AD&D 2nd Edition

Again, AD&D 1st Edition does not have a "natural 20 always hits" rule; the repeated 20s go in that direction, but the rules explicitly call out that some very well-defended types should be unhittable (parenthetical on p. 70: "At some point, the upwards armor class adjustment could also make such opponents virtually invulnerable, but this is less likely and not necessarily undesirable").

That first time the "natural 20 always hits" rule appeared was in AD&D 2nd Edition (DMG p. 53, here justified with, "There are no sure things, good or bad, in the unpredictable chaos of combat situations"). Those rules don't even have attack matrices like 1E did (it's switched entirely to use of formulaic THACO as a core rule), so there aren't any repeated-20s to do that job anyway.

D&D 3rd Edition

3rd Edition D&D (and everything from 2nd Edition on) keeps the core rule of "natural 20 always hits". However, the developers seemed not entirely comfortable with this, and/or they were looking back at the 1E rules, because in the DMG there's a variant presented:

This variant suggests that we treat  natural-20 not as an automatic hit, but mathematically as if a 30 had been rolled. This does the same kind of job as the 1E rule: some things are still unhittable, but there's a big extended "flat spot" of targets vulnerable to a natural-20 roll, and only that. In this case the flat spot is extended by an extra 10 pips (that is, 11 AC values hittable only by natural 20). 

Note that we could do the same thing in 1E/2E while avoiding either the table matrix or automatic hits. If we say that a natural-20 on the die counts as 25, then mathematically we get the exact same flat spot of probability that's shown in the 1E tables (a total of 6 ACs hittable only by natural-20). 

Original and Basic D&D

It bears noting that Original D&D had no ramp-down on the tails of its attack tables; the tables there show a progression all the way down to a "1" needed to hit (i.e., automatic hit), although armor doesn't go high enough to get into the "20" vicinity. On the other hand, the Swords & Spells supplement does extend those tables, and a single step past 5% to hit, the chance becomes 0% (i.e., automatic miss). The Holmes Basic tables show the same thing as OD&D, with some hits allowed on a roll of "1".

On the other hand, the Moldvay/Cook Basic Set (1981) does assert that, "a roll of 20 will always hit, and a roll of 1 will always miss" (p. B25), this being eight years before Cook inserted the same rule into 2E AD&D.

Effect in Reality

In theory, Gygax's intuition that the extreme-cases should switch to immunity-from-attacks gradually, not abruptly, is totally correct. If we look at data for shooting accuracy, for example, it's universal that the graphs look like an "S-curve", with a flatter spot of probability at either end as it tends to either 0% or 100% accuracy. Mathematically we call these (or really the inverse, i.e., a miss-chart) sigmoid curves, and among other things they represent the integral of any probability function (i.e., a cumulative distribution function). One such chart (from Litz, "Accuracy and Precision for Long Range Shooting", Ch. 7):

Based on an analysis we've done here previously (see very end of final chart), if you assume normal-distribution variance in accuracy, and 5% steps in the middle of the curve, then we might actually think that a tail "flat spot" of around 10 steps before shots become impossible is reasonable (i.e., the 3E rule might be better than 1E).

However, it's nontrivial to find examples in classic D&D where there's any practical difference between the (realistic) flat-spot distribution tail, versus the (super simple) natural-20-always-hits rule. The most obvious one would be in a wargame/battlefield situation, with a company of normal archers shooting at some fantastic type with AC 0 or better at long range (e.g.: some devils, tougher dragons, or a Lord in +1 plate and shield). Under the 1E rule, these figures are totally invulnerable (0% to hit); whereas under the 2E rule, we can expect them to be cut down in a single round by a sufficient number of archers (say, 100 or more at 5% to hit). Think about how much of a difference that makes in your game, say, for army commanders and superheroes on the field of battle.

So for most standard D&D dungeon-crawling encounters with PCs roughly matched to the opponents they face, we don't expect to observe any effective difference between the two rule options. Approximating the real-life "S-curve" effect by saying "natural 20 always hits" is actually a pretty reasonable rule simplification. In my own OED house rules I now do that, with a caveat that "DM rules on impossible cases".