Thursday, June 5, 2014

Arena – Man vs. Man


Battling Random D&D Fighters in a Simulated Arena, So As to Assess Average Advancement and Ability Scores at High Levels

Problem

It's always been clear that higher-level characters in D&D should have higher ability scores; thinking statistically, seeing an NPC at higher level provides evidence that they likely have better-than-average ability scores that allowed them to survive and gain levels. But how much higher? This has long been hand-waved in D&D, with the publishers and players simply making something up on the fly that felt right, with little consistency. Obviously none of us has ever archived the play history of enough PCs to assess what their ability scores were, on average, at a given level.

Method

Let's simulate characters battling and gaining levels in a computer program; the one I've written in Java is called "Arena", broadly modeling gladiators battling to the death in a Roman-style Colosseum. The model runs like this: First, generate 10,000 1st-level basic D&D fighters with random ability scores (3d6 in order; bonuses OED-style +1 for 13-15, +2 for 16-18). Equip everyone with normal chain, shield, sword, and helmet. Pair them up randomly and fight (all fights are to the death at 0 hp). Award experience pro-rated for standard D&D treasure awards (for example: at 1st level, multiply base experience by 20, because about 95% of XP at the level generally comes from treasure; link); level-up if appropriate. Replace the dead part of the population with 5,000 new random 1st-level fighters and repeat.

Note that this is ever-so-vaguely scaled to the overall Roman gladiatorial system. Historical analyses suggest that a professional given fighter might battle about 3-4 times a year (i.e., once per season; certainly healed up between fights), for a fairly high monetary reward, and that about 8,000 deaths occurred in arenas, per year, throughout the Roman Empire (link). My guess is that about 100 "cycles" of this action (one combat quarterly) would be approximately equivalent to the maximum gladiator career of about 25 years. Yet for broader data I've let the simulator run for 1,000 cycles below and not bothered to "age out" anyone in the system. Obviously, D&D characters engage in far more combats than anyone in real life.

Results

Here are the average results after a run of 1,000 cycles of randomized, paired combat:


What you can see here is that the vast majority (96.6%) of the gladiatorial population will remain at 1st level over any amount of time. The largest single bump in ability scores is actually between 1st and 2nd level; basically, a starting character needs a special "something" (like high ability scores) to survive to 2nd level, and thereafter can mostly "cruise" by consuming more and more 1st level characters that they're paired against.

The ability scores that make the biggest difference for these fighters seem to be (in order) Strength, Constitution, and Dexterity. Strength average is 14 at 2nd level, and then around 15 for levels 3+; Constitution average is 13 at 2nd level, and higher by one or two points thereafter; Dexterity average is around 12-13 for any level above 1st. Since the combat here is entirely melee (sword vs. shield; no ranged combat), the Strength ability is highlighted. And since the Intelligence, Wisdom, and Charisma scores have no effect in this action, they remain around the natural 10.5 average at any level.

But another thing: The characters don't just benefit from high ability scores, they also benefit from higher than expected rolls for hit points. For example: You might expect that a 2nd level fighter with 13 Con would have around 11 hit points (2(d8+1) ~ 2(5.5) = 11); but in fact the average is 13 points (i.e., about 20% higher). At 8th level the naive expectation would be 44 hit points (8(d8+1) ~ 8(5.5) = 44), but the average seen above is actually 56 hit points (almost 30% higher). Recall that 3E D&D, for example, actually did stipulate that NPC's at advanced levels had the naive expectation for hit points, which is not actually correct according to this assessment.

Another thing we might look at is the demographic curve at advanced levels and regress to some predictive formula; here it is below (x-axis is level 2+; y-axis is in number per 10,000 gladiator population):


Note that the preceding was based on purely random pairwise combats between all 10,000 members of the population. An alternative that I tried was to first sort the population by level and have the 1st person fight the 2nd person, etc. (thus recreating a "championship" bout, where the two best characters pair off, and a bit more like D&D adventuring where it's expected that PCs fight monsters of about the same level). But the result of that was quite comical; one very strong character would initially graduate from 1st level, and thereafter advance by perpetually massacring everyone else who ever managed to pop into 2nd level. After 1,000 cycles this usually results in something like 9,990 fighters at 1st level; 9 fighters at 2nd level; and 1 fighter at 8th level (with scores like Str 16, Dex 15, Con 17, hp 62).

Conclusions

These results begin to suggest what we might do for reasonable and consistent ability scores at higher levels. As a base we roll 3d6 for each ability score, and then something like:
  • 2nd level: Prime ability +3, secondary +2, tertiary +1 (total +6).
  • 3rd-4th level: Prime ability +4, secondary +2, tertiary +1 (total +7).
  • 5th-7th level: Prime ability +4, secondary +3, tertiary +2 (total +9).
  • 8th+ level: Prime ability +4, secondary +4, tertiary +3 (total +11).
(Edit: Or actually, don't  do that, because adding a flat bonus like 3d6+4 gives a range of 7 to 22, and no high-level fighter is going to have values in either the lower or upper end of that interval; the variance is all off. Instead do something like roll 3d6 for the good abilities and apply a minimum of 4 on any single die, or re-roll values below that.)

Also, at level 2+ we should boost hit points by +20% to +30% over the naive mathematical expectation, as mentioned above. Or use the somewhat common mechanic: re-roll any "1"'s or "2"'s on hit dice. (Edit: Or apply a minimum of at least half any die roll, e.g., minimum 5 on any fighter's d8.)

Personally, my preference is not to give any special modifiers to PCs at 1st level (in ability scores, bonus or minimum hit points, etc.). But if we want PCs to automatically be "special", then I'd be very willing to start them at level 2+, assume to some extent that they've already proven themselves exceptional, and grant the aforementioned bonuses as representative (and helpful to the players). 

Here's some ways we might expand on this work. Obviously, the current code only pairs off fighters-vs-fighters, and only those relevant ability scores are selected for advancement. I wouldn't hope to simulate wizard-vs.-wizard battles any time soon. But we might run a grab-and-dash simulation for a Thieves' Guild, where an initial skill roll (pick pockets or open locks), if failed, devolves to a melee fight or flight. Or we could switch from an arena of all NPC fighters to fighters-vs-monsters according to the OD&D wandering monster table and see how that goes (monsters without special abilities, at least initially). Or I suppose we could go more "realistic" and find a way to award XP in our arena without death to the opponent in every case, such that more people could survive to higher levels.

What do you think? Any other observations or suggestions?

Appendix


Here's the code to the Arena simulator (Java ZIP file).


43 comments:

  1. This is awesome. The runaway leader (if you go with a non-random simulation) is comical, but I wonder if it isn't more "realistic" for a fixed population at a given moment in time. An interesting thing to add might be to simulate multiple populations -- I don't really have an intuition for how many. The basic idea would be that within a population an individual tries to combat up, but across populations only the highest levels combat. This would ensure some turnover at the top levels, allowing some of those lower levels to continue moving up. ...Not sure how you'd deal with populations being out of "demographic step."

    ReplyDelete
    Replies
    1. Thank you! Interesting idea about the multiple populations, I suppose that could represent separate groups of men, orcs, goblins, etc. evolving and then coming into conflict...

      Delete
    2. This is fascinating! I would be interested to seeing the results with the attribute bonuses of the D&D Rules Cyclopedia. Using your D&D house rules; you would only need to allow the fighter a feat at 1st level. That way the fighter's ability to survive would be increased but no other alterations to your rule set would need to be made.

      Delete
    3. Good question. You're basically just talking about the +3 bonus for at 18 score, right? (My house rules agree everywhere else.) I just tweaked the code and ran under that assumption...

      Not much difference. The level counts and hit points are basically the same. The physical ability scores appear to be selected by about 1 point higher on average at the uppermost levels (Level 6: Str 16, Dex 13, Con 14; Level 7: Str 16, Dex 14, Con 15; Level 8: Str 17, Dex 14, Con 16).

      Delete
    4. Specifically, here's a run with B/X ability modifiers (+3 at Str 18, small differences from what I said immediately above):

      1 9623 5 11 10 10 11 11 11
      2 124 13 14 11 10 12 12 10
      3 71 19 15 10 10 13 12 11
      4 51 27 14 10 10 13 13 10
      5 49 34 15 11 11 13 14 10
      6 41 40 15 9 10 13 15 11
      7 20 48 16 11 11 13 15 10
      8 21 57 17 10 11 11 15 10

      Delete
  2. How much effect would having ratcheted hit point rerolls (ie, at each level all hit dice are rerolled, and the larger of the new result and the old result is kept) have to work towards the above-average survivor hit points?

    ReplyDelete
    Replies
    1. Hmmm, maybe I should write a simulation for that and see the results. Question when using that mechanic: Do you go through the whole re-rolling regime for say, an 8th level NPC? (i.e., you'd have to commit to that for it to possibly generate NPCs like we're seeing here.)

      Delete
  3. Heh. You did this just after I finished up my "fantasy demographics" spreadsheet which gives approximations of numbers and levels for the various classes in my AD&D 1E+ game. However, I simply made approximations for levels above 1. I'll have to analyze these results and apply them. I wonder if they'd be different with different XP charts - would Thieves have higher levels? Would Magic-Users die out at lower levels? It's probably harder to test M-Us in this way, though.

    ReplyDelete
    Replies
    1. Cool, that's exactly the kind of work that I was hoping this might be able to inform. Of course it's highly dependent on what assumptions we use for how "dangerous", combative, and rewarding a given career path is. I agree that making a simulation for magic-users is pretty much infeasible, given the time it would take, and also how much more dependent they are in D&D at being protected by the rest of the party at low levels.

      Delete
    2. My first pass through it seems to indicate that you have to make characters with classes more common than the DMG's 1 in 100. The spreadsheet showed that, in a world of 250 million people (one of the best guesses about the late Iron Age/early Medieval era), the highest level M-U (using certain assumptions and modeling it on the progression above) would be around 12th. Worse, Druids and Monks don't even make it properly to the fixed levels.

      Delete
    3. Eek. Even making classed characters 1 in 10, there aren't enough Druids in 250,000,000 people to work well (5 at 11th level, compared to the fixed 9 at 12th). I think that I may have to assume that non-Fighters (and probably non-Thieves) survive to 2nd level more frequently.

      Delete
    4. Yeah, note that I'm operating under OD&D rules where there aren't any recognized 0-levels. Or in other words, a non-fighter is mechanically equivalent to a 1st-level fighter (my big bulge in the population above).

      It may be interesting to look at the values I got when I did a similar project in the past (link; spreadsheet at bottom of page)? Not synchronized with this arena simulation in any way.

      Delete
    5. Ahahaha! If I'm not mistaken, that spreadsheet was my starting point for this one! I had simply forgotten where I'd copied the initial cells (I have since modified them considerably). Go figure, eh? One thing I do is assume that the number represents the quantity at that level or higher, so to get the total at a given level take the number listed and subtract the number who make it to the next higher level.

      What I've done in the last little bit is revise my assumptions: characters don't exist in an arena, so I've altered the chances upward (doubled Fighters and related subclasses, trebled Clerics, and quadrupled everyone else) of getting to second level. Then I looked at the XP charts and modified the rates for various classes based on how they are varied from the Fighter chart. Those are rough estimates, obviously, but the numbers end up looking more like a (A)D&D game should. It ends up meaning that, out of 250 million worldwide population, about 1.4 million will be Fighters, and of those only about 100k have reached 2nd level or higher (again, worldwide). There are about 1000 Monks worldwide, of whom only about 73 have made it to level 2 or greater.

      Delete
    6. Oh, that's funny and great. Thanks for working more on that issue.

      Delete
  4. As we tinker with a generation mechanic to replicate this, one thing that bothers me about a possible re-rolling hit points method is that might turn into an aggravatingly long series of re-rolls. So perhaps to avoid that we could apply a minimum of the average on any die (e.g., for fighters treat any 1-4 as 5; that gives about the right inflated average, and is coincidentally the same as the method in 1E Unearthed Arcana p. 74).

    Likewise we could maybe do this for ability scores. Setting a minimum of 4 on any ability d6 gives an average score of 13.5 (about the same as the good abilities in levels 2-6 above). Setting a minimum of 5 gives an average score of 15.5 (about the same as levels 7+ above).

    ReplyDelete
  5. I use a different method to generate NPCs
    ...
    Roll 2d8 to randomly determine a NPCs hit dice:

    2- 9 = zero hit dice, Non adventuring occupation*
    10 = Ordinary militia, guard, sailor or mercenary
    11 = 1 hd, class determined by high ability score
    12 = 2 hd, non-human belongs to preferred class
    13 = 3 hd, class determined by high ability score
    14 = 4 hd, non-human belongs to preferred class
    15 = 5 hd, class determined by high ability score
    16 = d6 +4 hit dice, class determined by DM _

    To generate ability scores for NPCs, roll in order:
    2d6+3 for ‘0’ level NPCs,
    3d6 for 1 to 3 hit dice,
    4d6 -L for 4 to 7 hit dice.

    *In a quasi-medieval society there are numerous non-adventuring occupations or trades available:
    . .. .
    STRENGTH based = fighter, as well as
    armorer, blacksmith, butcher, carpenter,
    cook, laborer, porter , stone carver.
    . ..
    DEXTERITY based = thief, as well as
    acrobat, baker, jeweler, peddler,
    rat catcher, sailor, shoe maker, weaver.

    CONSTITUTION based = barbarian, as well as
    bowyer, brick layer, dyer, farmer
    furrier, gardener, messenger, miner.

    INTELLIGENCE based =mage, as well as
    architect, cartographer, engineer, navigator, physician, scribe, shipwright, -smith.

    WISDOM based = priest, as well as
    book binder, brewer, engraver, fisherman,
    inn keeper, potter, tanner, teamster.

    CHARISMA based =sorcerer, as well as
    artist, barrister, fortune teller, herald, interpreter, merchant, minstrel, money lender.

    There are numerous sources and web sites that will help you gain an understanding and appreciation of medieval culture including, but not limited to:

    Medievalist.net
    hkcarms.tripod.com/occ.html
    The Middle Ages by Prof. Phillip Daileader
    www.luminarium.org/medlit/medprice.html
    The Fall of the Pagans by Prof. Kenneth Hart

    ReplyDelete
  6. For ‘0’ and 1st level characters
    minimum hit point total equals 1/3rd their strength adjusted by one’s constitution modifier.

    ReplyDelete
  7. Awesome post, congrats! Awesome comments, too: The tie-in to demographics is fascinating.

    ReplyDelete
  8. @ Delta: Quite interesting. Do you include prime requisite bonus for XP gain?

    ReplyDelete
    Replies
    1. No, that I snipped out (as per my house rules). Maybe I'll go make a switch for that as an option...

      Delete
    2. Interestingly, by the book in OD&D, this would then give some selection pressure to Intelligence and Wisdom (because of the rule that they add fractionally to the fighters' prime requisite; Vol-1, p. 10).

      Delete
    3. Okay, so this brings up an interesting observation. Technically by the OD&D Vol-1 rules, practically everyone gets the +10% XP bonus, because the modified prime requisite for even the average character is over 15. (Say fighter with Str 10, Int 10, Wis 10: effective prime requisite is 10+10/2+10/3 = 10+5+3 = 18!) And therefore this doesn't make any difference at all in the results, although you tend to get one NPC who managed to reach level 9.

      And then you're kind of debating exactly which XP modifiers to use or what effects on prime requisite other stuff has (OD&D, Sup-I, Holmes or B/X, or AD&D). Kind of the reason I like to slice it out of my game.

      Delete
    4. Here's a run with XP modifiers from raw prime requisite only (Strength not modified by Int or Wis): this seems to put a small upward pressure on Strength (+1 point?), and consequently a downward pressure on Constitution (-1 point?).

      1 9661 5 11 11 10 11 11 10
      2 99 13 14 10 10 12 12 10
      3 67 20 15 10 11 12 13 11
      4 50 26 15 10 11 13 13 11
      5 41 33 15 10 10 13 13 10
      6 44 40 16 11 10 13 14 10
      7 17 45 16 11 10 14 14 11
      8 19 55 16 11 11 13 15 10
      9 2 58 17 9 10 14 13 11

      Delete
  9. In Holmes you only counted int and wis over 9 (or rather you could only lower them to 9 when raising str), not sure the LLB had that limit or not.

    ReplyDelete
    Replies
    1. The LBBs did not (somewhat to my surprise). The first place that seems to appear is the Greyhawk Supplement I, in relation to thieves' Dexterity, but it fails to actually say whether it should be back-applied to the original classes.

      "Thieves use dexterity in the pursuit of their chosen profession. They may use 2 points of intelligence and 1 point of wisdom to increase their raw dexterity score so long as they do not thereby bring the intelligence and wisdom scores below average." (Sup-I, p. 8)

      Delete
    2. Take a look at the note at the bottom of page 11 of M&M.

      "Note: Average scores are 9-12. Units so indicated above may be used to increase prime requisite total insofar as this does not bring that category below average, i.e. below a score of 9."

      It's written in Old High Gygaxian, so its meaning might not have been clear, but Holmes was able to translate it.

      Delete
    3. Great catch, thank you for that! (Of course, the result of that applied literally is that almost everyone gets an 18 prime requisite.)

      Delete
  10. Can you tell whether hit points or ability scores are more important to survival?

    ReplyDelete
    Replies
    1. Hmmmmm... I don't what metric we'd use to assess that (I'd be open to suggestions).

      Broadly it seems in this model that there's a big boost in abilities going from 1st to 2nd, and fairly minor increases afterward; but hit points continue to be rolled above expectations as level increases.

      Delete
  11. Also, the numbers of people that you give, are they all alive after the cycles or do those numbers include both dead and alive fighters? Can you get a demographic spread from this?

    ReplyDelete
    Replies
    1. The numbers are for surviving fighters only. Each cycle runs the paired fights, clears out all the dead, and then hires 5K new Ftr1's. The statistics are computed after all of this is complete.

      Delete
    2. So a demographic spread for 1000 fighters is about:
      1st: 966 HP: 5 St: 11 De: 11 Co: 11
      2nd: 11 HP: 13 St: 14 De: 12 Co: 13
      3rd: 6 HP: 19 St: 15 De: 13 Co: 13
      4th: 5 HP: 25 St: 15 De: 13 Co: 13
      5th: 5 HP: 34 St: 15 De: 13 Co: 14
      6th: 3 HP: 40 St: 15 De: 14 Co: 14
      7th: 2 HP: 48 St: 16 De: 13 Co: 15
      8th: 2 HP: 56 St: 15 De: 12 Co: 15

      Cool.

      Delete
  12. A little late to the party, but I broke out a spreadsheet and made the assumption that those guys who did not advance to the next level would be the lowest in ability scores. Thus, I'm setting a minimum, and effectively re-rolling until a score hits that minimum.

    If we set the minimum ability score to 3, the average is 10.5. You all knew that.
    If we set the minimum ability score to 4, the average is 10.53.
    Minimum 5 = Average 10.63
    Minimum 6 = Average 10.79
    Minimum 7 = Average 11.04
    Minimum 8 = Average 11.37
    Minimum 9 = Average 11.81
    Minimum 10 = Average 12.33
    Minimum 11 = Average 12.92
    Minimum 12 = Average 13.56
    Minimum 13 = Average 14.25
    Minimum 14 = Average 15.00
    Minimum 15 = Average 15.75
    Minimum 16 = Average 16.50
    Minimum 17 = Average 17.25
    Minimum 18 = Average 18 (duh)

    So, for a 2nd level fighter, set his minimum Strength to 12, his minimum Dexterity to 9, and his minimum Constitution to 11.
    For a 3rd level fighter, set his minimum Strength to 14 and his minimum Dexterity to 11.
    For a 5th level fighter, set his minimum Constitution to 12.
    For a 7th level fighter, set his minimum Strength to 15 and his minimum Constitution to 13.

    ReplyDelete
    Replies
    1. Oh, very nice! That would an excellent use for my post on quick rolling NPCs.

      Delete
  13. What happens when you factor in say 1 in 20
    (or more frequent) combats being against a foe with an attack that requires targets make a save vs disability or death? D&D fighters don't just fight other fighters they also face icky monsters with poison and paralysis attacks.

    ReplyDelete
    Replies
    1. Similarly, just factoring in the fact that many or most combats won't be against other fighters in the pool will likely have a significant effect on the results.

      Delete
    2. Yeah, that's a question I'd also like answered that requires further research (programming). Ideally I'd like to have the whole OD&D encounter table list in play, but this was my first stab to see if others also found it interesting. I'll try to find time for that kind of expansion.

      Delete
  14. It looks like they need 8+ hp total to survive to 2nd level. That should explain the stats and HP of 2nd level characters. Also, max hp at 1st level as a common house rule, and official 3e rule.

    They also need about 11+ at 2nd level to make 3rd (usually surviving two hits from 1st level enemies). Looks like it from there you either roll slightly high hp, or have 16-18 Con and can roll average, or have good Dex and Str to finish fights quicker.

    Ah, the average HP beats the average damage output from a number of hits of your level, which peaks quickly to safely eliminate low-hp opponents and greatly improve your odds at 1st level.

    Interesting that a bit of Con and Dex can get you to 6th, but only big Con (and fair hit die rolls) survives the test of time beyond that, where you'll have paired off against ever-larger numbers of high-level opponents.

    ReplyDelete
    Replies
    1. Interesting observations. I have to admit I was surprised at the Con being so important; I would have guessed Dex (not being hit in the first place) would get priority.

      You got me curious and I did a run and printed out results whenever someone graduated out of 1st level. Count 472, hit points min 1, max 10, mean 7.3, stdev 1.7. So the average hit points for success is closer to 7, and some people manage to do it even with just 1 hit point.

      Also, we can back out the Con bonus and see the raw rolls for 1st level hp: mean 6.9, stdev 1.5. So there's a lot of those initial d8's that came up 7 or 8, but lots of people are succeeding with lower rolls, including 1.

      Personally, I wouldn't ever want to give PCs max hit points at 1st level; I'm of the school that 1st level is specifically for taking what you roll and having a tough challenge to survive. If that's not the desired game, I support starting at level 2+ and giving a boost to reflect the reason for survival that we see here.

      Also note that at any level it's very rare to face higher-level opponents in this arena; 96.6% of the time you'll always draw a 1st-level opponent from the broader population.

      Delete
    2. Thanks, the 1hp Fighter just got lucky, there's over half a million such folk will have started out after all.

      Oh, I realised I looked at it through goggles of survivor bias. One needs to consider the number fights each level character has had, what size pool they're from, and who their opponents have been at the time they faced them.

      First few rounds, you couldn't fight a 2nd level character, there weren't any. Now 1st level characters face someone of 2nd level or higher 1/3 of the time before hitting 2nd level themselves and very likely die right there.

      The first time someone got to 7th level, in 400 fights or so, they'd almost always fought 1st level characters like now, but they'd never fought higher level people. Lately 6th level characters have to face a few opponents of 7th and 8th level, which means there's less of them and they've got better stats.

      Surviving the first 200 rounds was fairly easy once you survived your first 12 and 25. Surviving the last 200 rounds to be a 5th level Fighter has involved being lucky about who you fight (expect 7 fights against leveled characters, 2 against 5th+).

      Snapshots of numbers after 100, 200, 400 fights would be interesting. I suspect it's culling the weak at 5th-6th level better than it used to, and if you started retirements now the high level folk would end up with similar stats, forcing even harsher restrictions on the mid levels.

      Delete
    3. Hmmm, you might be on to something there, in fact. Good observations, thanks!

      Delete