Monday, October 31, 2011

Book of War: Pricing and Simulation

I'm guessing that price-balancing the different unit types took about 80% of the overall work effort in the Book of War project. Something like that. (Note: Halloween content at the bottom of this post.)

Previously I pointed out I how I felt compelled to write a simulator program for mass D&D combat at the man-to-man scale, so that we could carefully investigate what the overall trends of that kind of action would be. (Here.) But even more importantly than that, and in fact predating that by a few years, I also wrote a simulator for the full Book of War game itself. While I was working on the game, I was usually ping-ponging between book text and simulator program, to make sure that things were working out in a reasonable way. This allowed me to run lots of head-to-head battles at proposed price levels, to try and get things cost-balanced as much as possible -- like, my estimate is that approximately 2 billion simulated games of Book of War have been run to date for that purpose. Here's the simulator program, for your consideration (in Java; under GPL v.2 license):

Now, pretty much everything I could practicably include from the game is in that simulator program. It handles all the the different unit types, figure sizes, terrain, weather, morale, most special attack forms, monsters, heroes, wizards, etc. There are switches for different core assumptions and optional rules, and the ability to either look at one detailed game at a time, or assess the overall results of several thousand battles (with some statistics thrown in to make it more efficient). Some limitations: It only throws one homogenous unit type face-on at another unit type at a time; it presumes a board covered with just one terrain type; it doesn't handle wrapping around unit edges; and stuff like that. As usual, there's no GUI, so you'll need a Java development kit to tinker with it. But I think as a primary first-pass for checking unit prices it's a pretty powerful tool.

My primary strategy when price-balancing was something like this: Initial values were set as per the Internet Medieval Sourcebook (currently offline), which are also quite close to the classic D&D men-at-arms costs (Vol-3, p. 23). Simulations used a random budget for each side in the range of 50-200 points (you need a variable range, because a fixed price-point will show biased artifacts from one unit type having a disadvantageous remainder). For a particular group of units, create a cross-matrix of every unit battling every other unit type some thousands of times (in each case with starting range, terrain, and weather randomized). At the end, report the percentage of times that each unit beats every other; and add up the number of favorable matchups (i.e., cases where a unit wins over 50% of the time). I generally wanted these "win" matchups to be about half the number of opposing unit types -- but arranging exactly that was impossible, and even getting them vaguely close was pretty hard. There was a lot of tweaking a unit type by 1 or 2 points, which would change all the other matchups, finding one or two that were then unacceptable, and needing to re-tune again and again. If I ever changed an actual game rule, then the whole balancing process had to be re-done again from scratch. This was certainly the hardest part of the game-design process, but at the end of it I was pretty happy with the results.

For example, here is the output from the matchups among just our "core melee types": pikes (P), medium swords (S), and heavy cavalry (C). Percentages of 50 or more are shown when the unit on the left tends to win against the unit on top:

Assessed win percents (budget 50-200):

.. P. S. C. Wins
P. -- -- 62 1
S. 52 -- -- 1
C. -- 55 -- 1

So what this shows is that, as desired, there is no single "best" unit type in this trio. Probabilistically speaking, on a pro-rated-cost basis, cavalry tends to lose to pikes (62% of the time); pikes usually lose to swords (52%); and swords generally lose to cavalry (55%). Compare to the desired "Close Combat Trinity" graphic from the post last Jan-3 (which really laid out our game-design goals for the year, as it were; shown again to the right).

An important part of these results is that they assume the terrain and weather percentages as shown in the Book of War rules. Terrain frequency has been carefully set so as to have this balanced trinity appear in the results (for example: rolls for terrain are fixed at one-quarter of the table space in square feet, which is actually based on both core unit balancing and aesthetic considerations for how the board would best look). Cavalry does best in open terrain, and so do pikes; but swords gain an advantage fighting the others in any kind of unusual terrain. (Or in other words, dial up the rough terrain frequency and swords appear to do better against the other types more of the time.) There's also an estimated probability for opponents flanking pikes and avoid their defensive benefit (dial this assumption up and pikes do less well against any other type). Over all possible combinations of these factors, we get the win probabilities shown above.

Next, here are more assessment results when we add some missile troops to the mix: longbows (L) and horse archers (HA):

Assessed win percents (budget 50-200):

.. P. S. C. L. HA Wins
P. -- -- 63 -- 50 2
S. 53 -- -- 50 -- 2
C. -- 55 -- 70 66 3
L. 59 50 -- -- 51 3
HA 51 56 -- -- -- 2

If we analyze this under the game-theoretical principal of strategic dominance, then we are again happy, because no unit type is "dominated" by any other -- that is, there is no clear "suboptimal" choice (in fact, every type is the "best"response to some particular selection by the opponent, except for swords, which are themselves the only type that can generally beat both pikes and longbows).

And here we look at all 12 historical types in the Book of War "Basic Game" rules (4 types of infantry; 4 types of archers; and 4 types of cavalry):

Assessed win percents (budget 50-200):

.. LI MI HI P. A. L. C. HC LC MC HC HA Wins
LI -- -- -- -- -- -- -- -- -- -- -- -- 0
MI 67 -- 50 53 -- 50 -- 50 55 51 -- -- 7
HI 64 51 -- 56 -- 54 -- 51 51 -- -- -- 6
P. 57 -- -- -- -- -- -- -- 70 68 63 50 5
A. 62 57 53 62 -- -- 80 83 -- -- -- 52 7
L. 55 50 -- 58 63 -- 72 79 -- -- -- 51 7
C. 68 63 59 65 -- -- -- -- 54 52 -- 63 7
HC 57 -- -- 55 -- -- 66 -- -- -- -- 60 4
LC 52 -- 50 -- 51 62 -- 61 -- -- -- 61 6
MC 55 -- 51 -- 54 63 -- 57 51 -- -- 61 7
HC 61 55 59 -- 63 70 56 65 67 61 -- 67 10
HA 61 56 56 51 -- 50 -- -- -- -- -- -- 5

Now, in this case, some types are "dominated" (apparently always-bad choices), and it was impossible to avoid that. Here's one example: In the first row, light infantry generally lose to any other unit type; this was unavoidable, because if we reduce the cost of light infantry in the game by even 1 point, then suddenly they'll actually beat every other unit in the game (an issue with the granularity of a unit that cheap). But even with the price at which we set them, they're still potentially useful in the game as a scouting, delaying, rough-terrain force (in fact, my girlfriend has used them to excellent results several times).

Nonetheless, let's go through a process of taking out units that clearly have some other unit with better results in all cases (that is, IEDS -- iterated elimination of dominated strategies):

Assessed win percents (budget 50-200):

.. MI HI P. A. L. C. HC Wins
MI -- 50 52 -- 50 -- -- 3
HI 50 -- 56 -- 53 -- -- 3
P. -- -- -- -- -- -- 63 1
A. 56 53 62 -- -- 80 -- 4
L. 50 -- 58 63 -- 73 -- 4
C. 62 60 65 -- -- -- -- 3
HC 55 59 -- 62 70 57 -- 5

So what's been eliminated here are: Light infantry, light cavalry, medium cavalry, horse archers, and heavy crossbows -- these types are not optimal in a frontal clash, while the remaining types clearly have some kind of sticking value in that situation. But do notice that even the types we apparently eliminated under this analysis generally have high mobility, and might be useful in strategic ways outside the scope of our simulator (like splitting multiple enemy units apart, or hit-and-run tactics, or gaining the flank or rear of an enemy, etc.) . So you should consider using these results as one tool in your game strategy guide, but certainly not the only one.

Another thing to take into account: The indicated win percentages skew differently for higher or lower budget sizes. For example, I picked the base budget size of 50-200 as a guess for the nominal largest single unit size that you'd see for the basic game. (Although when pricing more expensive monsters, heroes, and wizards, I had to increase the budget amounts or else in some cases you couldn't afford even a single figure!) Here's a look at some other budget levels and the effects thereof, just for the "core +2" unit types:

Assessed win percents (budget 20-100):

.. P. S. C. L. HA Wins
P. -- 58 66 -- -- 2
S. -- -- 50 -- 56 2
C. -- 50 -- 68 66 3
L. 63 57 -- -- -- 2
HA 52 -- -- 50 -- 2

Assessed win percents (budget 50-200):

.. P. S. C. L. HA Wins
P. -- -- 65 -- -- 1
S. 52 -- -- -- 61 2
C. -- 55 -- 74 78 3
L. 62 54 -- -- 50 3
HA 51 -- -- 50 -- 2

Assessed win percents (budget 100-500):

.. P. S. C. L. HA Wins
P. -- -- 64 -- -- 1
S. 58 -- -- -- 57 2
C. -- 72 -- 67 74 3
L. 67 57 -- -- 50 3
HA 52 -- -- 51 -- 2

So some of the balancing might be off if the assumptions regarding total unit and army size are wrong, but that was unavoidable lest we mandate some fixed unit point size. In any event, the overall tactical situation, unique placement of terrain, and combination of forces within an army will present nearly endless opportunities for finding new ways of units interacting with and against each other. That said, the price-balancing simulator provides a pretty solid initial foundation to the game.

As a final thought, this simulator program is intended to stand as the "acid test" for pricing new units in the Book of War game. (In general, converting from D&D stats to BOW is absolutely trivial -- but getting the price correctly balanced is entirely a different story.) I would never want to present a table of abilities and piecemeal values to be added up; the ways in which different stats and abilities synthesize pretty much totally preclude that. (See Monte Cook's famous lesson in that regard as to the 3E new-magic-item table.) What should be done, at least as a first pass, is to run new units through this simulator and see how they match up against existing types; manual tweaking for novel abilities and playtest feedback can then follow. (Now that I think of it, with some work the simulator probably could be adjusted to take a new unit and spit out a recommended price value, granted the existing basic types as known baseline; but it couldn't be done for the initial group, which would be inherently self-referential; and that particular work will have to wait for another day.)

Here are text-file versions of the assessment results shown above:

Scary Halloween Special!: Check out the website for some free "routed" markers for your Book of War game. Put one of these down next to an enemy unit, and watch how fast they run off the board in abject terror! (Offer not applicable in states with trolls or the mindless undead.)

No comments:

Post a Comment