Monday, February 16, 2015

Archery – Field Experiment

My current house-rules for archery are based on a combined mathematical model and game simulation program (link). This is the fruit of quite a bit of analysis on the archery game (search the blog for "archery", you'll find lots of posts). The most important observation is this: shooting man-to-man and shooting at an army are totally different tasks (the former may be impossible to hit, while the latter impossible to miss, at the same range).

This is based on some pretty good data in Dragon #58, originally from the book Archery, on hit rates for Grand Masters at different ranges (think: top-level fighters with several bonuses). Not having specific hit data, my model for beginning bowmen (1st level fighters?) was very roughly estimated by recent GNAS scoring, and guessing they're maybe one-tenth as accurate as Grand Masters. The table below recaps the expected hit rates from both the game rule model and the bivariate normal physics simulation. But were those reasonable assumptions?



Obviously, the only way to approach this scientifically is to run a test in the field (like, an actual field). At the end of August I visited my folks' place in Maine and got out my very old bow and arrow kit and set up a target to see whether my own accuracy broadly matched this math model. The equipment is a 30-year-old Bear compound bow with a 28" draw at 60 pounds (with no maintenance ever having been done in those 30 years; in fact, it spent several winters in an outdoor shed), shooting 32" target arrows. I made a 2-foot radius target (to match the old GNAS competition), and took a series of 10 shots each from 10, 20, and 40 yards distance (to match the increments in the prior D&D model). Due to time constraints, I didn't take any practice or warmup, and I haven't done any shooting in at least several years (and I've never done it with this target size or range).

Now technically the first thing I did was set up the target and several bales of hay in back of a metal trailer and shoot one flight of arrows that I had to start with. Each of these shots went entirely through the target, hay, into the side of the trailer, and entirely shattered from head to tail. So I didn't count those, and had to go down to the Kittery Trading Post to get another batch of arrows. Later that afternoon, I was back with this setup:



Here at the two flights of 5 arrows each, shot from 10 yards. I easily hit all ten times, although several of the arrows flew entirely through the target. Notice that even on the second set of 5 my accuracy was noticeably improved, grouping the arrows closer to the center of the target.




Here are the  shots from 20 yards. Again I hit the target all 10 times -- although more of the arrows are disappearing through the target. I think I started getting a fuller draw at this point, because on the second set of 5 every one of them flew entirely through the target.




At this point I started shooting from 40 yards away (from way down in the field, actually). Here I only got 2 arrows out of 10 to actually go in the target. Generally I was aligned correctly, but my shots were mostly falling low/short, although I'm sure that would improve if I got more practice and got the ascension right.




So, an admittedly small sample size, with very old equipment and an unpracticed shooter, but that's all I could accomplish on the particular day. Let's compare the results to our prior model:

That's not a perfect match, but the numbers do seem to moving in generally the same direction. Based on my experience that one day, I really couldn't miss an immobile target of that size from 10 or 20 yards distance. From 40 yards I was hitting a bit less than predicted for a "3rd class bowman", but I'm pretty sure with a little more practice at that range I could start doing much better than that, likely above 30%. (As a comparison, the next day I spent most of the afternoon shooting from 30 yards, and I felt like I nearly couldn't miss once I had the range down; over about 100 shots I missed only 2 or 3 times, for a 97-98% hit rate. The house rule game model would predict a 55% success rate, and the physics-model simulator about 48%.)

Generally it seems like with a little practice, my hit rates are better than predicted, which could be due to a number of factors: (a) I underestimated the skill of GNAS 3rd-class bowmen, (b) I have better equipment than that used for the base data (the Archery book was published in 1894), (c) I'm a better archer than expected -- seemingly the least-likely hypothesis.

Coincidentally, a neighbor's son who is entering high school as freshman came over the next day with his own archery equipment, and it was kind of stunning to see how the equipment and style has all changed completely in the 30 years since I got my own bow. His bow is much lighter and smaller (half the draw weight) with a bunch of built-in sights and range-finders and whatnot. Shooting is done with a bent left arm (whereas I need a big leather guard to prevent injury) and a loose grip, letting the bow fall out of the hand loosely on a cord after the shot (whereas my heavier bow would likely break my wrist if I tried that). Plus a trigger-button release is used, whereas I mostly chewed a hole through a leather glove and my fingers over the afternoon. He took a few shots and obviously had much greater accuracy than I was getting, which speaks to the rapidity of how much the discipline can change due to technology in a fairly short time.

Anyway: What we clearly see in the experiment is that hit rates may be 100% at close ranges, and then very rapidly drop off to near-zero with just a few doublings of range. Even if I could practice more at the 40-yard range, I'm sure that shooting from 80 yards out at a single man-size target would be practically impossible for me. But the flip side is that it would only take a few doublings in target size (4 or 8 men deep or wide) and I'd be back to automatically hitting on almost every shot. So in broad strokes the math does seem to be winning, and generally predict the overall dropoff in accuracy from shooting at a small target at distance.

Got any more data?


23 comments:

  1. If you haven't seen this yet, you should check out: https://www.youtube.com/watch?x-yt-cl=84503534&v=BEG-ly9tQGk&x-yt-ts=1421914688

    He's definitely a high level archer and he doesn't use modern style mechanics in his shooting. It's pretty amazing.

    ReplyDelete
    Replies
    1. I did see that, and I honestly love it and it's amazing. I also saw some pretty believable criticism (see "skepticallypwnd"), like to these tricks he's skipping a full draw/impact and prepping the arrows in special ways, etc. Still: super impressive.

      Delete
  2. I was going to mention Lars Anderson as one of my two things also. The other is that I went shooting in October at 25 30 40 and 50 yards. I was using a 60# wooden competition recurve with heavy aluminum arrows (wrong weight of arrow). With these arrows and shooting high I could barely reach 50 without overshooting by a great distance - also I was on a slope facing down hill. At 25 and 30 I had no problem. 40 I was about 60% iir. But this was four months ago and I didn't write anything down... So I could be way off

    ReplyDelete
    Replies
    1. Thanks, that seems pretty compatible!

      Delete
  3. I love this sort of stuff- equating real-life to fantasy game statistics. Two take-aways from your experiences I would have would be the almost automatic hit at close range (i.e. the sort of range you would normally see in, say, a dungeon room) and the need to analyze for a moving target instead of a stationary one. Maybe another reason to take a thief in a party- if he can move silently, hide successfully, and get into that all important close range, an auto-kill using a bow would make sense. Cool stuff.

    ReplyDelete
    Replies
    1. Right, I agree. I've totally removed the normal penalties for shooting into groups at close range for exactly those kinds of observations. Which is nice, because it does give the players (incl. thieves) more tactical options.

      Delete
  4. How high are your ceilings? An inability to "arch" your shot will limit indoor/subterranean range.

    ReplyDelete
  5. I've been thinking about ranged attack accuracy myself recently, and I think it's important to realize that accuracy under firing range conditions bears very little resemblance to accuracy under combat conditions. I know there are lots of studies of e.g. police incidents that indicate that the miss rate for firearms at a range of 10'-20' is staggeringly high. There is a HUGE difference between shooting at a target in a relatively calm environment, and shooting at a living being in the midst of combat. And if you're far enough away from the enemy that you can take your time and set up your shot calmly, the time of flight may be such that accuracy suffers significantly when shooting at an individual target. Unfortunately I'm not aware of any data for combat archery accuracy...

    ReplyDelete
    Replies
    1. It's a real problem for assessment. The best analogy I've come up with is documentation from SCA fights where combat archery is allowed, and I have to say that I was surprised that it seems a lot more accurate than I expected. This is among the reasons that my house rules got tweaked to be a lot more permissive about mid-melee shooting (although harsher about range penalties): http://deltasdnd.blogspot.com/2014/04/a-model-of-archery-for-d.html

      Delete
  6. Many years ago I could hit a rabbit or squirrel on the run at about 20 yards with a selfbow, recen tly I had chance to use my brother-in-law's compound bow and out of 5 shots I lost 2 arrows and only 2 struck the target true. No archery for 25 years has me very rusty.

    ReplyDelete
  7. I don't see how this relates to D&D since I don't see how you could qualify your ability and your target was inanimate.

    The target in Battle as opposed to a select animal or individual is a crucial distinction and flaw with Bow fire in D&D. Ive said before that I'll happily stand 70 yards from someone while they try to hit me with an arrow.

    The most important distinction (and it's never mentioned) is whether the target is aware he is being fired at or not. My rule is that you can only fire at a target who is aware of you at short range.

    In battle this again become unimportant because the target can never know from where he is being fired upon so the bow again becomes very powerful.

    ReplyDelete
    Replies
    1. I think you mean "you can only fire at a target who is unaware of you at short range"? I can kind of sympathize with that, because I've also seen it written how important that is in reality. But adjudicating that awareness in a D&D melee seems like a real hassle -- I'd prefer it just be subsumed by the whole surprise/AC/Dex core mechanic.

      Other than that, if you look at "The Model" (link), you'll see that those other concerns are accounted for, in that the in-game D&D archer is being given a total of about +12 bonuses to hit here for the static-target case. Accounting for battle conditions simply means not giving them that bonus, and then you're back to standard D&D with about 50/50 to hit at close range. The real point here is "what should be done about range modifiers?", and the results seem consistent: about -8 per doubling of range.

      Delete
    2. --I think you mean "you can only fire at a target who is unaware of you at short range"?--

      No, the point of my rule is that at short range the speed of an arrow is sufficient to make the target's awareness irrelevant. At medium and long ranges I have decided it is impossible to hit an aware target.

      By 'aware target' I do not mean an animal moving in a straight line (which could be hit at long distance). I mean an intelligent target who is watching the bowman.

      Delete
    3. Oh I got you; that's reasonable, thanks for the clarification. My own house rule currently on the books is that you just can't shoot at an individual over 40 yards (you might say I'm assuming awareness and non-helplessness), although you can shoot at a random member of a large group.

      Delete
    4. I would say that a target displaying Kent's "awareness" is actually performing a parry maneuver: focused on the shooter and the shot, devoting movement to dodging that attacker. Any attacks against him in melee would distract him from his dodging of the shot, so he needs to pick which attacker he's going to defend against. If your game has a simple "full defense" option, maybe he's scampering around generally making himself hard to hit. In either case, because he's devoting so much attention to defense, he must have lower or no attack capability that round (if he can declare defensiveness and also get normal attacks, why would any character ever not do that? And so we must assume that kind of mixed defense/offense middle stance must be subsumed in the regular attack roll vs. AC).

      Then is the target is actually able to see the arrow coming, and where is he going to dodge? This would be better modeled by the shot having random variance in where it lands based on the attack roll, and the defender simply choosing where he will go knowing only that the attacker probably was trying to shoot at him. Imagine a situation where a hundred arrows from a hundred archers are pouring down: how will this man dodge all of those arrows? Instead, he must be in a place when they all land, and whichever arrows hit that place must hit him. He can't turn himself inside-out and slip into a sideways dimension to avoid them all.

      But if you want to use a typical attack roll instead of determining grenade-like scatter, I'd just use the existing parry or full defense rules.

      Delete
  8. Now you need to start incorporating combat conditions. Your stats so far would work well for an archery tournament.

    It doesn't take into account light conditions.

    It is against a stationary target. It would be quite different if the target was moving and even more different if the target was aware. The target can be both proactive (trying to fake the archer out) and reactive (dodging the arrow when in flight, moving shield). Try looking up stuff about martial artists trying to dodge and catch arrows.

    Combat would make the situation different as well: being rushed because something(s) is trying to kill you; talking and coordinating with your party; having to be aware of your surroundings so nothing takes you by surprise; looking for and identifying targets; being attacked by ranged attacks yourself; etc.

    There are lots of statistics about police officer accuracy. e.g.
    http://www.cops.usdoj.gov/pdf/Deadly-Force-Statistical-Analysis.pdf
    http://www.theppsc.org/Staff_Views/Aveni/OIS.pdf

    There is a famous(too strong?) experiment called the Tueller Drill that determines how far an officer has to be from a knife weilding opponent to be able to draw and shoot safely and the distance is about 20 feet. Less than that the officer gets stabbed. This would be similar to a sword weilding fighter charging an archer with arrow in quiver.
    http://en.wikipedia.org/wiki/Tueller_Drill
    The link to the original article is in the notes.

    Keep it up!

    ReplyDelete
    Replies
    1. Yeah, actually: if you look at the last post (here), under "The Model", you see that I'm giving the D&D game archer here a total of about +12 bonuses for the target-shooting case. (Namely: +4 for target size, +3 for target null dexterity, +3 for shooter unmoving/unthreatened/aiming, and +2 for 20th-century equipment.) And then the results do line up nicely with both the "Archery" documentation and the bivariate normal physics simulator.

      So if you just back those out what you're left with is the standard D&D hit mechanic at close range against an uncooperative opponent (as one uses for swordplay, of course; about 50/50 for an unarmored opponent if they don't get initiative and close on you first). The residual conclusion is that the real change needed in core the system is to use something on the order of -8 to hit per doubling of range.

      Delete
  9. First thought: you're a better archer than I am.

    Second though: I suppose the ideal test would be to have multiple archers of differing skill levels do a static test to provide a baseline, then do a run through a Hogan's Alley firing at multiple pop-up targets with scoring for time and accuracy to see how the scores correlate to skill, but that's hard to set up.

    ReplyDelete
  10. Given that your shots at 40 yds were mostly short, it looks as though this is a systematic error that could be fixed once you had a proper feel for the range/elevation relationship. That could make a significant difference in hit probability. It would have been useful to measure dispersion about the centre of the impacts at different ranges and distance from the centre of the group to the point of aim. You would want a much larger target so you know where the "misses" are going. For practical hit probability, a man-shaped and sized target would make sense.

    ReplyDelete