Guest Post: Blind Playtesting by Jeff Hunt

Here’s another Guest Post for our series, What’s Your Story. This one covers his experiences with blind play testing. It’s an amazing story about taking blind play testing to the max! If you have questions for Jeff, post them below and we’ll make sure he answers them! My first question Jeff is – how did you find so many people to agree to blind play test your game?!

Shifting the Assumptions I have Made

Jeff Hunt – Game Designer

As an introduction, I’m just going to say that I’d been working on a prototype board game called Knight’s Quest for many months now, and have play-tested it several times with my regular group of close friends and game-pals.   Recently, I had the daunting opportunity to invite people from outside my close circle of gaming friends to come out and try my game.   I won’t make excuses in this post, but this play test came at a very busy point in my work life (much weekend and evening overtime to launch a large project at work).
With much help from my friend Kyle, we rented a community hall for part of the afternoon and the evening of our blind play-test.   We invited a total of 28 people to come join us in trying out the board game, with a session of 10 players in the afternoon and 14 players in the evening.  Kyle, Corey (my design co-pilot) and I were there to facilitate, answer questions and referee.
At this point I want to digress briefly.   I’ve played a lot of games.  I’ve liked board games and gaming since my teen years.   I have played a long list of games (that I won’t go into here) but I’ve never had someone from outside my close circle of friends playing a game I worked on or designed.  This was brand new territory for me, and to be honest, prior to the actual day of the event, I was quite nervous about the whole experience. Stage-fright shouldn’t have been a big issue, and the results of this play test session proved that.

The Morning Session
Kyle and I had arrived to the community hall early and started setting up the game tables so that they would be ready for the players to begin.  We had 3 rooms on 3 levels, and ended up with 1 active table on each of the three levels.   I gave a preamble speech to the players, thanking them for their time on a Saturday afternoon, and basically setting them free to experience the game with the sole help of the rulebook, quick start guide, and reference page I’d provided everyone.  Then I sat back, tied my own hands, and watched the various players attempt to decipher my rules and get their game started.
I was hit by an avalanche of learning from the minute this began.  Some rules that were very clear in my own (biased) mind were not as intuitive as I’d hoped.  Assumptions I had made (biased) about how gamers would treat game components and rules that (I thought) were explicitly laid out, were being interpreted three different ways at three different tables.   It was chaos – beautiful, beautiful, educational chaos.  I tried very hard not to jump in and advise or help until the players had had a chance to talk things over inside their groups, and only when it seemed like an insurmountable hurdle had come up to block their path toward a successful (and fun) play-test of the game.  If players came to an agreed upon solution to any fogginess in the rules, I simply sat back and watched how that interpretation played out.
I learned that even with a fair bit of technical writing experience under my belt, and years of publishing technical help guides as part of an IT career, talking about game stuff with people of different levels of gaming experience was a whole new sport.   I was going to have to go back to the beginning and re-identify all the unspoken assumptions and hidden pieces of brain-trust that hadn’t been successfully translated into my rules.  I was going to have to unpack them carefully, and then reassemble them in language that a non-gamer, non-me person could easily pick up and understand.
In short, the experience was awesome!  (I mean that.)  I know it sounds a little strange to be so excited about a moment when hard work and carefully crafted rules didn’t seem to line up into a perfectly successful play-test of the game the way I thought I had designed and described it to be played, so bear with me for a moment.
Quite a few of the people who came out to this were associates and contacts who were not really board game enthusiasts, so I was expecting a bit of a learning curve right off the bat. Some of them were avid gamers, however, and they were the most inventive in trying to find ways to make the game work for them.  I was taking notes furiously, both mental and hand-written, to capture any of the bumps, hurdles and blockages encountered by this group.
The whole time, Kyle, Corey and I were asking people if they were having fun, and trying to encourage a social fun aspect to the event.   All of the groups, with only a few gentle nudges in the right direction, managed to get several rounds of play-test under their belt, and were beginning to develop a feel for the play of the game.
About a half an hour prior to our first session ending in the community hall, we had the groups pack up the game and fill out a questionnaire about their time play testing.  Kyle and I had populated the questionnaire with some specific questions about mechanics, and room for some open ended questions and free-form comments from the players.
To my astonishment (and gratitude) two of the players from the morning session decided to come back to the evening session for a second try at the game.  To me this was a sign that I was on the right track to a fun and enjoyable gaming experience.   When the afternoon session was finished, we hurriedly packed up and vacated the community hall for another group’s use, and went off to find a way to kill the time in between the early afternoon session and the evening session.

The Evening Session

Having learned a little bit from the morning session and the learning curve involved with getting the game up and running without guidance, I decided to change my preamble speech and provide a 1-turn demonstration of play so that the players could get a look at how the intended play of the first turn would look.
This seemed to get the groups up and running much quicker than the morning session, and resulted in fewer over-all questions about the game, which I believe was a win for myself, the developer.  Again, the learning started flowing in almost immediately.  There was different learning this time, as the evening crowd had more people with gaming experience in the mix.  And we had the 2 returnees from the morning session who had freshly learned how to play the game that day, and may have come in with new strategies to try, or new things they wanted out of the game.
Because about 1/3 of the evening session included some people from inside my immediate gaming circle, it seemed like a very sociable event.  People were chatting and joking around with one another, and everyone seemed to be having a ton of fun.  We had the hall booked for 3 hours again, and allowed the evening session to run for almost 4 ½ hours as there were no other clients waiting for the space.
Again, at the end of this session, we had people fill out the questionnaire, and I thanked them all for their time on a Saturday night.  When it was over, I was exhausted.   I was happy – but exhausted.  It had been a super long day, but a super fun day too.   My game had been reasonably well received by people I know, and some I didn’t, and today, two days later, I feel extremely optimistic that the feedback provided and the observations about rules interpretation will greatly help me tune this game into something totally awesome!

What would I do differently?

If I hadn’t been so incredibly busy with my work leading up to this play-test, I would like to have taken one more very close pass on the rules prior to the event.  I would probably re-order the questionnaire so that the specific mechanics questions came first, and the general impressions questions were last – just so the most immediate feedback might cover some of the specifically complex or troubling mechanics that needed addressing.   I would have the play test in one continuous session over the course of an afternoon, rather than breaking it into two separate and time-limited blocks, and would probably do it in a residence rather than a public building – just to save a couple of dollars.
Other than those notes, I think that, for me, the play test was extremely successful.  I couldn’t wait to start crunching the data and tuning up the rules to make a fun game even more fun and accessible to everyone.

The Feedback

The feedback I received from the players was instrumental in helping me tune my design further.  In my work-life we operate under an iterative methodology.  In my school career, my work was subject to many critiques and subsequent iterations to improve the end product.   Applying this methodology to my game design seemed like a no-brainer.
Probably the most important parts of this feedback was understanding when rules I’d carefully written out in the rulebook were not being interpreted as I understood them.  Also important was finding out some unasked questions that I hadn’t considered or answered because I was so close to the project that they weren’t questions for me.  Aside from some minor mechanics tweaks that came out of various feedback, the biggest change to the game after this was to really take a long close look at the rules and ensure that there were plenty of diagrams accompanying explanations of how the game was supposed to function.
Blind Play-testing sounds a bit imposing, but I now consider it a primary and integral part of any other design I’m working on.
I also received some general feedback about the game being fun, or the theme being nice and that sort of thing, but I try to focus most on the comments that surround making the game play better, and making it easier to understand.  I fully expect that if a Publisher accepts the game into their portfolio that there will be artistic changes and tweaking – particularly since my prototypes generally have clip-art or basic diagrams for many of the printed components.

Conclusion

Blind Play-testing can be tough for a designer – and especially tough for a designer who has never experienced a critique or iterative project.   To set one up on the scale that we did for Knight’s Quest, you have to have some organizational skill, a budget of some kind (if you’re using public facilities), a way to find and invite people to your play-test sessions, and a way to solicit feedback that is meaningful.    Then you have to have the patience and forbearance to either let the play-test take place without you, or to keep yourself from influencing it by interfering with the play-test groups.  Finally, you have to do something with the feedback.
The first few parts can be pretty straightforward.  Invite a bunch of people you know peripherally, or don’t know at all to your event.   Bribe them with Pizza and soft-drinks if you like.  Let them play your game and fill out preplanned comment cards or questionnaires about it.
The best part comes afterwards, when you analyze and react to the feedback.   You can play-test all you like, but if you never evaluate feedback and use it to inspire positive change to your game, you’re probably just wasting your time.
It took me several weeks to go through all the feedback and determine what was useful feedback, what was “fluff” and what feedback could be discarded – and I did discard some suggestions.  Overall the experience was positive for me.  And I think it strengthened both my rule-writing skills and the game design itself.  I would do this again for all of my game designs. However I would caution any designer who is planning a blind play test that the comments are going to range from friendly to not, and treating them critically is going to require some critique-management skills.   Good luck if you organize something like this, and I hope all of your feedback is constructive!

-Jeff Hunt

I also want to thank my good friend Kyle (a co-owner of www.willworkforgames.com) for his work and dedication in helping to arrange and organize this event.  Without his encouragement (and pushing) it may have taken much longer for me to get a play test off the ground.

Advertisement