Project Horseshoe 2012: Kickstarter and Training Designers
Wednesday, November 7, 2012 at 11:16AM I'll report to you about two topics discussed at this year's Project Horseshoe conference.
Defense Grid 2's Kickstarter
So, I went to business school at MIT a long time ago, and one of the main methods of learning about business is carefully going over and discussing case studies. A case study is business situation where a company has one or more challenges, and decides what to do about those challenges. They usually have a bunch of data that that helps point the way toward what a good decision would have been.
Jeff Pobst of Hidden Path Entertainment presented what I think is the "perfect" case study about his company using Kickstarter to raise money for Defense Grid 2. They had a clear goal, they had a very large number of possible ways of going about it, and they had a mountain of data before, during, and after the Kickstarter campaign. We get the benefit of hindsight to see exactly what ended up mattering in all the confusion, and it even makes a good story because of the punchline.
If I understand things correctly, I think they were probably going to make this game no matter what happened with the kickstarter, but by doing a kickstarter they could potentially make a much bigger scale of a game. That makes sense, and I do think it's a fair use of the fundraising platform.
They asked for $250,000, but I think the REAL goal was to hit $1 million. What makes this case study so interesting is that they treated it like it was going to make $1 million, and they invested heavily and did tons of work to make that happen. They did so much work and tried so hard, that it created even more data for us to look at, to see what really ended up mattering and what didn't. While publicly, the project appears successful because it raised $271,726, Jeff's presentation showed that the real story is it wasn't as successful as it appeared. They really made less than $50k (that can be put toward development) on the kickstarter after subtracting all the costs they put into it. It was sure a good try though, and it meant they got to take a good solid swing at the $1 million mark, but not lose money in trying for it. It's also generous of Jeff to share this data with developers, so that we can all learn from it. Most companies consider it way too private to share.
Jeff Pobst seems to know everyone, so he was able to talk with Tim Schafer, Jordan Mechner and other big names about their experiences. Everyone says the video at the top of your kickstarter project is really important. Having a well known figure pitch the product also helps, and they felt they didn't have that so they put twelve different famous game industry people in their video, endorsing their product.
You have a lot of knobs to adjust in a kickstarter project. How many tiers of rewards should you have, what exactly should each one be? How important is the reward structure compared to all the other things going on? Is it super important to have some rewards that are really really expensive? Or does that only matter like 1% in the end? What about upselling people? That means getting a person who pledged to later decide to change their pledge to a higher amount. Jeff had heard that this was actually the single most important factor of anything: how good you are at getting that upsell.
There's a whole bunch of other things you might focus on too. Using Facebook and Twitter to make sure your current fans know about the kickstarter. Also, getting mentioned on various news sites. They hired a PR firm who was relentless in following up many, many times with every gaming site they deemed important to their cause. You can also used paid advertisements to get the word out: google ads, facebook ads, or ads targeted on specific sites like Penny Arcade.
Hidden Path also did some less traditional forms of promotion. They made new content in their current game that was designed to hype people for the new game. They even had the opportunity to work with Valve by putting some Portal 2 ARG (alternate reality game) stuff into their game, in order to promote Portal 2, and this also helped promote their own stuff.
Finally, Jeff had the clever idea of approaching companies too big to participate in kickstarter, and made them an offer. He said if those bigger companies would give him free stuff to give away, he could promote them and their brands through kickstarter. This also gave these companies reason to mention his particular kickstarter project. It took months of negotiation to pull off, but Jeff finally got 100 gaming mice from Razer and 600 video cards from AMD. He pulled it off.
And now for the look back at it all. Of all this stuff they spent all this time on, what really did anything in the final analysis? It turns out that the money from upsells is not what it's all about. Yeah it's nice, and it helps, but it's an order of magnitude less important than something else. The #1 most important metric was reaching new people. It was getting new people to even come to the kickstarter at all. You'd think the obvious way to do that is through the media. The PR firm's work, and all those news stories they got...except that didn't really work. The media was feeling kickstarter fatigue by then, and no one really wanted to cover yet another kickstarter project. Some sites started having policies explicitly against covering kickstarter projects, even. The OUYA was also announced the exact same day as Defense Grid 2, and perhaps they were lost in the storm there. They had metrics for exactly how many dollars of pledges each site's posts ended up being worth, and while it's something, all of that didn't amount to that much.
So now it's time for the punchline, the one single thing that in retrospect, was the whole ballgame. It was about directly e-mailing customers. The media efforts were ultimately not hugely successful in getting people's attention, but directly e-mailing their base was. The bigger your e-mail list of your customers was, the more reach you had. Real reach, the kind that does something, compared to the somewhat unreliable reach of 3rd party news sources. And this was also the problem. I kind of missed a detail here, but for some reason, Hidden Path had absurdly few e-mail addresses of its own customers. They had relied so heavily on Steam that they didn't realize when it came time to actually contact everyone...they couldn't. With 500,000 active players, they had only 6,000 e-mail addresses. Sending out those e-mails was one of the most profitable things they did, and had they been able to send out 500,000, it would have been a whole different world.
In the future, they will offer some free in-game item in exchange for getting your e-mail address.
Training Game Designers
Maybe you've heard the figure that you have to do something for about 10,000 hours to be good at it (a stat popularized by Malcolm Gladwell). And not just do it, but do it with "effortful study" or "deliberate practice." Game design is really hard to do well, so how exactly do you get any better at it? The default way is to work really hard on making game after game after game. We all agreed that yes, that will allow you to improve, but it sure is a damn slow way. It takes a really long time to make a game, and a lot of the work involved isn't exaclty game design.
Other fields have this same issue. To learn to play a difficult piano piece, you don't play the whole song every time. Instead you practice the hard part specifically. To get better at playing basketball, you don't actually play games over and over. Instead, you isolate parts of games, like practicing just 3 point shots, or just free throws. You also do sprints and physical training. Yes you practice playing baskeball games too, but the amount of training that is not literally playing full games of basketball is very, very high.
The question the group had was are there ways to do "drills" or something...anything...that would allow you to be better at game design other than the slow path of making several games?
The group tried to construct a list of "rules of thumb" in game design, like nuggets of wisdom. And from that, tried to construct various drills that would allow someone to get better at those things. The group felt it was necessary to list out a bunch of this stuff because "game design" is such a broad topic, that some parts of it might be learned in completely different ways than other parts. There's system design, there's creating the objects that go in a system, there's narrative design, there's tuning the feel of a game, there's managing a community to get feedback on a game, there's UI design, and then overarching things like how to provide creative direction on a project. There's more too, that's just a quick list.
Game Jams
The first and most obvious thing is to participate in Game Jams. Those are events where you make a game in a weekend with a small team or by yourself or something. I personally don't think I'd get anything out of that. Other group members though it was actually very valuable, but everyone agreed we were after new kinds of answers that weren't Game Jams.
Just Do It
I pointed out that there are some types of skills that there's not much of a secret to. You really just do them over and over and over to get better. For me, graphic design has been one of those things. I've made so many card backs, rulebooks, web graphics, boxes, and so on that I do graphic design like every day, and I have for years. I wouldn't need to do separate "drills" on it. Perhaps another example would be designing Magic: the Gathering cards. If someone wanted to be good at that, yeah there are concepts to learn, there are many ways to go about it, but I actually think just designing a lot of Magic cards, over and over, for years, would be a good way to do it. Get feedback on them of course, so you know what's good or bad, and learn from that as you do more.
System Design
What about system design though? It's one of my strengths, but some people have a really hard time with it. What could I tell them that would help them, what could they do to get better? It's pretty hard to tell someone how they could be better at making game systems. So then I thought...how did I get any skill at it? I had never really thought about this before, but actually I had a whole lot of practice at it. For my whole career, I've been plagued by a lack of programmers. Sometimes this was at struggling game companies, where I was charged with figuring out game designs before production started, then the company would go out of business before we made the game. I also led a pitch department where I wrote design documents for games we proposed to make, again without the benefit of actually being able to try them.
In all those situations, it forced me to think through designs more fully and more carefully than I would have had to otherwise. If programmers were plentiful, then when I had even the first half-assed idea I would have said someone implement this and let's try it. But "let's try it" was such an incredibly rare commodity that it was like launching a missile to the moon. It's something I'd get one shot if, it that, so the idea had to be as solid as it could possibly be. Thinking through all those designs and all those pitches, means that I had a whole lot of practice thinking about systems, more than 20 of them, without having to make 20 complete games.
One researcher in the group was concerned that this type of thing is not falsifiable, meaning you could do this and be terrible at it and have no idea. Yes I agree, and yet it probably helped me a ton more than doing nothing would have. When I finally got to actually make things for real, probalby having "fake done it" 20 times already helped a lot, even though we don't know what the quality of those 20 things would have been. Another factor is that my MBTI personality type is INTP, which is known as "the architect," aka the system designer. So perhaps my brain is just wired to think about systems, and maybe someone else who has a differenet kind of brain would find it much harder. (They would perhaps find a different aspect of design much easier though, something else that I might be bad at.)
Predicting Dynamics From Mechanics
I also mentioned that one of my hobbies is reading the rules to board games. When creating systems, one of the biggest things is being able to create something that has the *dynamics* that you want. That means you need to be able to use mechanics (rules of the game) that cause players to play in a way you want, that's the dynamics. To use a fighting game example, if you change the mechanics for throwing such that throws have lots of startup time, are easy to escape, are escapable for zero damage, and you put in option selects to make throws even easier to escape, then the resulting *dynamics* would be a really defensive game. Blocking would be really good because throws would be bad at beating it. You should be able to predict that without even seeing a single game played with those rules, but you can then test your belief by watching actual players play the game. Likewise, I can test my predictions of board game dynamics by finding out how board games are actually played. I can ask expert players, or find examples of their gameplay, or their forum comments, or whatever.
I never really thought of this as a game design exercise, but I guess it is, and I've been doing it for years. It's building up my predictive power of knowing which mechanics create which dynamics. And that's the next point I brought up to the group. Specifically in the realm of designing competitive games, there's a skill of being able to predict dynamics that come from mechanics. When it's a competitive game, there's a bit of an extra layer involved compared to other games. If we were making a single player game like Mario64, and we give Mario a bunch of mechanics, the main question we need to care about is how the average player is going to actually use those mechanics. For a competitive game, yeah we need to ask that still, but we need to ask another question that is very, very hard to answer: how will the absolute best players use these mechanics at the highest level of play?
Answering how some theoretical set of future players will use everything at the highest level, in a metagame that doesn't yet exist is pretty far fetched to be able to do. Short of making the entire game, finishing it, and getting a community of players to play it hard, the best you can do is guess. And the way to train your skills in guessing is to look at the tournament play of other games that you know something about. I know how Street Fighter is played at a high level, for example. I have at various times followed other games such as Magic: the Gathering and Starcraft. I'm familiar with how things are often used in opposite ways than designers intended, like maybe a mechanic designed for offense is actually used for defense. By seeing the patterns of how players use things at a high level in several different games, I can predict a little better what's going to go wrong in a new competitive game, and try to build a system to counteract that from the start.
Close Study of Other Games
One example I thought was interesting in learning from other games, was the example of an exercise for level designers. Sure it helps to look at the levels of other games, but the exercise proposed was to play a level in some game and actually map it. That means draw a picture of it, including all the spawn points for enemies. By going to that level of detail, you might see things you'd otherwise miss. "Oh that's why there is cover here. Oh that's why this part was hard, the specific places the enemies spawn made it harder than it normally would be," and so on.
Mental Models of Mentors
There are child prodigies in math, music, and chess. These areas are all about understanding the workings of systems. And yet there are no child prodigies in writing. To write a great work of fiction requires something I'd call "sensibilities." That takes life experience to develop. The question of what to subtract in a game design is, to me, similar to the question of how you become a good writer. In each case, you need sensibilities. So I thought about how I got to be a better writer, and a lot of it is hearing words of my past writing teachers echo in my ears. If I write an awkward sentence, I can just hear them cringe. I have mental models of them in my head, my own little simulatoins of them. I developed an understanding of what they like, and more often, what they hate.
I explained that I thought there was value in looking at very specific cases and knowing what various mentors would say about them, the opinions of those who already have the appropriate sensibilities. As one example, I mentioned that for Yomi's box, I subtracted away everything I could from it. As I worked on it, I asked myself what Steve Jobs would say. I knew he'd hate any kind of bullet points or quotes on the cover, and that he'd want the most minimal design possible.
This really resonated with other group members. One person said he got help on something from expert game designer Mark Cerny, and that it had helped him tremendously. Specifically, Mark sat down and played his game (still in development) for an hour, and spoke out loud about it as he played. This illuminated what he noticed, what mattered to him. It allowed others to build a mental model of Mark Cerny and his sensibilities.
It's not necessarily that these mentors will give you the "right" answer. It's more that each is a different lens to look through. The Sirlin lens is quite different from the Steve Jackson lens, for example. While I might be critical of one of his games for being too random, or not strategically interesting at a high level, it doesn't mean those games are "wrong." If you had a Steve Jackson in your head, he would remind you to be fun and funny and lighthearted and have his own valuable and different perspective. If you are making a competitive game though, it might help to know that I'd cringe at some property of your game.
While there are a lot of different skills in game design that you will have to train in different ways, the skill of "sensibility" is, I think, one of the most important ones. It's what allows you to be a creative director, to make calls on all the many tradeoffs that will come up. It's hard to be good at that without much experience, and frankly, it's hard to be good at it even with a lot of experience. So as a tool, I propose that you attempt to download the sensibilities of other experts as your guideposts.
Reader Comments (8)
I loved defense grid, and never heard about the kickstarter until I read the blog post. If I had been emailed about it I probably would have contributed.
I'm trying to get into Game Design at the moment and I'm especially keen on System Design and Mechanics.
The idea of creating game design drills is really interesting to me. How hypothetical was the discussion about creating these drills? I suppose practicing these alone would be feasible, but having some way to get feedback seems like it would be better for setting people on the right track.
Thanks for the write-up, reading about these developer conferences is always interesting.
-John
There were some pretty specific drills proposed. You could check out the paper that lists them, they will post that in late December. I personally "practice" by working on game design of new ideas as my hobby, when I'm not doing my real job of making boxes, rulebooks, cardbacks, and logos. Maybe get a friend who is interested, think through some designs in detail, then see what other people think. (Or uh, make the games if you stumble on a really good one.)
At first I couldn't relate to your description of having mental models of experts at all, but as I ready on I started to realise I am developing a mental model of you in my head. After reading so much of your work recently I'm starting to get some inkling of how you would view different design choices, especially how they affect competitive play. So thanks. And also thanks for the great pair of articles, I'm looking forward to reading more about this conference, especially some of those white papers.
I've thought about these "mental-models" of various people fairly often now. At some point I used to say, that I do not have an opinion of my own. I would simply listen to various opinions and statements from people I respect - to the point where I'd know what they would say at any given time - and take bits and parts from their arguments which I consider to be correct.
I have a question:
In which form did you actually think your "not-developed" games through? I mean, was it more text, more drawings, more brain storming or anything else? Where did you start?
The #1 way to train someone for systems design is GRIEFING. The best way to learn how to build a solid system is actually in the negative: build a system and just break it over and over relentlessly until you can't anymore and you will have a solid game. This is easier and quicker to do in your head than in a prototype, prototypes are for the breaks you couldn't have thought of ahead of time.
You can learn to break systems by playing a game in which you are a griefer or being griefed. Griefing for this purpose is defined as playing the game in an unintended manner which harms the experience of another player who is playing the game in its intended manner. Griefing can be a huge part of playing to win in a poorly designed system and, in any case, having a play to win mindset is equally crucial if you are making a game with explicit goals. Personally, I don't have a lot of fun ruining a game for others, but playing around griefers and evading their wrath gives me the same high as a bunny escaping a coyote's jaws. Whichever side you are on, you should constantly think about ways to break the game: strategies with highest ratio of meta-game resources (skill, attention, etc.) to in-game resources (levels, money, etc.) that are no fun either for the people your playing with or for yourself. Most players will simply avoid these broken strategies or flame or blame people who use them. Your job is to do them or have them done to you over and over again, repetitively, until you know how and why they work. You may often find, to your delight, that the game isn't broken, and that there actually is an engaging answer to balance the "griefing" playstyle. If you do, congratulate yourself, and then quickly resume playing the parts of the game that are no fun.
For this purpose, you want to stay away from anything that has a very high-level metagame. Most professional starcraft players can be considered "griefers" by the above definition, because they're playing the game in an unintended manner. When you're good enough to design games so solid that playing in unintended ways creates engaging new strategies instead of just breaking the game, you won't need this drill.
I definitely think that systems analysis (not necessarily griefing) is really helpful in game design. Breaking a system or finding gaps for abuse is pretty core to the idea of making a good, polished game.
Griefing -- that is, actually doing whatever abuses that make things un-fun for others -- makes you a troll. Testing to see if you can is fine obviously, but doing more than that is unnecessary.
I know my reply is over a month old, but I just noticed this article.
The image I received from other sites of Defense Grid 2's Kickstarter paints a different picture.
Some people called it deceptive or even downright fraudulent. From what I understand, the Kickstarter text was changed after people started complaining. Supposedly the pledge tiers originally didn't include Defense Grid 2.
So there was a Kickstarter for "Defense Grid 2" that was raising the money to release a DLC pack for Defense Grid 1. Defense Grid 2 was a $1,000,000 stretch goal to its own Kickstarter. You can hopefully understand why people were dubious, annoyed, or otherwise not exactly fully accepting.
In some ways, it is similar to Alpha Colony's second failed Kickstarter. The first time, they sought $500,000, but ended it with only around $100,000. So they did a second Kickstarter where they asked for $50,000. The second Kickstarter failed, ending $28 short of its goal. Normally, that would be the end of the story, except...
One of the developers then said that it was good the second Kickstarter failed, because "To be committed to deliver my dream game underfunded, understaffed, and sub-par, and to lose even more time and money would have been even more heartbreaking." They'd deliberately chose to set the funding goal below what they felt was necessary to make the game. "That is why I put the original game up for $500k -- because that was what I felt it would take to do it right and not lose money. We scaled it way back and added stretch goals to $300k on the second Kickstarter in the hopes that we could at least achieve the same $100k level we got before." My guess is that they were trying a psychological ploy, expecting more people would be willing to jump onto a lower goal project, because there tends to be positive press when a project rushes past its stated goal, and because there is more backlash against high Kickstarter goals.