Posts Tagged indie games

Banana Breakers: Indie Dev Learnings

Before trying a more ambitious game on mobile, I wanted to do a series of smaller experiments to learn a little more about the app submission / marketing process. I’ve got a few apps out already, but those were mostly hobby-fun-tinkering projects and not built with *serious face* revenue in mind. Experiment is probably the wrong word, since good little children who paid attention in science class know that a good experiment is designed to test a specific hypothesis, whereas with these first few games I’m mostly taking stabs in the dark to see what I can learn. What I’m hoping to learn, though:

  1. What tech should I build my games with?
  2. What platforms should I support?
  3. How can I get some exposure without spending too much?
  4. Which third-party ad/analytics providers should I use?
My first game, Banana Breakers, has been pretty helpful so far.

1. What Tech Should I Build My Games With?

I’ve been working on Banana Breakers (link to game site) on and off for probably four years, but in terms of actual complexity it’s a really simple game. I originally prototyped it in Flash a few years back (just the core mechanic of 4/5/6 words hidden in a grid), but it was Too Damn Hard. It was such an easy prototype, though, that any time some shiny new technology came out I would re-implement it to test out that tech. I have a Haxe version. I built an Enyo (javascript) version a few years back when I got a Touch Pad. I built a native iOS version using RubyMotion when that came out last year. When my friends at needed beta testers, I re-built it once again in Flash–but this time thinking of it as a mobile app. Eventually I ran into technical problems with Spaceport (they didn’t have 3rd-party code support yet), so I switched over to Adobe Air. When I heard about Futile (a 2d plug-in for Unity) and Loom, I almost built new versions… I may have a problem.

Banana Breakers was also a departure from my earlier games in that it would be free, but with ads and the ability to buy more coins. I also wanted to support incentivized video as a way to get more coins without paying, as those always converted fairly well back in my Facebook days. Finding a tech that supports all of those (well) turned out to be more challenging than I thought.

Adobe Air and Unity both support native extensions, but Air’s implementation is god awful. I lost an entire week just trying to get the “Hello World” of native extensions to compile with my setup before saying “screw it” and just going with off-the-shelf ANEs (some of which I had to buy). Because I needed so much 3rd-party support, I crossed Unity off my list pretty early in the development process. They have even fewer native extensions than Adobe Air, and at the time they were still charging for the iOS and Android plugins. Even now, the free indie license requires you to use a Unity splash screen and doesn’t come with access to native extensions. I could probably get over the splash screen, but the lack of extensions would probably limit Unity to only paid-download games for me.

So what tech did I end up going with? For Android and Kindle Fire, I used Adobe AIR (since I already had an AS3 version ready to go). I could have re-written using Starling to get better performance, but at that point I was ready to release something and get some data. I wasn’t happy with the performance of AIR on iOS, so I converted everything to bitmaps and build a native iOS version using RubyMotion. I’ll do another post later about how I’m using RM to build games, but for now my tech stack is pretty much RubyMotion and Photoshop (with some custom scripts to help speed up UI development). And it’s a lot of fun. I don’t know if it’s the right choice (iOS only), but so far I’m really happy with it.

2. What Platforms Should I Target?

There’s two ways to think about this:

1) Supporting multiple platforms is expensive and only worth it for a successful game.
2) It’s easier to be a big fish in a small pond–targeting less successful platforms might be a way to get traction/cheaper users/less competition.

My original thought was pretty much #2, assuming I could settle on a tech that was cross platform. In practice, Banana Breakers has pretty much made me change course entirely back to #1. Even though I originally built the game to be cross-platform, the amount of work needed to set up in-app-purchases, ads, and other native “touches” turned out to be pretty nontrivial (especially when you throw in device testing). At least for me, building a game cross-platform means it will take probably twice as long to build… and at this point I’d rather make more games.

I’ve also been disappointed with download numbers on both Kindle and Android. For Kindle, it seems as though free games don’t even appear in the “new games” section of the app store. That makes sense in hindsight (their marketing engine is designed entirely around promoting things that cost money), but it was still a bit of a shock. Android was slightly better (3-4 downloads a day), but nowhere near the 500-1000 downloads I got while on the “new games” sections of iTunes. Having a game out on multiple platforms would also mean (theoretically) that my marketing efforts would need to be split between the platforms. Realistically, though, I’ve been focusing all my efforts on the iOS version and completely ignoring the Android versions.

If I have a game that is a monster success, I’ll consider looking at other platforms. For now, though, the decision to go iOS-only for future games is an easy one. If anything, I’d consider doing desktop-PC on top of iOS before I would consider doing android or kindle. If I were to do a native version for any other platform (Windows Phone 8 is tempting, since I’m one of the few users), I would just do a native build for each platform.

3. How Can I Get Some Exposure Without Spending Too Much?

Distribution is probably the single biggest hurdle to making a living as an indie developer right now. I’m nowhere near figuring out, but I can run through what I’ve tried so far:

1) Release the game. Free. I got ~500 downloads just for releasing the game on iOS.

2) Review site blitz. Free (ish). ??? downloads. I sent out ( slash-filled-out-contact forms) somewhere in the neighborhood of 150 emails to various mobile review sites. The vast majority of them did not respond, and those that did were offering paid reviews. I got a nice write-up from JayIsGames (they’ve written up some of my flash games in the past as well), but nothing else. So far this seems like a giant waste of time, though I’m sure bigger sites do create a nice download bump. I think it’s worth to build relationships with sites that aren’t too scuzzy, but at this point I think efforts spent on this type of PR would be better spent doing blog posts and engaging on twitter / forums.

3) Gnome Escape. $250, ~200 downloads? I’ve read a lot about how shady pay-for-review sites. Gnome Escape was on one of the giant lists of publishers to contact for reviews, though, so I gave them a look. They say all the right words–authentic reviews, etc… but ultimately it’s just incentivized installs (with incentivized reviews on top of the incentivized install). I got a nice bump of 50-60 people a day for a few days (along with several who monetized) and some good reviews, but most of the reviews were spam-bot level in quality (just regurgitating my app description). My thinking was that 200-250 users for $250 is about $1 a user and not too bad… They delivered what they offered, but I don’t think I would use the service again.

4) AppFlood. $250 each, cancelled. Just to get a sense for how they worked, I dropped $250 into my AppFlood account and set about driving some paid installs. To get any sort of volume in the US, I would’ve had to up my bid to the $3-$4 range, and there’s absolutely no way Banana Breakers could recoup that. I want to try ChartBoost later (they have a lot more volume), but so far these seem pretty overpriced.

5) Free App Magic. $1000, ~8000 installs. The last thing I wanted to try was some way to get burst downloads (and hopefully make it onto the charts). There are tons of free-app-a-day apps that promote an app or two a day, but most of the big US ones cost thousands for a feature. I set out to find a slightly smaller one with a more reasonable intro rate, and FAM definitely fit the bill. The intro rate I got is probably not repeatable (I took the place of a game that cancelled at the last minute), but the results were great. They estimated that they drove ~4k installs, while the other 4k were organics from hitting the top ten in word games in the UK (most of their traffic is UK). The game stayed on the charts for a few days, but is now slowly dropping out. The 25-30 installs per day I’m getting now (a week+ after the promotion) is slightly better than the 5-10 I was getting before the promotion, but I’m not sure how permanent that will be. I consider this to be the only successful bit of marketing I’ve done (twelve and a half cents-per-user would be a phenomenal rate for a better-performing game, and if I could get similar CPIs at other daily download sites they’d be worth the higher fees).

6)  My Chartboost campaign was finally approved and I was able to run $250 through it very quickly, even at bids of $0.50 for US installs. I’m not sure how many installs are available at this price (I would guess that as you start trying to drive more volume the price would have to go up), but as an initial test Chartboost is another promising way to get users at a good price.

My original goal was to only spend $1k on marketing Banana Breakers, so at this point I’m $300-$400 over. It doesn’t perform well enough to keep spending, but hopefully I can take some of that knowledge on to better-performing games.

4. Which third-party ad/analytics providers should I use?

This has probably been the biggest shock with releasing a game. At Crowdstar we had an entire dedicated Business Intelligence department, a massive Vertica database to warehouse game event data, and dedicated analysts to help sift through the data. I’d always heard good things about Flurry’s event tracking, but from my point of view the data is sorely lacking. The install, retention, and session charts are pretty good… but there’s no easy way to hook it up to purchase data. And it’s not realtime. The data they provide seem like an okay way to track your installs and active users, but I don’t see how anyone could take specific in-game actions based on this granularity of data. I wanted to go entirely server-free for Banana Breakers (mostly because I’m cheap, and nothing wipes out meager indie game profits like server fees), but for future games I’m thinking I’ll roll my own. Helios (by the Heroku guys) looks strong, but doesn’t do exaaactly what I want. I’m leaning towards a super-simple redis-backed logging system with built in a/b testing, but it’s also tempting to abstract a little more and see if I can come up with a generic service for the things I want to do.

Chartboost > All Others

Ads have been a little better. Banana Breakers has interstitials (AppFlood & ChartBoost), banners (Flurry and AdMob), and incentivized videos (HyprMX and AdColony). So far ChartBoost is the clear winner–interstitials are blowing all the other formats out of the water. Banners are probably not worth having in the game. While their annoyance might drive players to upgrade, the game would have to be a total smash hit for the ad banners to pay for the extra time it took to make home screen layouts that support banners. I’ve been really surprised at how poorly the incentivized videos have done. I’m guessing that means either (a) most users don’t know they’re there or (b) the word puzzles aren’t hard enough to warrant users needing extra coins (which would also lead to low coin sales). Once I throw in in-app-purchases, the graph looks more like this:

So ads are actually outperforming coin sales by about 3:1. The numbers here aren’t huge (~$51 so far for coin sales, ~$136 for ads), but I think the trends are still pretty useful. If I’d had a back-end server set up, I would be able to tweak what portion of each type of ads go to each provider (home brew mediation). I’ll hopefully get that set up for future games.

Overall the budget for Banana Breakers was about $2600 + dev time, so it’s unlikely at that point I’ll make back the development costs, but it’s been a fun game to work on.

, , , , , , ,

1 Comment

Farewell Crowdstar

I recently left Crowdstar just shy of my three year anniversary. Before the memory of those three years fades into the crunch of trying to make my own games, I thought I’d look back at the best and worst parts of those three years. Before I get into that, though, it might be worth a little bit of background.

After Piqqem (my first job out of school) folded in January of 2010, I resolved not to get another job and to dive full time into indie game development. I’d only been a half-time employee for the last year anyway, spending the rest of the time focusing other random pursuits (iphone and xbox indie versions of Filler, a review site for other indie games, a couple of new Flash games, a movie showtime search engine that got a nice write-up on TechCrunch but was too expensive to run). One month into “going indie,” a recruiter emailed me pitching three different companies: AuroraFeint, Sibblingz, and Crowdstar. The story was the same on all three–game related, YouWeb funded, growing like a weed, desperately in need of engineering bodies. Instead of getting back to the recruiter, I looked up Peter Relan (the founder of YouWeb) and shot him an email asking how you get into the program.

We chatted on the phone and he thought I should have a chat Suren and Jeff, the two founders of Crowdstar. What actually happened ended up being more of a 3-4 hour “interview” of sorts. I talked with a couple of their Flash guys for awhile about what they were working on and games I’d made–one of them later told me that it wasn’t a real interview, but a “push-on” (their attempt to recruit me). After chatting SQL and ruby awhile with the guys at Sibblingz, I finally went into a big glass office with Jeff and Suren. Jeff immediately asked for a resume (which I hadn’t brought) and ended up settling for a LinkedIn profile chock full of all the bullshit made up websites and companies I’d been tinkering on for the last year or two.

We talked a bit about my background, but the thing that stands out the most was one question Jeff asked me: “Would you rather make a game that a million people play and like or 100,000 people play and love?” Crowdstar was clearly already in the business of the former, but up until that point I hadn’t really considered it. Only one month into my “make games to support myself” experiment, I was still making games that I wanted to play (and not really considering games as an economic vehicle). I gave him the 100K answer, which is clearly what he wasn’t looking for. Then again, I hadn’t signed up for an interview–I thought I was coming up to Burlingame for a chat about social games in general.

Peter invited me over to his house a day or two later and offered me the job of lead dev on Happy Island. His pitch was simple–working alone or even starting a new game company would put me way behind where Crowdstar was already, and he was willing to make me “CEO” of a game and let me run it like my own startup. It was a tempting pitch, but at first I was mostly in it for the money–I told him I wanted half a percent. Playfish had recently sold to EA for $300 million dollars and social games were exploding, so I figured if Crowdstar could grow to that scale and exit within a year or two I could clear $500k-$1mm over the life of a 4-year vest. I’m not sure he heard me right, because he said yes and asked if I could start the next day. I said yes.

As it turns out, Happy Island was in flux. Originally developed by Sibblingz, it was a flash game with a Rails backend. My first day was a Wednesday. Sibblingz was due to roll off the game on Friday, and Crowdstar had not a single rails engineer. As you might guess, that first week or two was crazy as hell–setting a good tone for what would follow. A rough timeline:

  • Happy Island (Feb 2010 to May 2010)
  • It Girl (May 2010 to April 2011) — also, I got married!
  • Cancelled FB Game (April 2011 to November 2011)
  • Closet Wars (Nov 2011 to Oct 2012) — also, had my first kid!
  • ??? (Nov 2012 to Feb 2013)

Rather than go through it all chronologically, I’m just going to ramble a bit about the ups and downs–mostly ups!

The Good: Working at Scale (and Scale Technology)

Before joining Crowdstar, I had experience with all the relevant technology–Ruby on Rails, MySQL, Memcached, Flash… but never at scale (well, maybe a little on the Flash side). The day I joined, Happy Island had 2.5 million DAU (still climbing on it’s way up to 3 million peak). The other web apps I’d worked on maybe peaked at around 100-200 DAU. When Filler launched and hit #1 on Digg back in 2008 (when Digg was cool!), it peaked at around 200k players per day. Not even 10% of what Happy Island was doing in normal day. On the web side, the biggest I’d done was a single cache server, a single app server, and a single MySQL box. Island was running multiple memcached boxes, a bunch of sharded, replicated MySQL instances, and a boatload of app servers (each running still more rails processes through Unicorn).

The first few months were crazy–coming up to speed on a new codebase (both client and server), writing sharded database code, adding new features, fixing bugs… and all the while monitoring the live deployment and waiting for things to go wrong. Operating a game at that scale basically requires you to think like web-dev MacGuyver. Something is always breaking or going down or getting hot, and hotfixes deployed in the heat of the moment are more like duct tape than carefully engineered responses. Crowdstar had (and still has) a great ops team, and I probably learned as much in my first two months at Crowdstar as any other time in my life.

Most of the hard work (architecting and scaling up) had already been done on Happy Island, though–I was mostly a caretaker. When I later rolled off and started up the It Girl team, we were building from scratch. I was lucky to work again with the guys at Sibblingz (now Spaceport), who are much better engineers than I am. For It Girl we ditched MySQL and Memcached entirely, instead building the entire backend on top of Redis (including an in-house ORM instead of ActiveRecord). Redis has since gone in a slightly different direction (everything in memory instead of just the keys), but for our purposes it was fantastic.

It Girl never got to the scale of Happy Island and we had a few quirks pop up with Redis, but all in all it was a mostly smooth ride from 0 to tens of millions of users. Not having to think about which shard your data is on, along with not having to worry about caching, simplified development tremendously (and is one of the reasons we switched to Membase for future games).

The Bad: Not Working at Scale

After It Girl, I jumped on to another Facebook project that eventually got cancelled. By that time the company had shifted to mobile, and that shift brought a lot of changes to how we architected our games. Because it’s harder to rely on an internet connection and harder to mess with the in-memory data of a cell phone, mobile games tend to be a lot more client-authoritative.  With simpler back-ends, there’s much less demand for server-side engineering.

Going from such a frenetic scale and pace (multiple deploys per day and constant monitoring) to something that felt almost pedestrian was a radical change. Part of this was due to the way mobile games operate. There’s no such thing as “push it and see what breaks”–mostly because of Apple’s onerous submission process (with 6-7 days being required for any client-side update). From what I know of console development, most mobile studios operate a lot more like our AAA cousins: develop -> QA -> release. It’s certainly a “healthier” way to run a business, but in many ways it felt like the taming of the Wild West.

At Crowdstar I was always the “rails guy” at a company mostly filled with PHP guys. The back-ends have now gotten simple enough that there’s not really a compelling argument for using Rails (actually, Closet Wars is a hybrid of Sinatra and Rails, but still) over PHP. Most of the back-end operations consist of either “get me this key” or “put this data in that key.” Had I stayed on, what was initially pitched as a Rails & Flash job would’ve been PHP & C++ going forward.

Addendum: Scale as a Shipped Game Title

I’ve noticed that my experience with scaling Facebook games has affected how I view other companies outside of Crowdstar. Being an engineer in Silicon Valley, I’m constantly bombarded by recruiters via email and LinkedIn. Before working on scale projects, I wouldn’t have thought twice about the size of a company and instead just looked at what they do. Now I’m a little reticent to even consider joining a company that doesn’t have a crapton of traffic–it’s less fun! (Not that my own projects will have significant traffic any time soon).

Likewise, it seems like there’s a perception of hiring people with scale experience that reminds me a lot of the AAA console industry. Most job reqs at companies like EA will explicitly state that one shipped title is required, but there’s no way to get that one shipped credit without getting hired by someone with no credits. It seems like a lot of companies are looking for people with scale experience and not willing to hire those without it. In that sense I’m incredibly lucky to have been baptized by fire at Crowdstar.

The Good: Doing it All

Once I had a handle on actually running Happy Island at scale, I got to start playing around with the product a little more. None of the Crowdstar games at that time had time travel cheats (i.e. skip ahead 8 hours), which made it difficult to do some gameplay testing. There were serious problems for high level users, so I took it on myself to rebalance the entire game’s economy. After building a “simulator” version of the game where I could play two-to-three sessions “per day” just by pressing a button restart the session, I played dozens and dozens of playthroughs and eventually created a pretty tight escalating economy. It wasn’t perfect, but I think it was pretty fun (and certainly extended the life of that game). Even though my title was “lead dev” at the time, I was in fact more like a producer–coming up with new features, balancing the economy, running the revenue reports… as well as a good bit of code.

When we later added the actual job title of producer (while I was on It Girl), I became the producer for It Girl (and the unshipped title and Closet Wars). Up until recently, though, titles didn’t mean a whole lot at Crowdstar. Being entrepreneurial (read: curious and willing to poke my nose into places I probably shouldn’t), I wore almost every hat possible: client-side engineer, server-side engineer, QA (before we had a “QA Department”), product manager, content manager, producer, game designer, data analyst.

Working at a startup with more tasks than bodies affords someone who likes to get their hands dirty (me!) tons of opportunities to move around and try new stuff. This was probably most true on It Girl–as we “grew up” as a studio and people took on more well defined roles a lot of that flexibility went out the window. Even on my last game, though–Closet Wars–I was putting a lot of time into being producer, doing game design, and engineering. One of the reasons I never really wanted to go into AAA games was the fear of being a cog in a great machine–the fact that I was still able to wear quite a few hats on team sizes approaching 20-25 people has done a little to allay that fear.

The Bad: The Hours (and Commute)

The downside to having more tasks than bodies is that there never seem to be enough hours in a day. This was especially bad when I first joined–we were still in hyper-growth mode and I was new to a lot of the tech. Because I had as much equity as I did (and, to be honest, because I wanted so badly for us to succeed), I always felt like I needed to be the last one out the door. For a long stretch on It Girl, I’d say I was putting in 80-90 hour weeks. I wasn’t the only one doing it, though–I remember many 2-and-3 a.m. nights with the Sibblingz guys. We had this sort of visceral feeling that the future of the company rested on our shoulders–I don’t know how true that is, but we definitely felt it like a force weighing down on us.

Those kinds of hours weren’t sustainable, obviously. After Crowdstar raised a ton of V.C. money and started shifting more into a “stable studio” mindset, the hours definitely calmed down into a more normal 10-to-7:30ish, 50-hour Silicon Valley work week. Even working 10-to-7:30 on most days, though, I had to tack on an extra half hour to each side for my commute. Factoring in 8 hours of sleep and a bit of time to make dinner, I essentially had 1 hour in the morning and 3 hours at night to fit in everything in my life outside of Crowdstar.

It took me most of that first year to “finish” ColorTangle, the game I’d been tinkering with when I started CrowdStar–a game that I’d estimated needing another month to finish. And it probably was about a month’s worth of work–just crammed into a few minutes here and there on the fringes of 8-10 months. With tinker time going out the window, I also cut back on almost all of my leisurely pursuits: exercise, moviegoing (and I’m a film major!), reading books, and playing other video games. I’m a firm believer that game development is a creative pursuit, and when your world narrows to the point that you’re not consuming any media and keeping up with your body, that creativity suffers.

Once the baby came (February 2012), that claustrophobic feeling of never having enough time doubled or tripled (and especially never having time for myself).  I’m sure that happens to all new parents, but I was on the verge of quitting for almost all of 2012 to try to win back some of that time. It certainly doesn’t help that I have an almost-neurotic desire to stay on top of all the current tech trends and try out every new gadget and technology that comes my way. What kept me going (besides my wife telling me not to quit) for most of that year was the fact that I had my own team (Closet Wars), we were making something that I legitimately thought was going to be great, and my team was awesome.

In that sense, I’m really excited to be working from home now–the hours and the commute were my #1 excuse not to exercise (except “i’m tired” or “i’ll start next week” or “slept funky last night”…), so hopefully I’ll be able to repurpose some of those hours into dropping 20 pounds this year (my pre-Crowdstar weight–damn you catered lunches!).

The Good: A/B Testing and Optimization

It’s one thing to read about A/B testing best practices on Hacker News, and quite another to run tests yourself on a massive scale and see what kind of impact they can have. When I joined Crowdstar, they were already A/B testing a little bit of stuff–but usually only running one test a time and modding on the user ID (which, at our scale, was probably random enough but not as statistically rigorous as it could’ve been).

While we were launching It Girl, it became pretty obvious that there were a lot of knobs and levers to tweak. The current mod-on-the-user-ID system would only let us run one or two tests simultaneously, and it was pretty brittle at times (was it user that end in 0-4 or 5-9 that were the control group?). I built a new system for It Girl that was not tied to user ID and was event-based, meaning we could split users into test groups at various steps in the user life-cycle. This was probably my favorite bit of code I wrote at Crowdstar, and now that I’m out I’d like to re-write it some day for my own personal use.

As an example–when we launched the boyfriends feature ,we knew we wanted users to have a minimum number of “clique members” (friends in the game) before they could access the feature… as well as a minimum level in the game. We were able to A/B test:

  • the optimum level to introduce the boyfriend features (testing for # of people who get a boyfriend)
  • the number of friends which should be required to get the first boyfriend (optimizing for k-factor… i.e. at 4 friends less people get a boyfriend, but is it more than ¼ of the people who were willing to invite one friend?)
  • the number of friends required to get each boyfriend from BF #2 all the way through BF #10

Between those tests and dozens of other tests we ran on price points, progression speed, and monetization features, I’d say that the A/B test system doubled or tripled the LTV on It Girl and taught us all a tremendous amount about player behavior. Maybe this goes hand-in-hand with running a product at scale, but it was also a hell of a lot fun to have access to that much data (in near real-time) and that much flexibility to test things.

The Bad: Time Off

When I joined Crowdstar, we were given two weeks of PTO (I think it’s up to 17 days now, which is better but still not great), which is totally standard for a new company. I travel a lot, so I was always scraping the bottom of the barrel when it came to PTO… and on several occasions either went way negative or took long stretches of unpaid time off.

Because all of the games we made were games-as-a-service, the usual lulls in the game product cycle just don’t exist. There was always something important going on, so picking a time to skip out for a week or two was always tough. Compounding my lack of time off was the fact that my wife works at EA–where she gets 4 weeks of PTO, the entire week of Christmas, and usually a couple of weeks of under-the-table comp time whenever they finish a game.

I had a bunch of people at work say things like: “how do you have so much time off?” — and I would always answer the same way: “I don’t.” I’m lucky that financially I could take the unpaid time (and was in good enough standing with the company to get away with it), but I always thought it was a downer that the majority of people worked so hard and got so little time off to show for it (though most people probably don’t travel as much as my wife & I).

To be fair, though, one mitigating factor for the lack of PTO was the company trips. My first year we went to Italy, renting a villa in Umbria and hacking for a week while surrounded by gorgeous sunflower fields (with day trips to Venice, Rome, Siena). My second year we had a hack week in Maui, and we just recently came back from Puerto Vallarta. These trips were awesome, but ostensibly still “work trips.”

The Good: Running a Team / Teaching

When I joined the company, most people were either flash engineers or server-side engineers. There were certainly other full-stack devs on board (most of the dev leads, in fact), but there wasn’t really anything pushing people to do both. On my teams I tried to push everyone into learning ruby and actionscript (as applicable). Ruby is fun and weird and awesome, and actionscript is easy if you’ve ever done any other dot-syntax-based-language ever. Having a developer who can implement an entire feature end-to-end just saves so much more time and stress over having to have two people agree on a spec and implement in parallel (for small stuff anyway, obviously that doesn’t scale).

There were some exceptions–we had a few truly truly great flash guys where it actually was faster to keep them in Flash and let other people support them. Even those guys, though, probably got more exposure to ruby than they would’ve liked (anyone can use the console, right?). That sort of education and guidance (mostly on the ruby side, but even in Flash) was the most rewarding part of being a dev lead (or even a producer) to me.

Even being a dev lead itself is pretty fun. Once a team gels pretty well, you get a pretty good sense for what each team member is capable of (or capable of being pushed into). I found the whole thing to be kind of like a puzzle–given the following pieces, how could I best match the tasks to the people to play on all of their strengths and push them forward? Maybe it’s because my mom is a teacher, but there’s something really satisfying about developing other people around you.

Once we shifted to mobile, though, I switched more onto product/design/project management and less of a lead developer–not having the really really deep knowledge of C++ and mobile that I had on the Flash side, I never had the confidence to try and be a leader there. That freed me up in some ways to focus more on design and product, but it was also pretty frustrating at times.

The Bad: Running a Team / Not Doing it Yourself / Not Doing Anything

While teaching was fun, there were parts of being a dev lead and product lead that I wasn’t so well suited for. I think I’m probably a horrible micro-manager. I can fit the whole product in my head at once, but I’m not so good at writing all that stuff down into a useful document and delegating. Because there’s no central repository for how things should work, people have to keep asking me how it would work. Some people will just go for it (which is sometimes awesome and sometimes not, but generally preferable). I ended up having a finger in every pie, but not really feeling like I had full control over any of the pies (I mean–I knew what was going on in all of them, but didn’t have enough time to fully understand any one pie).

Because I keep the whole product in my head, I have a pretty good sense of how I want everything to work (both at an experience level and technically). Trusting someone else to do something “right”, where “right” means how I would have done it, is next to impossible. The trick, which I was getting better at but never fully mastered, is to just let your people figure it out and make mistakes as they go. Back in the It Girl days, Pete Hawley (now at Red Robot) told me that the crux of it was basically to hire really good people you could trust and then get the fuck out of their way. I think I could get there eventually, but for now I’m still too close to the development and the architecture side of things. It would grate on me at times when looking at code people had checked in because that’s now how I would’ve done it–but most of the time it would be a perfectly reasonable solution.

At other times on mobile, I felt like I had nothing to do at all. As a producer, though, that’s the state you should strive to be in–if you have nothing to do, that means you’ve got your whole team right where they should be doing what they need to do. As we got closer and closer to launching Closet Wars I was able to jump on bugs and help fix things.  Bug-fixing, which can feel like more of a junior task sometimes, is actually a great thing for a used-to-be-programmer-but-now-manager to help out with. Fixing bugs is much much easier than adding features, as it mostly involves tracing logic errors and correcting them instead of architecting whole systems. Because I might be called away at any time if a fire came up, it wouldn’t be fair to the devs for me to take on a big feature–more often than not they’d end up stuck waiting for me to finish it.

The Good: Talented Artists

I saved this one for last, because it’s probably the most important. My background is mostly in making heavily procedural, art-light games–mainly because I suck at art and work mostly alone. I remember having a conversation on a Flash forum before I joined with an artist who was looking to partner up with a developer and split everything 50/50–my theory at the time was basically that art was a commodity resource and I’d never give up more than 10 or 20% for an artist. I should find that dude and apologize.

The absolute coolest thing about working at Crowdstar was working with the amazing artists there. Our art director had incredibly high standards for artists. It could be frustrating at times when we needed a body in the door for UI or other art positions (I remember interviewing a couple that I was blown away by that he said “meh” to). In the end, though, it definitely shows in the games. It helps that the games we built were essentially content-delivery vehicles (lots of digital goods and weekly releases), but I don’t think I would’ve stayed nearly as long as I did without some of the artists we had.

For nearly three years straight, I’d constantly get blown away by the things they came up with. You’d think after creating 5,000 garments for It Girl there would be absolutely nothing left in the tanks creatively, but I think the artists both on It Girl and Closet Wars (now Top Stylist) got stronger and stronger as they went. The stuff those guys churned out on a regular basis was just fun to watch.

Coming from a place where I wanted to make games but didn’t know any artists, I always remember being frustrated at how I was going to get around it… art was a barrier that prevented me from making what I wanted to make. Now, I have a pretty large network of artist friends who are willing to give me recommendations to their artist friends (or take on contract work themselves). And having worked closely with really really good artists for several years in a row, I feel like I can communicate a lot better with artists when I do bring them on board. Just because of that, I feel like I’m no longer limited in the kinds of games I want to make (which I think is huge!).

The Bad: The Breakup

I debated whether to include this section or not–I’d say my three years at Crowdstar was more or less 33 months of good times and three months of sour grapes. I worked on Closet Wars from November of 2011 through around October of last year, when the decision was made that it wasn’t hitting it’s performance metrics fast enough.

Even before the axe fell, though, cracks were starting to form. There had been a round of layoffs in the summer (which mostly unphased me, because I was having so much fun on Closet Wars), which spooked a lot of our best people into finding new jobs. My team mostly held together through the worst times, but by September or so we’d lost our lead artist, our UI artist, and were about to lose one of our best engineers. Zynga’s stock was in the shitter, which was making the whole industry look bad. I considered leaving then, but Peter and Jeff talked me into staying through the end of the year and seeing Closet Wars through.

A couple of weeks after I agreed to stay on, the decision came down to scrap Closet Wars and repurpose it into Top Stylist. A new game designer was brought on, and I was asked if I was willing to produce the hatchet job (that’s probably too harsh–Top Stylist is a pretty well designed game, it’s just not my baby). I’d only taken two weeks off when my son was born, and in California you can take up to 8 weeks of paid family leave. I thought a better solution would be to take a month off, come back refreshed, and then figure out where I could be helpful.

When I got back in early November, everything seemed different. I’m not sure if it was the month off or not, but it was almost like they didn’t know what to do with me. I was given the vague assignment of researching new tech. Mostly, though, I was just ignored. Not being on a team and not being assigned anything meant that most days I just sat there and did nothing. That’s fun for like a day, but I had just had four weeks off and I was raring to get to work on something. Small things would come up–ruby questions, questions about how we’d built things or how the art pipeline worked–but certainly not a full day’s work. It almost felt like a game of chicken, waiting to see which would happen first: getting fired or quitting.

I was travelling for a lot of December and January, and upon returning let Jeff know I was ready to leave. He asked if I’d stay on and finish one last project, but in the end the pieces for that project didn’t quite come together quickly enough and we kind of synched up and decided to call it a day (or three years!). In the end it’s kind of a silly way to go out, but I was ready to do my own thing and Crowdstar is a much different company than the one I joined three years ago. No longer really a “growth” startup, they’re in full-on game studio mode. They have a clear direction they’re headed and need people to fit into pretty specific roles to get them there. If I’d stayed, I probably would’ve dropped back down to being just a “server dev” or a “client dev.”

The only thing I really regret is that I was never able to do anything with all those stock options (which will just expire in 90 days). I could exercise them, but locking up that much cash in a studio that’s raised a ton of VC, over which I no longer have any influence, and which has no guarantee of an exit seems pretty silly when I could just invest that cash in myself and my own projects. It’s a little crazy to think the thing that acted as such a good carrot during the long hours of the It Girl days will come to nothing, but that’s just part of the startup game.

The last few months were a little strange, but overall I think it was a great decision to jump on when I did. It’s easy to wonder what would’ve happened if I’d stayed the course on doing indie iphone games back in 2010. Comparing what I know now about game design, free-to-play, art, operating a game as a business, even technology–I still love some of those old designs I was working on, but they probably would’ve been disasters. I’m really excited to get a “second chance” at some of those designs, but revisiting them through the lens of my Crowdstar experience. It should be a fun ride!

, , ,

No Comments

Thoughts on Hundreds

Sensational Headline

Several people have sent me emails and messages now that Hundreds is out, ranging from “reminds me of Filler” to “dude, they cloned your game” to “it even looks like Filler.” I’ve been casually following its development, so I finally downloaded it over the weekend and gave it a play.

My thoughts generally fall along two lines–an analysis of the game as a game and its similarities to Filler, and my thoughts on the game as indie theater (is that a thing?).

In general, though, I like Hundreds. Quite a bit. It’s pretty, it’s hard, and I clearly like the mechanics.

Similarity to Filler (Inflation Mechanic?)

To be honest, Filler is barely a game. I read an article on a new physics engine for Flash back in the fall of 2007 (APE – Actionscript Physics Engine), downloaded it, and started playing with it. I wanted to see how the circle particles interacted with each other and how many it could handle, so I made a little thing that let me mouse down to start growing a circle particle and release to drop it.

Something in that toy sparked a memory of Jezzball from years ago, and within an hour I had Filler more or less running in its final form. I refined it a little over the next couple of months (instead of growing the radius linearly, the balls “inflate” proportionally to their area/”volume” instead–they grow quickly at first and then slow down), but the shipped version was basically that prototype with some menus and ads.

Hundreds uses that same mechanic–inflating a ball–but to different purpose. Where Filler was an arcade game with a neverending level progression, Hundreds is a puzzle game with specific, authored puzzles. This works really well for the inflation mechanic, as it solves the one major problem I never really solved with Filler–lots of particles leads to lots of collisions which leads to slowdown. By introducing a finite number of particles in each level, the lag issue goes away entirely.

Hundreds also introduces many new types of particles where Filler had only the inflation-balls and the bouncing pop-balls. Most of the particles are pretty neat (I like the hockey pucks that you can slide around, but don’t much care for the chained-together-balls). For me so far (about 60 puzzles in), the puzzles generally fall into two categories: acrobatics and patience. In the acrobatic levels, there’s usually some sort of dexterous action required–batting a razor particle through a small opening, moving the pucks around to block razor particles, using two-finger controls to make quick arrangements. In the patience levels, it’s usually a matter of slow scanning, waiting for an opening and deftly inflating just a little. The former feel like totally new puzzles to me, but no matter the particle complexity the patience levels end up feeling just like Filler to me.

Which is fine.

I like Filler, and I like that same feeling in Hundreds. I certainly don’t own the inflation mechanic or even the combination of patience and inflation. I like the mechanic, so I actually wouldn’t mind seeing more games that use it. Over the years I’ve sketched/prototyped out a few such games that I’ve never gotten around to making:

  • a Filler-style tower defense game where you drop particles onto creeps, who crawl over the rubble at the bottom (prototyped–it’s pretty neat)
  • a bubble golf game — imagine a pinball layout (wedges/ramps) with several “air” catches that you’re trying to fill up. kind of like inverse pachinko, but you can control how big your ball is (sketched)
  • a bubble platformer — a little mario guy who runs around collecting bubbles on the sea floor who can “inflate” as a shield against danger… but also collecting too many bubbles makes you jump too high (sketched)

There’s probably a lot more!

In terms of art style, Filler doesn’t really have one. 0×555555 isn’t an art style, it’s just a color of gray that I like. The original Filler is minimalist in style because I wrote in entirely in code and didn’t own Photoshop or Flash at the time (which I used profits from Filler to buy). Filler 2 isn’t really any different than Filler (other than the challenges) — it’s more of a “here’s what it would have looked like if I’d had the resources the first time around.” I think Hundreds is a bit obtuse in places (I thought the game was broken on the first frozen particle), but in generally I really dig the colors and design aesthetic.

Hundreds as Indie Theater  ($2.99???)

So now that I’ve established that I like Hundreds… I don’t think it’s worth $2.99. “Worth” is a really strange word, though–something is worth whatever someone is willing to pay for it, and clearly people are willing to pay $2.99 for Hundreds.

But they’re not really paying $2.99 for the game Hundreds. They’re paying for a ticket to the indie theater.

Adam Saltsman (Canabalt, Old Spice Saves the World) and Greg Wohlwend (Solipskier, Gasketball) are fairly well known in the indie game scene, and they’ve done a great job stirring up attention before the release of Hundreds. Buying the game achieves two goals–it gets you the game Hundreds and it acts as a vote of support for its creators to keep doing what they’re doing. In that sense, this is another form of Kickstarter. We want there to be successful indie developers so that we have something to strive for, and supporting games like Hundreds lets us tag along for that ride and keep our own dreams alive. In that sense, it’s absolutely worth paying $2.99 for.

But tweets like this are probably a disservice to aspiring game developers:

"one might be forced to assume that there is in fact a large audience of people willing to pay more than less than a dollar for ok stuff", A. Saltsman

This exact same game by an unknown at $2.99 would be a disaster. Probably even at $0.99. By building a large social and media following and hyping its release well ahead of the launch day, they’ve managed to do something that most indies can’t–widespread press coverage, Apple featuring, and enough initial downloads to launch into the Top 10.

The message that’s being sold in the media is simple: “build a high quality original game and you can command a higher price point and be successful.” But I think Hundreds is more of a marketing success than a truly innovative and original game (not that it can’t be all three–I just think if you’re going to hold it up on a pedestal the most interesting piece is the marketing, by far). Certainly, the game could’ve been a dud and wasted all the hype leading up to its launch. A more accurate story might be “build a large internet following and you can successfully sell content to them (see Louis C.K.).” Game developers should always strive to build good games, but the takeaway from Hundreds should be how to launch a successful game and not necessarily how to design/develop a successful game.

I’m curious to see what will happen when the price ratchets up to $4.99 on the January 10th. I think it may be able to hold its chart position on tablets, but will probably drop like a stone on smaller devices. Which is actually kind of a shame, because I do think it’s a good game.

, , ,

No Comments

Filler for XBox and More Thoughts on Indie Games


The night before hopping on a cruise last month, Filler finally went live for the XBox (September 13th). Coming up on almost a month of data, I wanted to take a look back at the XBox version’s development process and comment on Microsoft’s Indie Games in general. I actually had a solid prototype working on the XBox over a year ago, before the service even launched. I had way too many other projects on my plate, so the XBox version kept getting pushed to the back. After finishing up the Batman: Brave and the Bold contest game in early July (I’ll do a post-mortem on that one as soon as I get a thumbs up), I felt it was finally time to clear the XBox version from my to-do list.

Finishing Up

From each time I’d worked on the game for a week or two at a time, there was quite a bit of code in place. I’d completely stripped down APE (Actionscript Physics Engine) and the original Filler codebase to be as abstract as possible, then moved that code directly into XNA. I won’t say I got it 100% MVC, but it was pretty damn close. The majority of work, then, was all about building up the actual display and menus for the XBox version. Building the interfaces was pretty easy–though the lack of an interface builder means it’s a lot of “tweak, compile, check, and repeat”–not so dissimilar from coding Flash games in pure AS3.

Having played tons of Indie Games for WorthThePoints, I also had a pretty good list of things about the service (and the XBox 360 in general) that bugged me. Top among that is the prevailing idea that one profile directly maps to one human player. While I know why they did it that way (more Gold accounts), I personally think the profile system in the XBox is terrible for multi-user households. Content is attached to profiles instead of machines, which means we’re always logging different profiles in and out to figure out which profile has the proper permissions or features unlocked in a given game (that’s directed at you, Rock Band). Because Indie Games are even more rigorously locked to single profiles, I wanted to build my own system which would function more like the “save” slots of the old SNES and Playstation games I used to play.


Besides adding a co-op multiplayer component, I really didn’t want to just release “another Filler” for the XBox. The original Filler was only tuned through about level 15 (the highest I thought anyone would make it), so I was totally unprepared when people were up in the 60′s and 70′s a couple of days after I released it. Besides the physics breaking down, the difficulty of the game itself completely plateaued past around Level 15. Both the number of lives and the number of balls allowed spiraled way out of proportion to how many were actually needed to beat a given level. For Filler 2, I tried to correct this by combining the lives and balls into one value.  The problem with this approach was that experienced players need way less balls than new players. Starting the game with 10 balls was reasonable for a first time player, but it gave expert players such a huge cushion to work with that there was never really any danger in losing.

The limit on how many balls could be placed was mostly put in place to put players from “cheating” and simply filling up the board with hundreds of tiny balls. To move towards making that the core win/lose state, then, was promoting a feature that was only added for convenience’s sake. For the XBox version, then, I wanted to get back to the notion of lives while thinking of a better solution to limit how many balls could be placed on screen. I tried lots of things: a power meter that depleted, holes in the floor that let small balls slip out, special enemy balls with a variety of powers (including popping small balls).

While playing with the notion of special enemy balls, I actually stumbled onto a pretty fun game mode which I may code up and release as a standalone version at some point. One of these special balls had erosive powers–every time it bounced into a filler ball, that ball would shrink a little bit. It was this mechanic which finally provided the difficulty curve I was looking for–though not as originally implemented. By making all the filler balls shrink at a constant rate (like filling, the “shrink” rate is proportional to the area instead of the radius, which means big balls shrink slowly and small balls shrink quickly), I actually solved three problems for the price of one.

The “fill the board with small balls” strategy became worthless–before the screen is even half way full, the first balls dropped in this manner start to “pop” and take lives away. Second, even when playing in the “proper” manner the new shrink rate adds a time-crunch component which was missing from the first two versions of the game. Most importantly, the shrinking balls smooth out the difficulty curve–they’re practically a nonfactor in first ten levels and don’t really start to have a major impact until around level 15… where the difficulty was previously plateauing. From a pure gameplay point of view, then, I firmly believe the XBox version of Filler is the most well balanced version to date. If I ever do another Flash version of the “core” game (and not a spin-off), it’ll be nice to have that part of the equation “solved.”

Peer Review

I submitted Filler to peer review for the first time on August 8th. It would go on to fail twice–both more for quirks in the framework than what I’d consider bugs on my part. The SpriteFont class takes an optional parameter when instantiating it, a single character which is used in the event that your code tells the SpriteFont to render a character which isn’t contained in that font (i.e. a character with an accent on it). Rather than doing something sensible like defaulting to a space (or simply rendering nothing), a call to render one of these characters causes the whole game to crash. Again, this is something that’s part of the XNA framework and not something inherent to C# itself. The fix? Adding a ‘-’ to the initializer for my fonts, which took about 30 seconds. The penalty? Besides the five days it spent in review before the crash was found, it had to spend another eight days in review “jail.” The published cooldown for a rejection from peer review is seven days, but your jail time doesn’t start until the day after you pull it from peer review (or get rejected, I assume). So if you pull it at 5 AM, it’s practically an 8-day wait.

After resubmitting the game on August 21, it took only a day for another crash to show up. If you try to open the marketplace menu with a silver account (i.e. a non-paying XBox Live account), the game crashes. All the accounts on my box are Gold, so I never even thought to create another one to test from. For a framework that does its best to handle all of the transactional details for you (trial modes are automatic, purchases are handled completely outside the game), it boggles my mind that the framework itself doesn’t pop up an alert with something to the effect of “You must be a Gold member to access this content.” The fix? Testing to see if the acting player is Gold or not and popping up an alert if they’re not–about five minutes of coding. The penalty? Another 8 days in peer review jail.

I’m not trying to say I’m completely without blame–if I’d spent another couple of weeks reading every detail of the framework documentation I might’ve been able to find these crashes before submitting. And I don’t think the “7-day” jail is a bad thing. The idea of it–to prevent people from dumping buggy games onto the service with no QA is noble, but I think it does more harm to well-intentioned developers than the “lazy” developers it was meant to stop. A better system might take into account the severity of the bugs and the submission history of the developer.

I was traveling when Filler failed the second time, so I was unable to submit again until September 2nd. Right around the time I submitted it for the last time, I posted some of my thoughts on the review process to the XNA forums:

What are essentially two framework quirks have cost the game nearly three weeks (it usually takes a few days before anyone bothers to review). These are mistakes I’ll know to look for next time (if I choose to continue developing for XNA), but the end result is that the submission process is unnecessarily cumbersome to first-time XNA developers–the very people Microsoft should be bending over backwards to court! I hope that there’s enough backlog some day for MSFT to actually hire someone to wade through it–a community relations job or a full-time tester.

This may sound douchey, but I have absolutely no interest whatsoever in playtesting or reviewing a game that hasn’t come out yet. I honestly believe I add more value to XBLIG by spending my time on development of new games and reviewing published games than by wading through unpublished games. Zman mentioned how few people are trying to make XBLIG a real business, and that’s because no one expects EA to playtest Call of Duty before it’s released (especially Activision!). You do your best to make your game, get it bug free, submit it to MSFT. If there are bugs you release patches.

Those sentiments were met with nearly universal scorn by the “main players” in the forum, but I stand by them 100%. At times it seems as though the Microsoft employees and most of the MVPs which frequent the forums (though certainly not all) have had rose-colored glasses permanently attached to their heads. There seems to be an overwhelming belief that if they hold hands and sing “Kumbaya” for long enough that the community will magically come together in a supporting environment where everyone does peer reviews. By voicing my opinions, though, the most common response was “don’t expect me to ever review one of your games. Good luck getting anything released.”

They claim that there are legal reasons Microsoft itself can’t approve apps, but that argument just doesn’t hold up–Apple has approved almost 80,000 apps since the XBox Indie Games channel launched (last November), and all without peer review. After pretty much giving up on ever getting the game out, some more reviewers checked it out while I was on the east coast and it finally got approved on September 13th–over a month after “finishing” the game.


Sales so far have been pretty tepid, which hasn’t been too surprising given other games’ lackluster sales. Though there are still a few days left, it looks like the game will sell less than 100 copies in its first full month (currently sitting at 83 with 4 days left). More disappointing than the actual sales, though, are the trial downloads: in almost a full month, only 2,520 trial copies were downloaded. 2,000 of those were in the first three days (while on the New Release list). In the last week, daily downloads have ranged from 2-6 trials per day. I’ve stuck with it for a couple of years now, so clearly I’m a fan of Filler’s gameplay–but I certainly don’t think it’s the greatest game out there. Still, it’s rated in the top 20% of all games on the service (at least, last time I checked). For sales to fall so flat so soon tells me two things:

  • Though I considered the work I put into the challenges, multiplayer, and profile systems enough to bump the game up from a $1 game to a $3 game, I get the sense that players (and customers) don’t actually give a crap about those features and would rather just have the core gameplay for $1.
  • As an ecosystem, the Indie Games Channel only has enough traffic to support the top 25 or so games in the system. The iPhone has enough users that even an “average” game can see several purchases per day, whereas an “average” game for the XBox will be lucky to get 2-3 trial downloads per day.

The breakout success of “I MAED A GAM3 W1TH Z0MB1ES!!!1″ has shown that there’s a place for content without any “extra” features at $1. If I had to go back and do it again, I would’ve just released the game without profiles and challenges months ago for the lowest price point. Extra work on the game doesn’t really get you anything on the Indie Game Channel (versus what we’re seeing in “Premium” flash games that can command user payments), so if I release more games for the platform they’ll definitely fall within that $1 “throwaway” style game.

, , ,