Convention and Formalization

The first time I really thought about conventions was about a year into my first programming job.  I had a coworker and good friend, Victor, who was looking into our process for adding new power-ups to the game.  He asked me to write up a document indicating what needed to be done just to duplicate an existing power-up and put some skeleton code in so we could use it in game.  I came back with a list of 14 steps that needed to be done, with some nasty gotchas involving human error.

Fourteen steps just to get started.

This was unacceptable; it might take a diligent person several hours or even a day to kick off development on a new power-up.  And this was a mobile game, we were intending to make power-ups every couple weeks until the end of time.  Starting a new one should be one click, or maybe a few simple things.  What was wrong with our process?

Well, there were a lot of things wrong, some we never ended up fully solving, some deep mistakes in the engine.  But Victor quickly saw a few easy wins.

All our power-ups had plenty of similarities.  They all used some menu and in-game images, some animations, and the same loading process in code.  However, nothing was assumed.  Every asset had to have its path written into code, an intermediate file generation step implemented, and code to add it to the menu list.  It was a nightmare of tediousness and copy-pasting.  We saw that some code needed to behave generically, and that reduced a couple steps.  But could we also generalize our data pipeline?

Victor pointed out something interesting, that all our assets used the same folder path structure and same naming conventions.  This was natural because people did the easiest thing when cloning a power-up, they copy-pasted and renamed quickly and simply.  However, this also meant that if we made an assumption that files would continue to be named the same way, we could in fact delete a couple steps.  We didn't need to put the asset names into our engine, we could derive those from the runtime id.  We didn't need to define folder paths explicitly, they all matched the power-up names. 

These were conventions, and furthermore they were conventions that no one had mandated, they came about organically, and the pattern recognition came second.  This was the opposite direction of most code problems I had encountered, where a convention was defined by the initial implementation and enforced by the compiler.  An object had to implement an interface because the compiler ensured that it did, and threw an error if it didn't.  But data doesn't have a compiler, right?

Then Victor told me that in fact, we could have checks like a compiler, beyond the informal conventions of our artists.  Sure, it's not always possible to prevent someone from moving or renaming their assets.  But if we used some tools we could virtually eliminate human error.

The first tool was verification.  In our toolchain, or at runtime, or both, we could check for the files we expected, and if they were missing, give obvious and clear warnings to the user that it needed fixing. Toolchain errors were useful because they were usually at the time and place that the error occurred, so they were most likely to be seen by the right person.  Runtime debug errors were useful because anyone playing the game would complain about the errors, and things would get fixed quickly.  

The second tool was automated correction.  If a file didn't follow our convention but had a common mistake, we probably didn't need a human to fix it.  We could give a notification message, but automatically rename a misspelled file, search in nearby directories, or at the very least, reduce the amount of work that a human needed to do by giving them a single prompt, confirming that a file should be renamed, or generated, or whatever else had been missed.

The third tool was setup automation, where the engineer didn't have to be a part of the process at all.  Maybe artists could drag and drop a folder into a tool, and the tool would take care of everything.  Tools don't have to just deal with assets, they can also generate code.  If the tool knows the file, we could generate the C++ code required to link an asset to the engine, even if the engine didn't support dynamic assets.

All these steps together formalized some simple conventions we had in our naming schemes.  It took an existing human understood concept and made it a machine understood concept, and it allowed us to automate things that weren't spelled out in code.  The formalization of conventions is useful all over the place to add value and prevent human error.  It's pretty clear to me now how fundamental this is, how a compiler is a formalization of language into computer terms, and how tools and runtime code can extend that formalization even further.

Perhaps even more interesting to me now is the identification that non-engineering members of the team can make it obvious what conventions can be eligible and useful to formalize.  By looking at their process, their naming conventions, and their points of tediousness, an engineer working on tools can find clear, easy wins.  We ended up cutting half our power-up steps through formalization, and now it's a first class resident in my refactoring toolkit.