Architecture is a Democracy
Today I'm going to throw out some thoughts I've been having about software architecture. Why is architecture so interesting? It's the land of the dreamer, the big picture thinker, the mountain mover. It's implications are great, and all responsibility is held by the one, the chief architect. He or she crafts the perfect blueprint while all other coders scramble to construct the low coupling, high cohesion brilliance. It takes a genius architect to solve all the high level problems so the rest of the team can work like clockwork to divulge the mythical perfect project. We all grind day to day so one day we get promoted to that lofty position and steer our own grand ship.
Wake up, Alex, you're flashing back again to old college dreams. The glorious picture I painted hasn't really panned out. What? No, the architects aren't gods in the sky looking down on the rest of the organization. They don't make perfect blueprints that withstand any unknowns, and they don't bless us mere mortals with abstractions that miraculously clear away the grime of day to day code. Actually, the reality is that architecture isn't an oligarchy, it's a democracy, and that's pretty nice for us mere mortals who want to contribute.
I have several propositions that have come out from my expectations versus reality. The first is:
Architecture is everyone's job.
Everyone has both the responsibility and the power to affect the architecture of their project, and architecture permeates from the highest to lowest levels of code. Architecture is making decisions that have implications on related subsets of a codebase. It's what defines the program flow, the management of complexity, and the understandability of components.
Common decisions we must make involve bug reduction, performance, readability, consistency, maintainability, and plenty more. Guess what? Every line of a program is related, by the definition of a program. However, we are able to model complexity mentally by slicing our procedures any number of ways, be it an api, a feature, a function, a paradigm, a library, a standardized convention, an imperative instruction sequence, or the whole damn thing.
Speaking of the whole damn thing, I like to think of a program as an entire universe with constraints, as opposed to something built from nothing. At the first line of development, any direction is possible, any program, any idea is a ray shooting out infinitely from the origin. I visualize the possibilities as a sun radiating out with incredible brightness. As code is written, the possibilities narrow, until we have an area where our implementations become extremely easy to introduce, but other directions are just as hard to implement as from the start, or in fact even harder. The light has become focused in effect. I find this viewpoint really important for how decisions are made with code.
Architectural decisions always both enable and limit.
It's not just a force that allows any possibility to be implemented with ease. It's a series of decisions, trade offs, and preventative measures, in addition to the goal of making some things easier, more efficient, or better by some other metric, in the long run. There is no way to doing so without reducing flexibility in other areas, and architectural decisions complement each other differently. There is no blueprint for a game engine that can do anything or an AllWebsitesMadeEasy.js framework. There's also no universal web protocol or GPU design.
There is no optimal architecture for any set of requirements or scope.
There is an endless list of unknowns in software: missing requirements in pre-production, shifting requirements over development, shifting requirements after release, the innate inability of humans to understand the full problem space of a non-trivial program, mistakes, bugs, imperfections of collaboration, and so forth. In the game industry, where I spend my time, the user always has the final say and we don't know our users until we give them something to play with, so how can we make our code resilient to any possible feedback?
As an architecture approaches some theoretical exact design, it doesn't become more useable and extensible, it becomes uncompromisingly brittle. Remember, every program has a human element to it (even in programs written by programs), and humans are imperfect. One mistake in foresight or constraint satisfaction and the house of brittle cards come crashing down. Architecture in those cases make it harder, not easier, to satisfy requirements. The best decision-makers have foresight to make code flexibly where it should be and constraining where it shouldn't. At the end of the day though, we have to accept that in some places we will be right, and in others, we will rewrite.
A note. Beware the full system rewrite! The one who has the inclination to undergo such an endevour is often cursed to replace a rickety house with another rickety house. Poor architects are quite keen to throw the baby out with the bathwater, so just like the rest of the decisions I've mentioned, think long and hard about this one.
A solid codebase is built in increments.
Why do good architects not throw out their programs? Foresight on flexibility is a factor I've mentioned, but incremental refactoring is it's close relative. Code suffers from entropy just like everything else, and if we ignore that which is already written, technical debt WILL accrue. We should be willing to rewrite code often, to the point where it goes hand in hand with adding new features. The reasons are abounding. The approach removes the burden of perfection, allowing a system to be written simply and refactored later. It protects our programs from buckling under their own weight at inconvenient times, like before a due date or at the introduction of an ambitious, unexpected feature. It improves our understanding of our and other engineers' systems. It gives us the opportunity to apply things we learn to old problems.
Your languages, frameworks, and tools constrain your architecture.
Maybe I was a little overzealous by claiming an empty project is an endless universe of potential. By the time we've written our first line of code, we've already made some implicit architectural decisions. It's obvious, but worthy of reminder.
Although, it's not so obvious in existing systems as it is for new ones. Not evaluating APIs sufficiently before committing to them in a project already in development is something I've seen firsthand, and it's a problem. When there is a choice, or in the common case of designing our own API for internal use, I think it's critical to evaluate how a component may constrain or circumvent the painstaking choices we've made in our existing application. Even when we are plugging into a specific 3rd party API like Facebook or a game services platform, it's important to think carefully how our existing code interfaces with it. Evaluating APIs is big topic but essential for good architecture, so I have to mention my favorite lecture on the topic by Casey Muratori, Designing and Evaluating Reusable Components. It definitely continues to have a significant impact on how I write and evaluate components.
"One design pattern does not an architecture make."
Similar to architecture being everyone's job, there's no simple philosophy or pattern that comes even close to removing difficulties at every level. Every application is different and nuanced. If it wasn't, us programmers would be out of a job, probably from outsourcing or robots, or maybe outsourced robots. We can use SOLID, MVVM, TDD, DI, REST, C#, Node.js, or whatever buzz word we want, but we can't claim that our architecture problems are solved from a couple high level decisions alone.
Code conventions and philosophical choices that affect every line of code can assist with consistency across large teams but they aren't the whole story. Architecture includes all the high level and low level application specific decisions we make, and a lot of those key decisions can go against things that work in other applications, or even in sections of the same program. Every rule has an exception, and software has a lot of exceptions.
I've found that a key indicator of a strong engineer is that he or she evaluates several options before coming to a conclusion, and does so from a combination of top down and bottom up design. The classic book Code Complete has an excellent section on top-down versus bottom-up, so I'll just summarize by saying I have found that a well-founded architectural decision cannot be made without observing the potential implications at all levels. The ability to think critically in solving architecture problems has to be just as important a skill as any other field. A competent scientist wouldn't publish a paper without addressing counter-arguments, a good investor wouldn't buy a stock without researching the business, and we shouldn't jump into an implementation without considering the nature of our problem.
Domain knowledge is an architect's toolkit.
During the whole critical thinking phase, we have to come up with reasonable solutions from somewhere. I've seen that most beginners can't come up with more than one or two potential solutions from scratch. They feel lost when encountering a specific problem and are unable to predict how it will scale. Worse, they often don't really understand the problem until most of the way through their implementation, or even after they have finished! The most obvious difference between a smart beginner and a smart lead or architect role is the vast expanse of domain knowledge that simply can't be manufactured through rational thought alone.
Recalling applicable design patterns, whether from a commonly available list or from past experiences, is the most straightforward way to come up with potential solutions quickly. I've found that the combination of pre-defined patterns, key observations of the problem itself, and a principled evaluation process to narrow down the best solution, have greatly strengthened my architecture and general problem solving skills. This process also can provide strong reasoning to not use a design pattern and keep things simple! I have a lot more to say about this topic, but I'll save it for an example-driven supplement on how I evaluate problems, stay tuned.
It seems like domain knowledge is the simplest but most time consuming piece of the puzzle. Coding, reading, learning from mentors, and solving hard problems are a few things that I think make it inevitable to improve.
I'm still learning what it takes to be a truly awesome architect, but the more domain knowledge I acquire, the less time I find that it takes to run through the options to see an exceptionally harmonious design choice. Even though an architectural decision may be to solve a problem for an axis or slice of a program, that intuition really makes the relationships between the axes, and thus the entire program, clearer. It's easier for example to see how the high level decisions affect the low level ones, and vice versa.
If I can make one overarching conclusion out these propositions, it's that architecture is not a lofty high level role, it's a fundamental and integral part of programming, and every engineer should believe it's his or her duty to excel at it. Forseeing design problems adds value all the way from developer time to the end user experience. Let's own our craft and make no excuses!
