Software as Prose

What makes a good software implementation? What makes this so hard to define? Although computer software "engineering" and hardware engineering have been around for approximately the same amount of time, the design of computer hardware for some reason has reached the level of definition and discipline that characterize most engineering professions. Software, on the other hand, is still practiced much more informally, and although there has been progress through the years in the theories of computer science and the practice of software "engineering", the typical implementation of software rarely invokes the same level of definition and discipline that other engineering professionals would indeed call "engineering."

I speculate that this is because the writing of software is so accessible. Given the wide variety of languages and tools available, just about anyone who can write a paragraph in their own natural language can also write a simple program in some programming language. And just as the range of literary quality varies in the writing of prose, so does the range of software quality range in the writing of code. In fact, in attempting to describe "good code", "good design", and "good architecture", it is useful to draw parallels to prose writing, as most people have read a good fiction book at least once in their lives. (By "people", I mean "meeting the minimum requirements as a sentient organism").

So what makes a good story? There are conceptual layers of any good story, and some general rules for the execution of those layers that most writers are taught. The truly great writers, of course, carry beyond these concepts and rules to something truly superlative. The true artists are able to implement these concepts almost intuitively. Not everyone is a great artist. (Witness, for example, this dreadfully dull tome.) But there is still plenty of good reading out there, and it all has some characteristics in common.

At the lowest level, of course, is the use of language. Knowledge of syntax is a minimum requirement - sentences such as "Drinked bar him goed" make little sense. Along with basic syntax must come some basic sentence design to be effective - sentences such as "He at bar the went get some drinking," while possibly parseable, certainly is hard to read and doesn't accomplish the purpose of communicating what happened as well as "He went to the bar to get a drink." But syntax isn't nearly enough - even an expressive sentence like "Parched, he sauntered up to the bar, rested the weight of his arms on the cool marble countertop, and signaled the bartender for his heart's desire - a cold, frosty mug of the brewery's finest," doesn't carry the reader very far without more context.

An understanding of the next concept of writing structure, the paragraph, allows a writer to group related sentences together into conceptual chunks. These chunks help the reader develop and maintain an organization of concepts above the sentence level. This chunking exploits a fact of human short term memory. Short term memory typically can hold up to seven items at a time. The items could be digits - say, the random set of 9372839. However, it is well known that by "chunking" the data into a larger organizing structure, humans can remember more individual items (although not more than 7 "chunks"). An example of this is a telephone number - when "chunked" into area code, prefix, and extension, we can easilty remember 10 digits. This same concepts applies to paragraphs. By "chunking" sentences together, we can essentially keep the concepts conveyed by more than seven sentences at a time. This chunking allows the author to create a richer context in which each new sentence takes place. You can relate each new sentence to everything that has happened in the prior 7 paragraphs or so at a sufficient level of detail to draw connections and relationships to what has come before, therefore creating a richer context for the tale. Chapters, of course, continue this conceptual chunking. While the level of detail of each layer of "chunk" grows less, the organizing framework allows the human mind to contain a level of context at each layer that aids in the understanding at each layer.

Most programmers have by now recognized some parallels to the way programs are written. Command lines, loops, function/method definitions, subroutines/classes - these are the ways in which a programmer "chunks" software into conceptual layers so that the brain can handle the necessary context and detail at the appropriate layer of abstraction.

Students are taught to write at this level, and most can and do. Most can write a rudimentary essay, and every now and then insert an elegant turn of phrase that can be admired. Of course, stories that have any staying power have a bit more than those elements listed above, don't they? Most authors who write for a living learn over time to master the other elements of good prose - or they find a new profession (or starve). Many people who start programming can also generate a working bit of code in fairly short order - there are a large number of working VB applications in the world. (This isn't meant to imply the use of VB means that the code quality is poor - writers both good and bad use the english language). It is how a language is used, combined with the other elements we explore below, that separates the pros from the hacks.

What are some of these additional elements of good stories? They're numerous, and this isn't an english course, but here are some examples. Good stories have characters, carefully named, with a just as carefully planned depth and scope. There is a plot, or in the better prose, a number of subplots that exist somewhat independently, but intersect with each other in carefully planned ways and times. Even better stories have a well thought out "backstory", an assumed series of events that happened prior to this story that helps give it context. In works that take place in other places and times than ours, there needs to be a carefully thought out framework in which the story takes place - "world building" it is called in fiction writing, and it extrapolates a social, political, physical, and temporal framework in which the characters and plot take place. A well conceived world has an "internal consistency", in which there aren't any inherent or glaring contradictions in the way the elements of the world interact. For instance, if you assume that the story takes place on an airless world, then a lack of airlocks, or someone firing weapons that would penetrate walls - these create internal inconsistencies. It just couldn't work like that if the story were "real" and consistent. Just so, the behaviors of well conceived characters are internally consistent - someone who has never exercised in his life can't all of a sudden show martial arts prowess, or go running a few miles for help. Some part of the readers brain cries out in pain at the inconsistency with known models of the world.

There are other levels of conscious design in a story that raise it above a poorly written essay such as this one. There is an understanding of who will be reading it, and the effect certain words or phrases will have on their ability to understand and the emotions they can generate. There is a commercial context, in which perhaps the key characters remain and a potential follow-on story suggests itself in the last paragraph which leaves open the possibility of a sequel. There is a timelessness, in which certain popular expressions or pop culture of the day is limited so as not to rapidly date the story.

So too it is with software. There is the design of data entities or object, carefully named, with a carefully designed depth and scope. There is the plotting of chapters (code modules) into a story (product). A more complex design consists of separate components that are loosely coupled, with clear interface points. There is a "world" or framework in which each of these elements exist and interact in a consistent way and at predictable times. There aren't glaring inconsistencies in how UIs or algorithmic logic is implemented in one component versus another. There is an understanding of the team that will need to read it and maintain it, and a code style that lends itself to clarity for the reader. There is a commercial context where tradeoffs of costs are made - buy vs. build, design complexity vs. run-time complexity. There is a timelessness, where certain popular tools or popular techniques of the day are limited so as to not rapidly render the system rapidly out of date.

These elements are what I mean by "architecture." And just as good writers have managed to extract the components of good writing over the years so as to be able to teach it at writer's workshops, so software engineers are starting to extract the components of good architecture so that future software engineers might design better products. I'll be pointing to some examples of these in the coming weeks.