Programming kept simple, part one

or, grounding and negotiating programming advice by applying the Principle of the Least Power.

Why do real life codebases quickly approach the state of disrepair? Why does most program complexity not serve to solve the problem? Why do programmers keep disagreeing with each other?

Shouldn’t software development be a solved problem after so many years? Finally, why is there a subtype of developers who write less code and whose codebases, while not perfect, maintain better clarity through project lifetime?

We are what we have been taught

Let’s take a look at the instruction we receive. Programming education and culture teaches us a number of things:

• Absolutes, like algorithms or the workings of a particular programming language or a VM, for which the more we know, the better, though some knowledge may end up never applied. This sort of knowledge has clear scopes of application.

• “Best practices”, which are not absolute, not always best, and almost always loosely-scoped. They also tend to vary wildly between languages, communities, and individuals. They like to contradict each other. Best practices can even be harmful when applied outside of their scopes.

• Showmanship, ego- or experimentation- driven practices, where “cleverness” or “aesthetic” of an implementation is a goal of its own.

We can accept the absolutes and at most argue which of them should be taught. We can reject the showmanship. This leaves us with “best practices”, a mixed bag of usefulness and harm, in other words - confusion. The isolation between the industry and the academia causes the programming community to miss out on structures capable of distilling productive guidance.

What’s good vs what sticks

Beware of the sophists. Anything can be packaged as “best practice” and advertised in blogs, videos, or books. Appealing ideas aren’t guaranteed to be correct and correct ideas aren’t always appealing. The willingness to advertise poor ideas is also inversely proportional to one’s willingness to self-critique.

Responsible instruction of a practice requires understanding and communicating the scope of its application. Responsible learning is analogous, just from the other side.

The most glaring pattern of appealing-but-wrong instruction is “let’s do everything like X”, which we’ve gone through quite a few cases of in the last decade, including:

• microservices, when some thought leaders slapped “micro” in front of the otherwise proven and well-understood concept of services (in infrastructure). I am still waiting to hear a sound rationale for the “micro” prefix.

• NoSQL, when we had an opportunity to re-learn why relational algebra was invented and that not all data is hierarchical.

• Blockchain, when some of us decided to overengineer information exchanges, most of which could be replaced with RabbitMQ + public key cryptography. So far, the only real applications were cryptocurrency systems, used for speculation, scams, money laundering, and extortions.

Another pattern could be named reductive reasoning that resonates. A perfect example of it is The Testing Pyramid, which I won’t be linking here in order to stop boosting that article’s PageRank. The original source is easy to find and so are its criticisms, which only grew over years.

1. It draws a nice picture showing a high number of unit tests at the foundations of the pyramid, followed by less common integration tests, followed by relatively few end-to-end tests. This concept is extremely easy to grasp, which contributes to its virality.

2. It assumes that test run time is the most important factor in choosing which kind of testing to prioritise. It completely ignores time spent to mock out enough dependencies to make unit tests possible. It assumes that these mocks will be correct. It doesn’t seem to realise that integration tests can also be reasonably fast.

3. It ignores reality, in which:

   • Most projects are integration-heavy, not algorithm-heavy. You ship services, not units.

   • Most non-trivial bugs happen at interactions between components.

   • Laziness exists and so do tight deadlines. Most mocks will be of poor quality.

   • Project-wide sweeping changes eventually have to happen, at which point having tests confirm that different units work together is critical.

   • Naive followers of this advice end up mocking the database, which, in correspondence to the earlier points, is a tragically bad idea.

But the damage has been done and the harmful idea circulates. Now let’s copy this pattern a couple of times: Let’s take a nice-sounding picturesque idea that we like. Let’s then present its advantages as argumentation, ignore its disadvantages, and not consult those advantages with reality. Doesn’t it look familiar? Some snarky readers may say “it looks like Substack”.

This atmosphere of chaotic advice is not a good space. It makes development decisions arbitrary and unpredictable, which in turn results in arbitrary, unpredictable, and (given multiple developers) incoherent results. Real projects cope with it by basically letting programmers produce anything not-too-outlandish and expecting others to tolerate whatever comes. This is not engineering - this is cobbling things together.

“Best practices” must be scrutinised, properly scoped, and ranked. Filling our minds with more advice won’t solve the problem of chaotic advice. If overabundance of ideas is a problem, maybe the solution is buried between those ideas? Is there a meta-idea we may have missed?

Parsimony - minimalism

Let’s examine the principle of the least power. The principle of the least power says that a solution to a problem should utilise the least powerful tool(s) available that still solve the problem well.

A given tool’s power could be defined as the the number of classes of problems that can be solved with it. In other words, the more a given tool can do, the more power it has.

Importantly, the principle of the least power does not talk about concrete actions but helps choose advice to follow. It is known to mirror the Ockham’s razor, which prohibits explanations of needless complexity, most famously in natural sciences.

When applied to the chaotic world of software development, this principle:

• makes solutions predictable by rejecting overly powerful and thus overcomplicated building blocks

• replaces constructs of personal preference with simplest constructs fit for the problem

• brings KISS and YAGNI with it

• only allows complexity that pays for itself, for example:

  • solutions that require state are the only things that use state

  • call sites that need polymorphism are the only places that do dynamic dispatch

This is starting to look like engineering. If it is so great, why aren’t we all thinking like that?

Gluttony - maximalism

I believe that the primary reason for programs consistently acquiring accidental complexity is programming language designers’ preference for powerful building blocks being available at fingertips, which also shapes the expectations of developers, resulting in a feedback mechanism.

This isn’t exactly the “when all you have is a hammer, everything looks like a nail” situation - it’s more analogous to being handed a ginormous multi-tool - the only tool at our disposal - and treating this situation and the resulting clumsiness as normal.

This multi-tool is the class keyword, though I hope to conclude this article with something more informative than a typical anti-OO rant.

How it goes wrong

Classes can do state, polymorphic dispatch, “is-a” hierarchy, interface inheritance, implementation inheritance, information hiding, and namespacing - all in one package. A class does not communicate why it is needed. It just exists, and the reason for its existence is implicit. Add poor discipline and we have what I call the Petri dish problem:

1. Programmer A creates a class because they need to implement an interface. Result: we have an interface.

2. Programmer B sees a class and carelessly adds instance variables, turning the whole thing mutable. Result: we have an imperative ball of mud.

3. Another programmer adds implementation inheritance for code reuse (because instance variables made factoring out common code into a function impossible without refactoring to turn instance variables from step 2 into arguments). Result: we have an imperative ball of mud and a nightmare of arbitrary implicit dispatch.

4. As more work is done over that bit of code, the situation only worsens. Refactoring is costly and tedious, so it doesn’t happen. Misery continues until code is removed, typically because it tends to accumulate inefficiencies around itself, forcing a rewrite.

A programming language can make scenarios like this less common by providing an explicit separation of the aforementioned features:

• state as a mutable struct or a state machine (e.g. a generator)

• static dispatch (obj.method()) as syntactic sugar for method(obj), aiding autocomplete

• polymorphic dispatch can be a property of a method, rather than a type (vtables are an implementation detail). There exist free functions which are methods, e.g. Clojure multimethods of Python’s functools.singledispatch.

• interfaces as type-level collections of function signatures

• information hiding (privacy) - property of a binding

Apart from implementing APIs like DOM, which hardcode inheritance in its spec, I have not found arguments for implementation inheritance. This use case is marginal enough to be ignored.

Conclusion and where to go next

The role of an engineer isn’t to make use of all tools at one’s disposal. It isn’t to show off their understanding of the newest tech trends. It is to solve problems with carefully choosen tools, communicate the choices well, and make others understand their work with ease.

The best time to retire the gluttonous advice of class-based programming was 20 years ago, when the results of its application were apparent. The second best time was one year later. This is the 20th best time.

What can you do now? If your code is stateless, stop writing classes. If your code is stateful, push meaningful side effects up the stack (towards the caller). Write procedures and functions, pass inputs and return outputs. Avoid internal side effects and only do mutation in well-defined points. This is worth its own article, which I may write at some point. Right now I can refer to the Simple Made Easy talk by Rich Hickey.