More isn’t just more…

Most have probably heard the saying ‘less is more‘ or know of the ‘keep it simple and stupid‘ principle – these are general and well accepted principles for design and architecture in general – and something that any software architect should aspire to. Similarly, Richard P. Gabriel (a major figure in the world of the Lisp programming language, accomplished poet and currently an IBM Distinguished Engineer) coined the term ‘Worse Is Better‘ formulating the software version of what Christopher Alexander (a ‘real’ architect famous within software engineering) might have called ‘piecemeal growth’ – e.g., start small and add later.

But what happens if we continue to add more, even if they are just small, simple pieces?

The suggestion is that more isn’t just more.

A bit of history digging reveals that, in 1962, Herbert Simon (awarded the ACM Turing Price in 1972 and Nobel Memorial Price in 1978) published The Architecture of Complexity describing the structure and properties of complex systems. He concludes that complex systems are inherently hierarchical in structure and exhibit the near-decomposability property – e.g., a complex system is essentially a system of mostly independent systems. In 1972, P.W. Anderson (indirectly) builds this line of thought in “More Is Different“. Anderson argued that an understanding of how the parts (e.g., systems) work does not equate to an understanding of how the whole works. We  can all relate to the point that biology and medicine as science can explain (to a large degree) how our bodies work, yet when all these parts are ‘combined into a human body, concepts like emotion, intelligence, and personality form parts of the ‘whole’ – distinctly not medicine or biology. In Simon’s terminology, the human body is a system of systems exhibiting the near-decomposability property – otherwise, heart transplant wouldn’t be possible.

The near-decomposability is equally a property we desire as part of software design – although better known as modularity. Software architecture is basically ‘to separate into components or basic elements’ based primarily on the basic principle of information hiding first formulated in Parnas‘s seminal paper from 1972: On the criteria to be used in decomposing systems into modules. But as Richard Gabriel argued in his essay, Design Beyond Human Abilities in 2007, there is a key difference between software modularity  and the near-decomposability property of systems of systems.

Within a software engineering context, a module is traditionally defined by what it does rather than by who produced it – the later is the definition used by Simon and Anderson.

This is a significant difference – and one we need to get used to as our software systems become increasingly complex.

Those of you who know, Conway’s law from 1968, shouldn’t be surprised though. The ‘law’ states that ‘a software system necessarily will show a congruence with the social structure of the organization that produced it“. In order words, the modular structure of a software system reflects that of the organisation’s social structure – e.g., who rather than what. This doesn’t imply that you should skip your ‘structured design’, ‘object oriented programming’ or ‘patterns’ course at uni and head down to the bar to socialise instead, but I think there is a number of implications that especially software architects need to pay attention to when dealing with a system of software systems:

• The architecture will be at the mercy of the organisational structure or social networks. Although I don’t suspect that software architects need to become experts in social science or organisational behaviours, it will be helpful to understand the basics. To get you started, I’d suggest a couple of books: The Hidden Power of Social Networks: Understanding How Work Really Gets Done in Organizations and Getting Things Done When You Are Not in Charge
• Software architects can no longer pretend that they are software versions of ‘real’ building architects. We’ll need more skills similar to those found in policy makers (but not politicians?), negotiators and communicators. Robert Schaefer’s ‘Software maturity: design as a dark art‘ is one place to start.
• The duplication of software functionality or applications within organisations or commercial enterprises isn’t necessarily a bad thing in itself, but instead we need to focus on warning signs such as duplicated organisational roles (or responsibility overlaps) or lacking communication channels (related to the first point). I know many might be up in hands if they heard statements such as this one – wasting resources are never good! – but we need to be serious about the ‘cost of re-use‘ versus the ‘cost of (almost) re-implementation’. Even when viewed within the context of Total Cost of Ownership, my claim is that the former isn’t always the clear winner.
• Focus on interoperability instead of integration. So what’s the difference? NEHTA‘s Interoperability Framework captures the difference as the ability to maintain integration over time at minimum effort, e.g., the ability to maintain integration despite changes to or replacement of the communicating systems. Other references include the very comprehensive ‘A Survey of Interoperability Measurement‘ – if you can’t measure it, then you are not going to get it.

Today’s realities of software development are essentially about adding, changing or removing parts of an existing complex (software) system through a continuous process of negotiations, bargaining and policing – and our ability to do this is directly linked to our software’s ability to interoperate.