Complex Systems: A Motorcycle Tour Through the Museum

In a recent article I posted here, one key piece of reasoning depended on the properties and behavior of Complex Systems. This sort of thing will come up again and again. Partly it’s me (I find Systems Theory increasingly indispensable as an investigative, analytic, and descriptive model); but partly it’s objective reality. It’s getting harder and harder to ignore, or defer responding to, Systems-driven global phenomena that have real near-term consequences. It all seems to be coming together in a spectacular crunch, and that’s not coincidental: this global trend is driven by the increasing interconnection and scale of human activity.

IT plays a huge part in this, and has to respond in two ways. First, Complex Systems show up in our application problem domains more and more often, so we need to figure out how to work with them. Second, IT itself gets Complex, and when it does we're better off if we understand the behavior of Complex Systems. This is something of a problem: Systems Theory is generally not well known, especially in our field, and there’s a lot to it.

A Physics professor of mine once used the phrase “motorcycle tour though a museum” as a rueful lament about how small a glimpse, of how vast a subject, undergraduate Physics courses could actually provide. I’ve stolen this from him, and I hope he’d approve.

In that spirit, here’s a basic overview of the general nature, shape, and implications of Complex Systems. Don’t expect theoretical precision. I’ll just try to make things conceptually accessible at a pretty high level.

Now you probably know enough to decide whether to read further. If you do, grab your boots, jacket, do-rag, helmet, assless chaps, mirrorshades, whatever you think you need for the road; and let’s roll.

A Note on Terminology

The words “Complex” and “System” have specific technical meanings in the field of Systems Theory. They also, however, have different meanings in everyday speech. The Dewey Decimal System, for example, is certainly “systematic” in the way most people understand the word; but it’s got nothing to do with Systems Theory. In order to emphasize the distinction, I will be capitalizing the formal terms and any derived forms (e.g. “Complexity”) in this article.

Complex Systems Are All Around

Any discussion of Complexity runs into the fish-can't-describe-water problem. Complexity is such a pervasive part of, well, everything, that it's hard to stand aside and talk about it.

We ourselves are, as individuals, biologically Complex organisms and psychologically Complex personalities. We work for Complex organizations, exchange information across Complex networks, buy and sell goods and services through Complex markets. We live in highly Complex societies governed by Complex political systems and sustained by Complex infrastructures. Our global civilization itself is Complex, immensely so.

That’s only the beginning, though. We are only a small number of the living organisms on the planet, and all of us live within Complex ecologies of mutual interdependence. The planet itself is Complex in its responses to energy throughput, most especially its meteorological and oceanic systems.

Our software, the devices that execute it, our programming languages and frameworks, are not necessarily Complex Systems in their own right; but the activities of developers, when working on those things, usually become Complex; and at runtime, if the application in question is big enough, or the dependencies are numerous enough, Complexities can explode in a big way.

This all matters because Complexity causes weird and unanticipated behaviors. Personally and professionally, everybody who reads XML Today is a Complex System immersed in other Complex Systems. So, we are all likely to behave in weird and unexpected ways; and so is just about everyone and everything else.

Welcome to the world of Systems thinking, you beautiful freaks.

What’s a System?

There are a lot of deep and technical descriptions of the nature of Systems. I will just give some broad ideas and rules of thumb. These points are all somewhat fuzzy. Don’t worry about it: in Systems thinking, fuzziness is a feature not a bug.

• A System is any grouping of identifiable components that interact by changing each other’s state.
• When a component changes state, the change usually triggers an interaction with some other component (or modifies an interaction that’s already in process.)
• Interactions therefore propagate from component to component, working changes as they go.
• The structure of the System is mostly defined by these internal interactions. In Systems thinking, component parts matter not because of what they are, but because of what they do.
• A System is dynamic. If there’s no motion, if there’s no change, then the components can never interact. A static structure is not a System - not the kind Systems Theory studies, anyway - no matter what people call it.
• Very importantly: interactions between components are not instantaneous. Systems act over time. This feels particularly weird to those of us in IT, because we have gotten overwhelmingly accustomed to the logic of discrete states, i.e. the execution of logically independent instructions in sequence without regard to actual physical time elapsed. More on this later.
• Systems Theory does not limit the number and kind of interactions between any two components. Components can interact in different ways, by various paths.
• Because of the way interactions propagate, it’s often the case that an interaction will eventually come around, full circle, and have a significant effect on the originating component. This is called a “feedback loop”; the presence of such loops is the fundamental characteristic of Complex Systems.

Some of these points, and the topics they open up, deserve a little more discussion.

Systems Thinking: Modeling vs Analysis

Analysis, roughly speaking, is the logical technique of picking things apart to see how they work and what’s going on. It’s one of the best tools we’ve got for a whole lot of things. We use it in IT all the time. Unfortunately, it doesn’t work worth a damn on Complex Systems.

Anyone who works with Complex Systems soon discovers that, unless a System is trivially simple in structure, analysis turns into a receding horizons problem, in which every successful piece of analysis just uncovers more analytical work - work that, in turn, has to be done (now that it’s inconveniently been defined) in order to complete the analysis.

This suggests that discrete analysis of System behaviors is not going to be very cost-effective, and often not even doable. Can it really be this bad?

Well, yeah. Try to get used to it. Analysis proceeds by attempting to isolate some contributing factor and studying its effect on the outcome. Complex Systems don’t behave like that. Analysis, when you get right down to it, assumes the ability to work with a snapshot of data. Systems don’t give that snapshot up very easily, and even if they did it would be large and hard to work with.

The best we can normally do is to observe regularities, and responses to inputs or parametric conditions, in the System’s aggregate behavior. Then we can try to develop a simplified representation of the more significant things that are going on.

In other words, we need to do some modeling.

Modeling is about looking at the behavior of the whole, rather than the details of the moving parts. A model is an information construct we can afford to build and study, representing a real System that is itself beyond our analytical capacity. The model is more easily grasped, so it helps us to reason about the thing that’s being modeled - and often to make usefully accurate predictions.

This is how Economists, Meteorologists, Ecologists, Power Grid Engineers et al do their work. They develop models from observations, and then use those models to interpret and anticipate real-world developments. (Well, OK, Economists develop models based on the evident political preferences of the people who pay their salaries. But whatever.)

Anyway: when working with Complex Systems, it’s ordinarily better to think in terms of modeling. Analysis, powerful as it is, is likely going to leave you short.

Systems Thinking: Stocks and Flows

Components can hold state, but only certain kinds of state are meaningful in terms of Systems thinking. It’s an oversimplification, but useful, to say that component state

• can vary over time;
• is significant for some interaction that takes place in the System.

There is one kind of state that is especially important for Complex Systems: the stock. A stock is a quantity whose behavior and properties are scoped to the entire System, and is transferable between components: for example, money in a financial System, or energy in a thermodynamic System.

Transfer of a stock from one component to another is called a flow. Flows conserve quantity of stock: for any given flow, the amount of stock removed from the source must be the same as the amount received by the destination.

In general, stocks are conserved within a System overall. There are two specific exceptions to this: a component in which quantities of stock are introduced into the System from outside is called a source; a component in which quantities of a stock are removed from the System (or equivalently, made permanently unavailable for any subsequent flow interaction) is called a sink.

One complication about the whole conservation of stock thing: a component may transform a quantity of a stock into a quantity of a different stock, according to a specific “exchange rate” paradigm. Euros can be converted to Yuan, heat energy to motion or to electromagnetic radiation, and so on.

Stocks and flows are subject to a strict form of bookkeeping. All sourcing, sinking, flows and transformations have to be accounted for, or else your System model is pointlessly incoherent.

Flows and transformations of stocks are the majority of the significant action in most Complex Systems, especially the real-world ones.

Systems Thinking: How Feedback Loops Work

Feedback is the principle that an action will propagate through a System and find its way back to affect the initiator of that action. In real life we say, “What goes around comes around” and it makes all the sense in the world. Real life is, of course, a Complex System. When that which goes around does, in fact, come around, we have observed the action of a feedback loop.

The simplest kind of feedback loop is one acting directly between two components. The first one does something to change the second one, which then turns around and affects the first.

Many, or most, feedback loops are not so simple. They propagate through intermediary components. As long as a component’s interaction with another component works its way back around, somehow, to the originating component, and modifies it so as to trigger (or modulate) another interaction, that’s still a feedback loop.

This is the primary reason that System interactions have be modeled as taking place over time. It isn’t just to more faithfully replicate a real-world behavior. It’s an absolute necessity in order to have something that works at all: in the logically instantaneous world of discrete states, a feedback loop is a guaranteed race condition or deadlock.

Systems Thinking: Oscillation

Oscillation occurs when feedback loops are in balance, so that they induce, or constitute, a kind of back-and-forth swinging motion of some kind.

Oscillation is a lot more common than holding still. A flag waving in the wind is an example of Complex modes of oscillation, and we consider it entirely natural because, well, because it is. If anyone saw a flag holding dead still in a stiff breeze, it would seem shockingly unnatural.

Oscillation can be stable, in which case it keeps doing the same thing (think of the engine in your motorcycle, when you’re neither accelerating nor decelerating.) It can be simple, like a pendulum, or composed of many different modes that all add together according to the mathematics of harmonics - that’s what you hear on a crowded street, multimodal oscillation of the air molecules at your eardrum.

Oscillation is the default pattern of interactions in a System.

Systems Thinking: Positive vs Negative Feedback

Put together the concept of feedback and the concept of oscillation, and you immediately realize that feedback happening within an oscillating System is not at all the same thing as feedback happening in some kind of stationary context.

This yields a really interesting categorization of kinds of feedback:

• positive feedback goes with the movement, thereby reinforcing it. In doing so, it drives oscillation. It will push something in motion, push it up against the limits of that motion - and then turn around and pull it back the other way. A kid on a playground swing learns to use positive feedback in order to get going, and to push the swing higher and faster.
• negative feedback does the opposite thing: it opposes movement. Precisely modulated negative feedback can arrest swings, decelerate a movement, bring a component to a soft landing. If the kid in our example doesn’t want to just bail out, she can reverse the sense and timing of her motions, using negative feedback to slow the swing down more decorously. (Or she can drag her feet, which is negative feedback of a different kind: rougher on the shoes, but easier to time.)

The difference between positive and negative feedback is really all in the timing: whether the kid on the swing goes higher or slows down all depends on how she synchronizes her arm, leg, and body movements with respect to the swing’s arc.

How Systems Behave: Emergent Behavior and Self-organization

A common exclamation among people who are observing a Complex System is “Hey! How’d that happen?” (Yes, I’ve cleaned up the language a bit. Startled people say colorful things.)

Complex Systems are full of surprises. It’s very easy to make largely correct general predictions about how they will respond to certain kinds of inputs or perturbations. But the more precise your predictions are, the more likely they will be turn out to be nonsense. Complex Systems simply do a lot of things that you probably wouldn’t have thought of; and even if you do know what to expect, the details are inherently hard to predict.

These unexpected actions within Systems are generically called emergent behavior. Emergent behavior can look an awful lot like a planned action, so much so that we often resort to the language of volition to describe System behaviors. But emergent behavior is not the result of any guiding purpose. It’s the aggregate outcome of the blind working of System dynamics.

When emergent behavior displays some staying power - when an emergent characteristic is sustained over time, or when it recurs after disruptive events - the System is said to be self-organizing. Self-organization is a fairly complicated topic, one we can’t really stop to dig into. But one of its manifestations is common enough, and important enough, to merit a closer look.

How Systems Behave: Complex Systems Self-regulate

Take any Complex System and introduce a destructive perturbation into it. The System may break spectacularly, by flying into unrecognizable bits, or collapse into a diminished version of its former self.

But many Complex Systems that you would expect to break seem to know how to take a punch. Your perturbation will definitely modify the System’s behavior: it will shudder and shake, but instead of collapsing or shattering, it will absorb the perturbing influence, gradually damp down the shudders, and converge back on something not too unlike what it was before.

To the extent that a System can do this, we call it resilient. Resilient Systems are generally operating with a lot of spare capacity, and can work within widely variable inputs or conditions. One fairly appalling insight we derive from this is that efficiency is the polar opposite of resilience. The more resilient a System is, the less efficiently it uses resources; and the more efficient a System is, the more brittle it will be when confronted with change.

The recovery behavior of a resilient System, manifested as the System’s tendency to resume business as usual, is called homeostasis. Homeostasis is the mechanism of stability in Systems, and it’s why we speak of Complex Systems as self-regulating.

This shows up a lot. Introduce a new, invasive species into an pond, and very often the new organism’s triumph will bog down as key nutrient stocks are consumed and formerly rare predators flourish. Collapse an economy (e.g. Russia in the 90s) and an informal network of goods and services springs up to keep people fed, clothed, warm, and drunk. Try to introduce a new programming language or technology at an organization, and people who are comfortable with their developed skills and senior positions will find ways to torpedo the new technology’s use, and regress the organization to the status quo technology stack.

Note that, in each example, we are talking about resilient Systems. Ponds, like any longstanding ecological biome, evolve towards resilience; Soviet Russia’s economy was comparatively inefficient in its use of resources, which ensured a lot of slack for the people who had to undergo the disruptions of Perestroika; and any organization in which the IT professionals can devote a lot of time and energy to counterproductive politics is an organization with plenty of spare capacity.

Does this kind of response always happen? and when it does, is it invariably successful? No, of course not, because some Systems are not very resilient. But homeostasis is a natural behavior of Systems, and can be expected if a System’s nature and condition allow for it.

Nice Trip...

So, that’s the very sparse, oversimplified, introductory overview of Complex Systems: dynamic, interactive aggregate entities whose behaviors are so complex and feedback-driven that they must be modeled rather than analyzed. Complex Systems can be expected to exhibit self-organizing behavior, and to sustain themselves in the face of various challenges.

This particular road trip has gone on long enough; but Complex Systems have a lot of relevance for contemporary computing, and Systems Theory seems to be underexamined in the field generally. There’s a lot more to be visited.

This article is the first of several that will try to crack that door open a little. The next Systems subject to be considered is to look at Systems in light of the central project of XML Today: representing information in coherent document form. Given what we’ve just looked at, what kind of information structure works well for representing Systems?

Spoiler: it ain’t XML.