Complexity and Scale

A post on KurzweilAI.net last week caught my eye. It excerpted a recent article in NewScientist entitled "Why the demise of civilisation may be inevitable", which declares that society's increasing complexity also increases its fragility -- and the energy needed to sustain it. What the gang at KurzweilAI.net missed is the nature of scale in complex systems dynamics.

In fact, this is the second time in a month that KurzweilAI.net has come up short. The other time was a report on a pandemic influenza detecting chip -- conveniently appearing just prior to a Pandemic Influenza Tabletop Exercise I recently participated in. But their report was wrong about current capabilities: first responders can identify various strains via an emergency polymerase chain reaction (PCR) test in 2-3 hours, not "days or weeks" as reported on their 'blog.

KurzweilAI's alarmist reporting on social complexity -- and the concomitant "solution" of "reducing" society's complexity by reengineering our institutions at a smaller scale -- shows scant attention to the essential role scale plays in complexity. To wit, if a phenomenon appears random or unpredictable at a fine scale (e.g., turbulent flow), it can be predicted at a large scale. Conversely, phenomena that are unpredictable at a large scale (e.g., ethnic violence) can be predicted at a fine scale. [The link is to the Ethnic Violence page at the New England Complex Systems Institute, and to a paper published in the journal Science last September presenting their models. These models showed a 90% correlation between single-parameter predictions (after wavelet filtering) and reported incidents in Bosnia-Herzegovina, and even higher correlations in India.]

The underlying assumption of KurzweilAI (and Ms. MacKenzie at NewScientist) is that complexity is directly proportional to scale: the larger the scale, the greater the complexity. Similarly, the smaller the scale, the less the complexity.

However, when you decouple scale from complexity, you begin to see a better fit with our observed reality. Conventional military operations (à la Schlieffen Plan, the Fulda Gap, and OPLAN 1002, to name a few) entail massive amalgamations of forces for the express purpose of simplifying the commander's perspective. Rather than drowning in the minutiae of individual soldier movements (or even platoon or company-level engagements), theater commanders -- with the helpful MIL-STD-2525 symbology, similar to NATO APP-6A -- are able to think in terms of Corps and Division elements (and, lately, Brigades -- the primary warfighting organizational element of the U.S. Army). Therefore, large scale -- but low aggregate complexity.

When the battlefield loses its conformity, though, the scale decreases -- while the complexity increases! Consider which of these two scenarios are more "complex":

1) U.S. Army V Corps blocking the Soviet 8th Guards Army in the Fulda Gap, or ...
2) U.S. Army V Corps serving under the Coalition Ground Forces Commander in post-OIF Iraq.

Many have described the inherent complexity of loosely-coordinated small forces combating a monolithic adversary, most notably the contributors to the Small Wars Journal, John Robb, and the gang at Defense & the National Interest. Perhaps the Kurzweil crew would benefit from paying more attention to these "ideas at the intersection".