Scale and (Over)Simplification

Following up yesterday's post on "Complexity and Scale", and the alarmist notion that society is bound to collapse because of its increasing complexity, let me turn the tables and describe a worrisome trend: that of oversimplification in the face of complexity.

Prof. Yaneer Bar-Yam of the New England Complex Systems Insitute (NECSI) has done pioneering research into the dynamics of complex systems. The first textbook on the subject was written by Yaneer in 1997, and a more-accessible (i.e., less math) introduction on applying complexity science to real-world problems (Making Things Work) followed in 2004. One of the most fundamental concepts in complex systems is the trade-off between complexity and simplicity when related to scale. Greater complexity at a large scale means greater simplicity at a fine scale, and vice versa.

Where we get into trouble is when we ascribe simple models that are inadequate for the complexity at a given scale. For instance, a hierarchy is limited in its inherent complexity to the complexity of its leader. Yet we persist in building simple hierarchical organizations (e.g., CPA and its successor organizations in Iraq) when the dynamics of the environment call for a more modular, diffuse network of organizations. Ashby's Law of Requisite Variety (c. 1956) asserts that, for a system to achieve stability in the midst of perturbations, its number of control variables must be greater than the number of possible states in that system.

Therefore, the most efficient organization in a dynamic, complex large-scale environment is not a Napoleonic hierarchy with a single overarching authority -- but rather a distributed, loosely connected network of specialized subnets that are empowered to act in response to system perturbations. Pop quiz: does the latter statement better describe the organizational paradigm of the coalition forces in Iraq, or of the various other force structures there (Mahdi, Badr, AQI, etc.)?

It is interesting to note that General Odom's recent Senate testimony (h/t Abu Muqawama) associates the decline in violence since General Petraeus's " ... reflects a dispersion of power to dozens of local strong men who distrust the government and occasionally fight among themselves. Thus the basic military situation is far worse because of the proliferation of armed groups under local military chiefs who follow a proliferating number of political bosses." Increased complexity at a higher scale due to the diffusion of military authority to lower scales.

The implications for the conventional force structure of the U.S. security infrastructure are profound. To borrow terminology from Tom Barnett, not only does this mean "Leviathan" can't do "SysAdmin" -- it means that the idea of a centrally-organized SysAdmin is doomed to failure.

Now that the study of self-organized criticality is 20 years old, which describes when a critical point in a dynamic system acts as an attractor, perhaps we will see commensurate change within our organizational models. For instance, the "Incident Command System" of the Federal Emergency Management Agency (which was derived from interagency evolution in response to wildfires, which was in turn derived from the military's deliberate planning process) defines standards to facilitate rapid organization, information sharing and decision-making.

Organizational models that facilitate effective (and appropriate) exchange of information, and -- most importantly -- allow the organization's evolution in the face of cooperation and competition are more effective in contending with the complexity of our world.