Exploring the science of complexity series (part 18): Concept 9 – Self-organisation

This article is part 18 of a series of articles featuring the ODI Working Paper Exploring the science of complexity: Ideas and implications for development and humanitarian efforts.

Complexity and agency – Concepts 8, 9, and 10

Certain kinds of systems are made up of individual adaptive agents acting for their own purposes, and with their own view of the situation. Such agents can be powerful in shaping the system. A special class of complex systems is made up of adaptive agents (Concept 8), which react to the system and to each other, and which may make decisions and develop strategies to influence other agents or the overall system. The ways in which these actors interact can give rise to self-organised phenomena (Concept 9). And as agents operate in a system, changes in the system and changes in the other actors can feed back, leading to co-evolution of agents and the system (Concept 10).

Concept 9 – Self-organisation

Outline of the concept

Self-organisation is where macro-scale patterns of behaviour occur as the result of the interactions of individuals who act according to their own goals and aims and based on their limited information and perspective on the situation.

The concept of self-organisation echoes emergent properties, and the fact that a complex system cannot be understood as the sum of its parts, since it may not be discernible from the properties of the individual agents and how they may behave when interacting in large numbers. For example, studies have shown how highly segregated neighbourhoods can arise from only low levels of racism in individuals¹, as well as how cooperative farming practices can arise from the interaction of self-interested farmers owing to the necessities of farming methods and the local ecosystem². The market is probably the exemplary self-organising system. As Nobel Laureate Ilya Prigogine has put it³: ‘the economy is a self-organising system, in which market structures are spontaneously organised by such things as the demand for labour and demand for goods and services’.

Detailed explanation

Westley et al.⁴ argue that:

Bottom-up behaviour seems illogical to Western minds … we have a hierarchical bias against self-organisation … [which is displayed in] our common understanding of how human change happens, especially in organisations. Our popular management magazines are filled with stories of the omniscient CEO or leader who can see the opportunities or threats in the environment and leads the people into the light. [However,] self-organisation is critical to achieving [change].

It has been argued that in international development there is a ‘reliance on top-down oversight by, and help and guidance from, global agencies … local and regional actors are of secondary importance’ ⁵.

While many have argued that complexity science as a way of determining social policy may be an articulation of free market ethos as opposed to regulation or government involvement, it is important to note that self-organisation is in fact a neutral concept. A study of the role of self-organisation in the Rwandan genocide [by Bhavnani]⁶ argues that the levels of mass participation can be partly explained by a self-organising emergence of a violence-promoting norm among the Hutu community, such that killing Tutsis became the norm among members of the ethnic group. This self-organisation, which Bhavnani demonstrates using ABM, was driven by complex patterns of interactions among individuals forming Rwandan society. The study concludes that the frequently attributed causes of the genocide – including the death of the Rwandan president in a plane crash, the ethnic tensions and the post-war culture in Rwanda at the time – are at best partial explanations that need to be related to the bottom-up collective processes of violence through which the genocide unfolded. It should be apparent that the various ordered structures and patterns that emerge from a mass of complex interactions need not necessarily be ‘good’ for all individuals who are part of them.

Self-organisation, then, describes how the adaptive strategies of individual agents in particular settings are able to give rise to a whole range of emergent phenomena. To take one commonly cited example, in situations where systems exhibit high levels of unpredictability, adaptive behaviours among individual agents can lead to the emergence of resilience.⁷ The agents may try many strategies in the hope that some of them will be able to survive whatever challenges or surprises they encounter. For example, businesses often develop many diverse prototypes but abandon them if they do not meet specific targets. This strategy requires a willingness to accept many failures and/or to deal with the same challenge in different ways. Self-organisation can be seen as emergent phenomena arising from the interactions of adaptive agents.

In the case of weak agents (or reactive), self-organisation may be driven entirely by their interactions with other actors. For strong (or cognitive) agents, interaction may still be important, but their internal cognitive structures, beliefs and perceptions are also significant.

Work on change processes in adaptive ecosystems⁸ shows that self-organisation need not necessarily be about change, but equally can relate to resilience in the face of change. The capacity for resilience can be seen as a cyclical process with four (continuous and often simultaneous) stages: release, reorganisation, exploitation and conservation. Consider this in relation to an aid agency. Release (or creative destruction) involves the destruction of some existing organisational structures. This often frees up essential resources, such as an organisation restructuring to enable growth in new areas, or cycles of destruction in economies releasing innovation and creativity. After release is reorganisation, where there is competition for the newly available resources. New opportunities are sought and connections made, similar to the number of new, small projects that are established following a restructuring, competing for space. Then comes exploitation, where ‘the system invests heavily in the dominant species or winning proposal’ ⁹, and available resources are drawn on heavily; the themes and ideas that gained the upper hand now reap the rewards of resources and skills, growing rapidly. The fourth stage is the conservation or maturity stage, where the dominance of elements limits the opportunity for new growth; programmes have now grown large and dominate the landscape, leaving little room for each other to expand and allowing scarce resources for smaller, more innovative projects to emerge.

The idea of resilience through self-organisation is also present in the concept of autopoiesis, which describes how living systems ‘are organised in such a way that their processes produce the very components necessary for the continuance of these processes’ ¹⁰. In the context of social systems, an interesting manifestation of autopoiesis is the processes through which systems and organisations have a capacity to change their internal structure while maintaining their externally and internally perceived organisational identity. In this sense, autopoiesis and homeostasis may be seen as an adaptive agent’s response to ‘edge of chaos’ systems. A previous ODI working paper¹¹ argues that autopoiesis can be observed in the ways that community groups can reorganise within themselves in order to effectively manage available natural resources.

Example: Leadership and self-organisation

The traditional perspective of leadership is based on a view of organisations as mechanical systems in which basic laws are in operation^{12 13}. From this view, organisations are made up of highly prescriptive rule sets, formalised control and hierarchical authority structures, leading to well defined responses to a changing but knowable world. The aim of the organisation is order, and leaders are expected to contribute to this stabilisation through directive actions, based on planning for the future and controlling the organisational response. However, in complex systems, the future cannot be predicted, and change cannot be directed. Marion and Uhl-Bien¹⁴ suggest that in complex systems there is a need for different leadership qualities.

Empirical research over a five-year period, including an in-depth review of a values-based organisation^{15 16} has highlighted a number of vital leadership qualities. Leaders of self-organised adaptive agents are characterised by their ability to 1) disrupt existing patterns; 2) encourage novelty; and 3) use ‘sensemaking’. Each of these is worth exploring in a little more detail.

Leaders disrupt existing patterns in organisational behaviour by creating and highlighting conflicts, rather than stabilising the organisation. For example, in the organisation under study, successful leaders made several radical and controversial changes, internally and externally, and used communications to bring attention to these changes as a way of highlighting the importance of the ongoing change. This contrasts with the traditional leadership approach of creating predictable behaviours by minimising conflict and eliminating uncertainty. Another way for leaders to disrupt existing patterns is by acknowledging and embracing uncertainty, refusing to back away from uncomfortable truths, talking openly about the most serious issues, and challenging institutional ‘taboos’. This can encourage more open thinking about these issues, and provide legitimate ground for new ideas and patterns to emerge. Again, the traditional leadership approach was to shy away from difficult conversations and focus on hoped-for certainties.

Leaders encourage novelty, looking for innovation rather than innovating. They do this by generating and reinforcing simple rules which provide ‘tenacious rigidity about principles and complete flexibility in how to go about carrying out the principle’. Facilitating interactions was also key, in that it enabled staff to start interacting with each other in new and different ways. Instead of creating a single ‘assembly’ point, the successful leaders kickstarted many small group interactions, increasing connections between people and creating a richer and more unpredictable dialogue within the organisation. This contrasts with the traditional model of a leader as using command and control approaches, and maintaining strict hierarchies of reporting relationships.

Finally, leaders acted as ‘sensemakers’, interpreting rather than creating change. In any organisation, leaders should work to give meaning to what is happening, but this is particularly the case in complex organisations. In such organisations, successful leaders need to act as ‘tags’ ¹⁷. Tags enable specific behaviours by directing attention to what is important and what things mean. Leaders become tags when others recognise that they symbolise deeper messages of change. Leaders also make sense of emergent events through reframing, either in the principles of the organisation, or in the context of the hoped-for changes and how important they are. And leaders label behaviours in ways that provide coherence and shared understanding. Using language carefully, leaders are able to articulate meanings, lend weight to collective action, and clarify the hoped-for image of the organisation.

The overall conclusion was that the leaders of the organisation played a key role in radical transformation of the organisation, not by specifying it or directing it but by creating the conditions which allowed for the emergence of such change.

Implication: Use enabling approaches to empower actors at different levels of the team, organisation and system

An appreciation of the potential for social patterns and structures to emerge in a self-organised manner from the perspectives of the specific actors adds a new focus for development interventions. First, one should be prepared to approach problems as patterns of self-organisation; secondly, this needs to be recognised so that more beneficial and robust solutions can emerge. Given this, agencies should look to give actors the freedom to do this and help establish feedback mechanisms to make sense of their environment.

The actors are not restricted to the poor – from the complexity perspective, all actors are in ‘local’ interactions, whether they are at the top of organisational hierarchies or at the outside of agencies receiving assistance (Khan, personal communication, 2007). The key message is that change, order and resilience cannot be imposed from the outside or from the top down, but that these can be achieved through the adaptive tendencies of individual agents operating throughout a system. The occurrence of self-organisation in complex adaptive systems demonstrates how structured and organised relationships need not necessarily have been the result of the work of a coordinating body or have come through some hierarchical process.

An understanding of self-organisation means looking for the possible balances and interplays between the forces attempting to organise a system from the top down and the reactions of the agents within a system to each other and to the environment in which they act. This has resonances with policy processes in international development. The work of Lipsky¹⁸ on street-level bureaucrats highlights that, while in some situations hierarchical actions may be appropriate and possible, in other situations it may be more important to facilitate the self-reorganisation of elements in a system, to produce a new order that is more beneficial for those involved by creating an ‘enabling environment’.

Looking at development through the complexity lens shows that the landscape of various issues and problems is shaped by patterns of self-organisation through the continual interactions and interconnectedness of different agents acting within, adapting to and changing their environment. This requires a different approach to the current ‘command and control’ culture of many governmental bodies, and a departure from the ‘mechanistic’ mode of thinking behind policy.

To foster change it is important to look to facilitate the rearranging of these agents and interactions in a way which will produce the desired effect. This involves ‘mobilising the power and the resources to change things’ by looking ‘to unlock resources claimed by the status quo’ ¹⁹. To do this, it is important to understand the tensions, feedback processes and simple rules stemming from actors’ incentives, beliefs and actions, which hold the current pattern together. As the success of one organisation depends on those with whom it interacts,

… different individuals and organisations within a problem domain will have significantly different perspectives, based on different histories, cultures and goals. These different perspectives have to be integrated and accommodated if effective action is to be taken by all the relevant agents’ ²⁰

Since many patterns and structures observed in development emerge from the characteristics and adaptation of other actors, it is important to understand how actors see their own position and their environment, and how they have come to this position through a need to satisfy various needs, etc. This implies that ‘one key principle for negotiating change is an appreciation of others’ positions’; one of the ‘key components of emotional intelligence’ ²¹.

Appreciating the value of those affected by a problem’s perspective on it and, in general, the resilience of self-organised structures should encourage an understanding that many problems may be better addressed through facilitating self-reorganisation. This principle can be seen in the rise of ‘governance’ in development, as a way to allow countries to better deal with their problems themselves rather than through an externally applied solution. Jessop²² argues:

in the face of intensification of societal complexity … [we should see governance as] the complex art of steering multiple agencies and institutions which are operationally autonomous from one another and structurally coupled through … reciprocal interdependence … Governance appears to have moved up the theoretical and practical agenda because complexity undermines the basis for hierarchical top-down control.

Self-organisation speaks for the potency of looking to influence all those people and organisations affected by a problem, to strengthen relationships among those working towards complementary goals and to bring them together to facilitate change on a large scale. This is echoed by those emphasising the importance of relationships management to aid agencies²³ and the ‘informal networking’ approach to community development²⁴. It also leads to a focus on multistakeholder participatory processes.

In order to self-organise for their benefit, actors must have the freedom to act and get feedback on their actions to better adapt to their situations. Incorporating such freedom within the accountability frameworks of a development agency might involve granting an ‘earned autonomy’, where a unit might be allowed freedom to determine its course after proving its worth. In this situation, a centrally given policy would look to ‘establish the direction of change, set boundaries not to be crossed, allocate resources and grant permissions where units can exercise innovation and choice’ ²⁵.

In order to be able to successfully adapt and organise themselves in the face of different problems²⁶, actors need feedback about their local environment. The poor position of many communities may owe to a lack of information about the environment, leading to them not being in the best position to deal with new problems affecting them. However, ‘many agents … without seeing the dynamics of the larger system, can produce a self-organising strategy that effectively deals with a complex and out of control environment’ ²⁷. Therefore, local-level and ‘rapid-feedback’ indicators ‘help individuals, agencies and businesses make the best choices for their own actions’, and they can ‘work together to improve the system [and their position in it] so long as they get feedback and so long as they have the capacity to respond’ ²⁸.

It is important to note here that complexity science is absolutely not a means by which to justify a particular kind of ideological approach. To be explicit, self-organisation does not in and of itself give a justification of the ‘invisible hand’ of the market leading to beneficial outcomes. This is because there is no proof of the assertion that agents self-organising for personal economic benefit produce the emergent macroscopic effect of ‘generally beneficial for all’, or even that it results in ‘overall economic benefit’. In fact, Arthur’s work on complexity in economics²⁹ shows how systems of reasonably rational agents (operating under ‘bounded rationality’) do not lead to a system that behaves reasonably rationally overall.

Next part (part 19): Concept 10 – Co-evolution.

Article source: Ramalingam, B., Jones, H., Reba, T., & Young, J. (2008). Exploring the science of complexity: Ideas and implications for development and humanitarian efforts (Vol. 285). London: ODI. (https://www.odi.org/publications/583-exploring-science-complexity-ideas-and-implications-development-and-humanitarian-efforts). Republished under CC BY-NC-ND 4.0 in accordance with the Terms and conditions of the ODI website.

Header image source: qimono on Pixabay, Public Domain.

References and notes: