Authors of papers on prospective practices often take one of two stances: 1) they argue that these are learned crafts (and certainly not a science), and question the need for greater theorisation or greater use of theory; or 2) they, increasingly, question whether they have an adequate theoretical foundation, point to gaps, and/or try to further theory building and use. For instance, Angela Wilkinson argues scenarios are a practice “in search of theory” and has explored how theories like complexity theory could help to advance such practices. Rafael Ramirez and colleagues, in Business Planning for Turbulent Times, argue that “scenario work remains theoretically underdeveloped” and explored how Emery and Trist’s “causal textures” theory could advance its grounding. Reflection on my own work has led me to reexamine and to also question the theoretical foundation of such practices.
This post briefly considers the role of theory, the types of theories that have been drawn on and could be drawn on, and tensions between the “learned craft” and emerging “applied science” camps.
Do prospective practices need a theoretical foundation?
I’ve come to see theory as playing a wide range of potential roles in scenario work or in other forms of prospective exercises. The most obvious use is grounding the causal inferences that inform a scenario in relevant theory. Scenario builders form hypotheses about what might occur and why, which are often just based on their own mental models. Scenario-builders could also apply relevant theories.
Where such theory is used, too often economic theory dominates the analysis e.g. as per economic forecasts. Other social scientific theory such as sociological theory is also relevant. Practitioners often rely on vague scenario “archetypes” which are based on little substantive theory, or apply recipe-style approaches (e.g. scenario methods developed by the Global Business Network).
Another use of theory is to explain and better understand how, why and when particular practices work. For example, Ramirez et al in Business Planning for Turbulent Times argue scenario planning is only a suitable technique under certain conditions. Causal textures theory is used to make this case and to help to explain when scenario planning is useful and when it is an unhelpful approach. Theory could, thus, help practitioners to more effectively consider the suitability of particular methods and tools.
In my own work and research I’m especially interested in understanding the impacts of prospective practices, e.g. their contribution to cognitive changes, organisational change, systemic change, or broader social change. Clearly a range of theories could be used to help to understand these impacts (and the relevant causal processes) and, hopefully, maximise their positive effects. For instance, theories of social learning, collective action, or organisational change could be used.
It is also worth considering the theoretical underpinning of specific areas of practice such ‘corporate foresight’. For example, scholar-practitioners such as Rene Rohrbeck who have tried to theoretically ground corporate foresight draw on innovation management models and theories developed in strategic management (e.g. of organisational ambidexterity, competitive advantage, etc).
These thoughts are only a small selection of the possible uses of theory. Also important are theories developed in fields like cognitive psychology and cognitive science which tell us about how people process information, form judgements, and consider the future. The theories also point to common biases which often influence scenario-building and inform the rationale for such practices.
In summary: theories can be used to underpin and inform forward views, and explain those that are developed (e.g. considering the shaping influence of cognitive processes and biases); and, secondly, theory can be used when seeking to explain the effects of a particular intervention or when developing related intervention theories. The latter aspect is the focus on my current doctoral research.
Additionally, three broad categories of theory are worth noting and considering:
1) Grand theories and “theories of everything”
During my Masters studies at Swinburne University (see their strategic foresight program) we studied Ken Wilber’s integral theory (see www.kenwilber.com). For example, his books A Theory of Everything and Integral Psychology were required reading for two units. At this time Richard Slaughter was a professor at Swinburne University and advocated a new form of futures inquiry known as “integral futures”. Integral theory is often used as an orienting framework (i.e. used as a heuristic tool), e.g. the four quadrant model or as guidance for integrative research. Others use integral theory in a more deterministic ways, based on claimed evolutionary processes (e.g. of cultural or psychological evolution).
Other examples include grand sociological theories such as Anthony Giddens’ “structuration” theory (e.g. which has been used in scenario planning as per this paper), and theorisation of environmental limits to growth (e.g. the Limits to Growth project) in which scholars advance far-reaching claims about the nature of growth (especially exponential growth) and its consequences. Neo-Malthusians such as Paul Ehrlich, Richard Slaughter and Jared Diamond develop and apply deterministic theories of such limits.
I’m increasingly of the view that such grand theory is an inherently flawed enterprise. The social world is in a permanent state of self-transformation. We consequently shouldn’t seek timeless social laws (as per a “social physics”), or expect to discover uniform or unchanging patterns of behaviour. The search for general laws that apply uniformly to the social world – regardless of time or place – and causal relations that occur universally is based on the model of the natural / physical sciences.
We can further consider this in relation to theoretical scopes and analytical levels, e.g. as per forms of macrosociology and microsociology and related multi-level perspectives.
2) “Middle range” theory and meso-level structures and processes
The concept of middle-range theory was developed by Robert Merton, who argued for:
Theories that lie between the minor but necessary working hypotheses that evolved in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization and social change (Merton 1968, p.39)
Merton was particularly concerned that the development of all-encompassing systems of concepts would prove to be futile and sterile. In particular, he critiqued the theory of structural functionalism developed by sociologists. In contrast, middle range theories, as described by Geels (2007), “do not consist of elaborate frameworks … instead, MRT consists of a limited set of interrelated propositions, aimed at understanding limited topics”. The resulting analytical models and the theorised causal mechanisms “are not deterministic, but indicate how one concept influences another”.
A middle-range theory should be specific enough to be tested by empirical research, yet also sufficiently general to cover a range of phenomena. The more general and abstract a theory is (as per grand theory) the more difficult it is to test it empirically.
In my work I’ve fruitfully used middle range theories of technology dynamics such as those focussed on the roles of future expectations in technological change and on socio-technical transitions (e.g. see Geels 2007; McGrail 2010). Regarding expectations and new technologies, various patterns have been observed (e.g. hype cycles) and theories aim to explain these patterns. These theories consider how actors address and manage the uncertainties that are faced whenever new technologies emerge.
A related term that’s often used is meso-level theory – meso meaning middle level or intermediate space – such as in the theorisation of meso-level structures (e.g. organisations, and communities).
I’ve previously written on this blog about the concept of a “strategic action field” and field theory which is an example of a sociological theory focussed on the mesolevel. A strategic action field is defined as a stable social world or, more technically, as a ‘mesolevel social order’. These orders can be further defined “by the relationships between all of the players who view themselves as members of the field” (Fligstein & McAdam, 2012). Stability and change in meso-level structures and in field dynamics helps to explain change in organisations, markets and policy domains – and can also be used to anticipate change.
3) Micro-level theories (e.g. from microsociology)
Here theory considers things like group dynamics and interaction in micro contexts. For instance, microsociology is the study of everyday behaviours in situations of face-to-face interaction. It’s possible to consider how micro interactions affect larger social processes and how macro systems affect more confined social settings. Giddens et al (2012, p. 25) outline the perspective:
Micro studies illuminate broad institutional patterns. Face-to-face interaction is the basis of all forms of social organization, no matter how large scale. In studying a business corporation, we could examine face-to-face behaviour to analyze, for example, the interaction of directors in the boardroom, people working in various offices, or workers on the factory floor. We would not gain a picture of the whole corporation this way … yet we would certainly contribute significantly to understanding how the organization works.
Given participatory prospective practices tend to take place in workshop settings, or other face-to-face interaction, there is likely much to learn from micro-level theories. For example, the concept of “group think”, which is part of group dynamics theory, is often referenced in the scenario literature.
My own evolving thinking on the roles, use and abuse of theory
I have some time for the folk in the “learned craft” camp. There is a valid sense that practice-led fields inherently involve experimentation and have an element of artistry. There are good reasons to be skeptical about a “science of strategic interventions” (as sought by these practitioners).
However, I’m also persuaded by those who assert that there is no practice without theory (indeed theories of change are often tacit in practice and policies) and those who argue that prospective practices are under-theorised (such as scenario planning scholars like Ramirez et al). When I consider prominent examples of such practices I’m struck by how little theory is formally drawn on.
Surely if we are able to develop a stronger understanding of how, when and why particular practices ‘work’ and don’t ‘work’ (even if this necessarily is, and remains, a partial understanding) then this can only serve to improve these practices. And this will require going beyond the anecdotal case accounts of practitioners that continue to dominate the literature on these practices.
To my mind at least, and to take an example relevant to my own research, if a prospective practice aims to contribute to collective action or to enable social learning then we would be wise to draw on relevant, robust social scientific theory. Success may remain patchy and far from guaranteed; however, it should be possible to improve the design, targeting and delivery of such strategic interventions.
If you have thoughts on the use and roles of theory in prospective practices I’d love to hear them…