[Note: this post will be updated over the next month as I continue to reflect on the past 2 years]
When I was just starting out on my PhD I had noticed a few key trends and associated issues that I wanted to explore further. Four really stood out and were related to my work:
- The first was that scholars and practitioners were talking about scenario exercises (and similar exercises) as being a type of intervention, not simply a planning tool or a forward-looking assessment. E.g. Wright and Cairns (2011, p. 5) describe the scenario method as “a mode of facilitating challenge, not through direct confrontation and opposition, but through setting alternative understandings alongside each other for reflective comparison”. They further argued that the scenario method helps to reduce “strategic inertia”;
- The second was the shift from single client (or organisational) processes to much more collaborative exercises which some practitioners argued is related to the need to evolve these practices for tackling wicked problems and related societal challenges (e.g. Wilkinson & Mangalagiu, 2012);
- The third trend was more concern about the inadequate use and discussion of theory in the context of scenario practices. Chermack (2011, p. 30), for example, went so far as to assert that “the topic of theory is generally absent from the scenario literature”; and
- The fourth is the limited amount of evaluation that is conducted and the dominance of practitioner self-report case studies in the literature – i.e. practitioners reviewing and commenting on their own exercises / activities, not independent evaluations of these exercises.
A related assertion made by Wilkinson and Mangalagiu (2012) is that there is a need to better understand “how futures bear fruit in the present” and, additionally, how it bears fruit in different contexts such as in “inter-organisational settings aimed at tackling wicked problems and puzzling challenges”. How, and under what conditions, do such practices translate into action?
These four issues noted above informed my thinking and led to the scoping of evaluation research looking at scenario practices that are being used in the context of wicked problems. This also led me to review the evaluation literature and, later, to focus on ‘realist’ evaluation. But initially I was mostly thinking about gathering evidence on an area of practice to help to build its credibility.
Further learnings and lessons now shape my more detailed research and emerging thesis, including:
1) The need to address the black box problem: The goal of understanding “how futures bear fruit in the present” is an important one that points to a desire to better understand the impacts of such exercises, along with ‘how’ such benefits and wider impacts are generated. A key learning is that this needs to go deeper than current research to examine the underlying processes through which impacts may be generated (e.g. political processes, social processes), i.e. to open the ‘black box’. A thought provoking example of the sort of research I mean is Sarah Kaplan’s paper on “Framing contests” which opened up the ‘black box’ of organisational strategy-making under uncertainty.
In a recent paper I discussed the challenges of uncovering and substantiating causal links e.g. for an impact evaluation. This is often a challenge in social research. The great advantage of an experiment (and the use of experimental research designs e.g. in evaluation) is that it is designed to give the researcher greater control and associated confidence when making causal inferences. However, in real-world contexts this is often not a workable approach. Some suggest that the judgement of the market is the best proxy measure we can use to assess the effectiveness of scenario exercises or ‘futures’ work; I’d like to think theoretical advances will mean we can do better than that, but it’s tough to do rigorously.
2) Uncertainty as both a problem and an opportunity: uncertainty is often characterised as a major problem facing actors and organisations. This is true in many cases and is often part of the rationale for conducting a scenario exercise or a similar process. However, for some actors a heightened sense of uncertainty is an opportunity. Thus rather than using methods to cope with uncertainty some actors seek to exploit or to heighten the sense of uncertainty in order to help generate change (Fligstein, 2001; Fligstein & McAdam, 2012). Under conditions of increasing social turbulence (or “fluidity”) related actors’ skills and resources are influential.
Sarah Kaplan’s ethnographic study of organisational strategy-making (cited above) captures part of this well. She writes that “where the basic meaning of the situation is up for grabs, information from the environment cannot be comprehended as a set of easily recognizable signals”. A related problem is that methods like environmental scanning are inherently limited in this situation. However, where the “meaning of the situation” is up for grabs there are new opportunities for change agents. Kaplan also argues that uncertainty “opens up the possibility for new actors to gain power”.
3) For a deeper understanding it is necessary to also consider “social” models of action: what understanding of how actors behave is assumed (both individual human actors and collective actors)? Are the ideas of economic theory or rational actor theory the only ones considered, or are other understandings also being considered? A related key learning over the past year or so has been the need for a “social” view of what actors seeks to do (e.g. in markets, in organisations, etc) – i.e. a perspective informed by a sociological view of actors. An example I have found useful in my research is useful is that actors try to put together institutional projects such as to stabilise their situation (to enable action) or to help address the problem of market formation.
4) Linked with point 3, participation in “learning” practices often has a range of motivations
5) Processes of institutional change and institutional inertia are crucial: a scenario exercise (or similar exercise) will be shaped by and shape these processes. And in order to understand institutional change and institutional inertia we need to understand the causes of social change and stability and how and why people engage in collective action. This has led me to delve more deeply into sociological theory and inquiry, both economic and political sociology.
During the first year of my research my initial core focus on gathering evidence on an area of practice broadened to more deeply examine the theoretical underpinning of such interventions and what shapes their effectiveness (consistent with a ‘theory-driven evaluation’ approach).
Now two years into this research, I also increasingly see prospective practices as a ‘site’ (amongst many others) in which broader processes and dynamics play out – or, perhaps more accurately, these practices are embedded or “enmeshed” within these processes and dynamics. The processes of institutional change and inertia noted above are an important example. For example, the Future Grid Forum (FGF) was convened by CSIRO in the context of rapidly growing contention regarding Australia’s electricity system. The FGF would have been shaped by this context and may have, in turn, influenced related processes in ways that reinforced or reduced inertial forces constraining change. In other words questions about non-incremental change and how it occurs (or doesn’t) can be explored.
(Similar thinking is evident in theses like Alireza Parandian’s thesis on ‘constructive technology assessment’ methods which also advanced knowledge on how newly emerging technologies are ‘handled’ and general dynamics of technology development and its co-evolution with society)
Perhaps, more fundamentally, social research on prospective practices can also advance: 1) knowledge on the practices and styles of reasoning through which sustainability threats are understood and managed; and 2) more actor-oriented understanding of “transition” processes (e.g. socio-technical change). Those are rather lofty goals but perhaps also worth keeping in mind.
Chermack, T.J. 2011, Scenario Planning in Organizations: How to Create, Use and Assess Scenarios, Berrett-Koehler Publishers, Inc, San Francisco.
Fligstein, N. 2001, ‘Social Skill and the Theory of Fields’, Sociological Theory, vol. 19, no. 2, pp. 105-25.
Fligstein, N. & McAdam, D. 2012, A Theory of Fields, Oxford University Press, New York, New York.
Kaplan, S. 2008, ‘Framing Contests: Strategy Making Under Uncertainty’, Organization Science, vol. 19, no. 5, pp. 729-52.
McGrail, S. 2014, ‘Rethinking the roles of evaluation in learning how to solve ‘wicked’ problems: The case of anticipatory techniques used to support climate change mitigation and adaptation’, Evaluation Journal of Australasia, vol. 14, no. 2, pp. 4-16.
Wilkinson, A. & Mangalagiu, D. 2012, ‘Learning with futures to realise progress towards sustainability: The WBCSD Vision 2050 Initiative’, Futures, vol. 44, no. 4, pp. 372–84.
Wright, G. & Cairns, G. 2011, Scenario Thinking: Practical Approaches to the Future, Palgrave Macmillan, New York, NY.