Every year, The Economist, in its “The World in…” series, assesses it successes and failures regarding its past yearly forecasts (e.g. for 2012). This is an exemplary behaviour that should be adopted by all practitioners: if we are to deliver good and actionable strategic foresight and warning, and to improve our process, methodology and thus our final products, then we should always evaluate our work. Having now completed our last series of updates on the state of play for the Syrian war, we can now start assessing how our own scenarios and indicators fared so far, if they need to be updated and the potential methodological improvements that we should endeavour.

Evaluating the scenarios

As the Geneva conference took place (see previous post), we have obviously been in the case of scenario 1 – Negotiating Peace for Syria in Geneva, in its version 1.2 – all but the Salafis.


Thus the overall definition of this scenario and the possibility to see it happening were correctly anticipated, including the fact that the Salafi-Nationalists would refuse to participate in the negotiations. The “Jihadis” were, of course, excluded as foreseen. The fact that the regime of Bashar al-Assad was not represented by a person acceptable to both the regime and the “opposition” – as imagined in the scenario – implied that the negotiations could not be successful.

We moved thus to scenario 1.2.2 – Back to civil war – Salafis’ advantage?  The likelihood to see this scenario happening rather than its alternative, scenario 1.2.1 – An unlikely very fragile external peace was also correctly anticipated (the likelihood was portrayed through the thickness of the lines on the overall presentation for the scenarios – see also explanation in report in pdf – Potential Futures for Syria in the Fog of War).

The latest efforts by the Friends of Syria, notably the U.S. and Saudi Arabia to strengthen militarily the SMC and the “moderate opposition” (see previous post), may also be seen as an attempt, as far as the U.S. and Western countries belonging to the Friends of Syria, to counter the advantage we anticipated the Salafi-nationalists would have in this scenario and that appeared as likely before Geneva.

We are here at the heart of the difficulty of evaluation as evolving understanding by various actors and related decisions have the potential to alter the likelihood and the content of scenarios. In the case of a product delivered publicly by a think tank, thus without the capacity to trace with certainty any influence on decision-making, we cannot know if actions that could, in the future, infirm scenarios are a sign of failure (we failed to anticipate the main potential chains of decisions for actors and the related scenarios) or success (we anticipated properly and a change of understanding related to the scenarios led to new actions).  This would not (or should not) be the case if influence on decision-making can be traced. Indeed, in the case of a strategic foresight and warning process being established or created within an institutional framework (be it for a single product/issue or for the full organisation of units/offices) care will be taken to make sure this traceability can be ensured.

scenarios Syrian War, indicators, evaluating scenariosWhatever the case, methodologically, the new military strengthening of the “moderate opposition” emphasises – as underlined in the introduction of the full report on the future of Syria – that the scenarios we created and that were focused on the Syrian actors and the Syrian battlefield, should be complemented by a detailed state of play for all external actors, then leading to revised scenarios and potentially to more cases and narratives. The difficulty then will be to find the right balance in the number of scenarios, enough to cover most “lines of plausibility” but not too many that would be confusing to readers and decision-makers.

We also correctly considered that the Kurds would be involved in a peace conference, and that their participation would initially stem from the Supreme Kurdish Council (SKC) and estimated that their presence or absence would not influence the immediate prospects for the conference. In reality, in Geneva, we indeed had a Kurdish representation but if it evolved from the SKC as anticipated, this was only temporary, and it led the Kurdish National Council (KNC), which finally went to Geneva as only one part of the overall Syrian opposition delegation, to leave the SKC, while the SKC sponsored the implementation of the autonomous Rojava (see The Kurds and Rojava, State-Building in the Syrian War).

sipanhemo1 scThus, we underestimated the internecine fights within the Kurdish groups. We also potentially underplayed the Kurdish response to the lack of support and interest for the Kurdish issue both within the Syrian “opposition” and within the international “community”, considering their determination to see their fate taken into account and not to be again a powerless pawn in others’ game, as shown by the creation of Rojava. As a result of the evolution within the Kurdish nexus, the potential strategic importance of the Kurds has most probably changed, and the various existing scenarios should be supplemented to reflect this evolution. Existing scenarios should also be revisited in this light.

Scenarios, cicil war, SyriaConsidering furthermore our increasingly unstable world, as underlined by the crisis in Ukraine and Crimea, the potential scenarios regarding a partition (that was mentioned – see grey box – but not detailed as it was estimated to be outside the chosen time frame, i.e. 5 years starting April 2013), even if the Syrian Kurds make sure to emphasize this is not their aim, may similarly start to be addressed more in detail. We see here at work the difficulty to make scenarios under the fog of war as well as the complexity to address timeline and timeframe notably in a global, connected world in transition.

Checking the indicators

Let us now check the indicators we had identified to estimate the likelihood to see scenario 1 happening.

The question we had selected that presided over the choice of indicators that should be monitored in the future (i.e. once the scenarios finalized) is “What could enhance the likelihood to see such a scenario happening?”

When an indicator is monitored, the facts that are collected and correspond to the state of the indicator at a specific moment in time are called indications. In other words, an indicator is a variable that may take a range of values in reality. We then use the indications collected to make an assessment of the possible answer question related to the indicators.

To take an analogy, if we imagine many roads leading to scenario 1, each indicator could be seen as a traffic light opening or closing each road, and each indication would be the colour taken by the traffic light. Furthermore, at each traffic light, the longer the driver would stay the higher the odds it would choose another path or decide about doing something else than going to scenario 1. If all traffic lights show the colour green, then all drivers should arrive quickly to scenario 1 while the likelihood that each arrives are also enhanced. The arrival of as many drivers as possible enhances the likelihood to see scenario 1 happen. Of course, reality is more complex, most of the time there are more than one traffic light on the same road, while some roads and some drivers may be more important than others.

When we also asked the question “What are the supporting facts increasing the plausibility of scenario 1?” to be able to estimate what was the likelihood to see this scenario happening at the time of the creation of the scenarios, we were looking for the indications existing when the scenarios were written.

  1. The civil war in Syria drags on: Yes
    • meeting syrian weaponsWith specific evolution favouring diplomatic talks between international powers: Yes (notably as a consequence of the chemical disarmament following the August 2013 crisis – see corresponding part in Update 7 October 2013, Al-Assad regime groups). The type of evolution over the winter 2012/2013 that made scenario 1 plausible in the first place continued and thus reinforced the likelihood of the scenario, until making it a reality. For the sake of clarity for future monitoring, what was used to assess probability in May 2013 could have also been rephrased and made explicit for the future, rather than left in an implicit form.
  2. Fear by external actors to see further use and spread of chemical weapons: Yes, as exemplified during the August 2013 crisis.
  3. Heightened fear by external actors to see the Syrian conflict spilling over further: Yes, considering the evolution in both Lebanon and Iraq, which only worsened with time. Yes also, in the “diplomatic” field, when we consider the hardening of the Saudi position, for example.
  4. Difficulty to implement rapidly, efficiently and with a high likelihood of success any other solution: Yes, with the American and European fear and inefficiency (to moderate according to country) in reacting to the chemical weapons’ crisis, on the one hand, and with, on the other, potentially, the difficulty of Russia (and Iran) to always influence the regime of Bashar al-Assad (see for example Iran’s reaction to the chemical weapons crisis – e.g. Marcus George, 16 Sept 2013, Reuters, and the speed with which Russian minister of Foreign Affairs Lavrov picked upon and supported the chemical disarmament’s solution).
  5. Cost of intervention for intervening countries: Yes, especially for the U.S. and Europe, not only in economic terms, considering public expenses and budgets, which continue being reduced, but also in terms of popular support, as again shown during the chemical weapons’ crisis.
  6. Increased violence and multiplication of attacks during negotiations as actors will seek the strongest bargaining position possible at the negotiation table: Yes. Indeed, no ceasefire took place while new military positioning took place among the various actors’ supporters (see previous post).
  7. Discussions, declarations, bargains and twists regarding the participation of various actors: Yes, as the month leading to Geneva showed, including the “affair” of Iran’s invitation by the U.N. – e.g. BBC News 20 January 2014, or the impact of Geneva on the Kurdish groups (see above), or the reconfiguration of the Salafis groups and their relationship to the NC and the SMC. This was (and still is for the future, when another peace conference will take place) a particularly important indicator as from the composition of the delegations and their relative strength depends the result of the conference.

As all indicators turned “green and even greener” (see analogy above on traffic lights) as time went by, then it meant that the likelihood to see scenario 1 happen, as well as the proximity of the event on the timeline both increased. The content of the indications (those facts that are collected and correspond to the selected indicators) told us which sub-scenario would happen and enhanced the already identified  likelihood to see the peace talks succeed or fail and thus to see sub-scenario 1.2.2 take place, which is equivalent to a specific version of scenario 2.

This evaluation shows first that scenario 1 and its sub-scenarios, as well as the related indicators are valid and can thus be kept for the future, as one day another peace conference will take place.  It also shows how both strategic foresight and strategic warning should be utilized jointly. It then underlines that the scenarios should now be supplemented by more sub-scenarios, especially to consider the potential novel strategic strength of the Kurds, as well as the change in policy of some of the Friends of Syria. Meanwhile scenarios envisioning a partition of Syria may now be added in more detail.

By checking scenarios and indicators, updating and supplementing them accordingly, we make them a coherent organic whole, which takes stock of reality as time passes and thus remains useful to policy and decision-makers.

Featured image: U.S. Secretary of State John Kerry joins with attendees the Syrian Donors’ Conference for a group photo in Kuwait City, Kuwait, on January 15, 2014. [State Department photo/ Public Domain]

One thought on “Evaluating Scenarios and Indicators for the Syrian War”

Comments are closed.