Most people in the public sector are familiar with the criteria-based VFM frameworks based on the ‘E’s (economy, efficiency, effectiveness, and typically cost-effectiveness, with equity and in some cases ethics). They have become almost an industry standard. However, in sectors typified by uncertainty – in particular but not exclusively international development – they have some key limitations:
- They tend to be heavily indicator-based and involve significant data burden in environments where data availability is often constrained, making them often impracticable;
- They rarely acknowledge the importance of context in shaping what is feasible – what ‘good’ VFM looks like in a given circumstance – and hence often feel misaligned with the way people actually manage resources to create value;
- More fundamentally, they are built on assumptions of perfect knowledge, even though in practice all but the simplest of initiatives operate with risk and uncertainty.
Just to be clear, I don’t propose ditching the E’s: conceptually they are foundational to VFM and provide a very useful diagnostic tool. But I am saying, use the E’s as a starting point, not as the means to structure and communicate the VFM assessment. In this regard, contrast the lengthy frameworks typically produced by development programmes with the clear-sighted, readable and influential VFM reports produced by my former employer, the UK National Audit Office.
In the development sector, I often see frameworks and indicators substituting for careful thought about what VFM means in a particular situation. But in reality most initiatives are trying to deliver VFM under uncertainty – where constraints abound and outcomes are outside of the initiative’s direct control. We need instead simple, useful approaches that recognise this. With this in mind, I’d suggest starting from two fundamental VFM conditions that need to be met:
| VFM condition | Comment |
| 1. Resources must be used optimally to support the initiative’s objectives | Optimal resource use for VFM under uncertainty means identifying the constraints on what is possible, taking appropriate risks to advance the objectives and, at the same time, ensuring that resources and risks are managed as well as possible. |
| 2. Over time, sufficient value must be generated to justify the initiative’s costs. | Even if the first condition is met, VFM under uncertainty requires not that outcomes are guaranteed but that over time sufficient results are being (or likely to be) achieved that justify the (continued) effort. |
Breaking these conditions down a little further, ‘optimal’ resource use in the first condition implies:
- Context – the operating environment, the intervention type, the objectives – imposes restrictions on what is possible in VFM terms and this needs to be recognised[1].
- Outcomes are uncertain and so the ‘reward/risk’ balance needs to be thought about and aligned with stakeholders’ tolerance for risk – an overly cautious/less ambitious approach or an overly optimistic/highly aspirational approach are both likely to reduce VFM.
- And within the parameters imposed by context and risk tolerance, resources and risks still need to be managed economically and efficiently – failure to do the management basics well also undermines VFM.
For the second condition, acknowledging that outcomes are not guaranteed means, by definition, accepting there’s a risk of falling short. The test for VFM under uncertainty, therefore, cannot be simply whether the initiative succeeded, but whether the prospects of success were reasonable. As long as the first condition is met, it is possible, in the short term at least, for a less-than-effective initiative to nevertheless provide VFM. However, the second condition puts a limit on this: continued shortcomings in achievements over a longer period without a rethink of strategy would represent poor VFM.
So what is needed to strengthen the assessment of VFM under uncertainty? In my view… a few suggestions and an illustration:
- Avoid formulaic reliance on standard sets of indicators and instead develop an approach that is far more risk-informed, anchored in context and driven by key resource management considerations.
- By risk-informed I mean, don’t try to address every element of each ‘E’ but identify where the biggest risks to VFM lie for the particular initiative under review – noting that these will differ significantly between types of initiatives and context[2].
- By anchored in context, I mean identify the constraints that determine what good use of resources looks like in the particular context – and build these limitations explicitly into the evaluative framework.
- And by management considerations, I mean use an approach driven primarily by key questions (not indicators) and focused on the points where decisions about resource use are made or controlled.
- And finally, incorporate uncertainty around Effectiveness more visibly into the VFM assessment[3] and engage stakeholders in an honest conversation about risk tolerance (what chance/degree of ‘non-Effectiveness’ are they willing to bear in order to pursue the objectives?)
And to illustrate, below is a figure. It proposes a set of key questions that could be applied to a portfolio programme. Of course, this paper argues for more tailoring and selectivity based on context so I’m not suggesting this is the ‘model’ to follow. But hopefully it provides some ideas that can be adapted to different forms of delivery and need. The figure comprises ten key questions – all of which can be mapped to the ‘E’s but are grounded more clearly in programme management processes. These are still challenging, but for me much more meaningful.
[1] For example, in some contexts Equity aims involve trade-offs with Efficiency, while in others a low skills base constrains use of market-based approaches to Economy.
[2] As a thought experiment, think how the major risks to VFM would differ for a major infrastructure project in the UK compared with an advisory programme on public sector reform in Cambodia.
[3] For example by rating the ‘risk and reward’ associated with different activities or by using more formal, probabilistic methods to estimate ‘expected value’
