Economic evaluation methods 

Increasingly, the notion of value for money is being included within the stated evaluation objectives when government agencies carry out evaluations of their policies and programmes. A number of terms in common use, such as ‘value for money’, ‘cost-effectiveness’ and ‘cost-benefit’ analysis, are mistakenly used as if they are all the same thing. Here’s a quick summary of the different economic methods.

Cost-effectiveness analysis (CEA) expresses cost-effectiveness as a ratio between costs and effects, with effects being measured in natural or physical (rather than monetary) units – e.g., average cost per life saved by a new medicine.*

Often cost-effectiveness of an intervention is expressed as an ‘incremental cost-effectiveness ratio’ – that is, the additional cost of an intervention compared to its next-best alternative (e.g., the ‘old’ drug currently used to treat the same condition), divided by the additional effects it delivers. Great when an outcome can be narrowly defined, not so great if the intended outcomes are broad and multi-faceted.

A close cousin of CEA, cost-utility analysis (CUA) introduces an extra dimension – the human value of the effect. Empirically derived measures such as quality-adjusted life years (QALY) and disability-adjusted life years (DALY) scale the ‘raw’ measurement of extended lifespans to take into account the quality of those additional years. CUA rests on some simplifying assumptions (such as risk neutrality) but adds a useful lens which can drastically alter the conclusions of a study in some circumstances.

CEA and CUA can tell us about relative value for money by comparing alternative interventions – but not whether an intervention is ‘worth it’ in absolute terms. So while they can yield valuable insights, they don’t go all the way to solving the value for money puzzle.

Cost-benefit analysis (CBA) deals with effects in quite a different way: it expresses both costs and effects in monetary units, to determine the net present value of an intervention. This has some conceptual appeal in that it boils everything down to one magic number, and provides the opportunity to throw a whole range of outcomes into the single measure. But there’s a catch – you need a sound basis for valuing outcomes in monetary terms (what is the value of a life?). CBA is a pretty useful way to prioritise potential economic infrastructure projects, but has serious limitations when evaluating social investments.

Multi-criteria analysis (MCA) involves comparison of a defined range of options against a specified set of criteria. While there is no universally accepted normative model of how multi-criteria choices should be made, the ‘purest’ forms of MCA draw on multi-attribute utility theory and involve numeric scoring, weighting and summation of evaluative judgments.

MCA represents a potential way to address some of the limitations of CBA and CEA (though a common criticism is that this comes at the expense of the ostensible ‘objectivity’ of CBA and CEA). It involves explicit identification of criteria, together with a system for integrating criteria to reach a unified evaluative judgment. MCA can incorporate the results from CBA or CEA within a broader framework, thus drawing on the strengths of these approaches while at the same time permitting consideration of broader criteria. However, MCA favours cardinal numeric scoring and weighting systems over other (ordinal or qualitative) approaches. Moreover, MCA tends to rely on the values of decision makers at the exclusion of other stakeholders.

So, these formal techniques all have their place, and also have their limitations. Context is important when choosing methods. What if none of these methods are a great fit for the context? I’ll save that for another post

* Recommended reading: Drummond MF, Stoddart GL, Torrance GW. (1987). Methods for the Economic Evaluation of Health Care Programmes. 1st ed. Oxford: Oxford University Press.

 

March 2011 / updated April 2014

Comments are closed.