Explicit values in evaluation
Evaluation is about determining the value of something, whether it’s a policy, program, e-bike, or coffee machine. In everyday life, we are all constantly evaluating things. How well did the Prime Minister perform in her last media interview? Which school should we send our kids to? Do I need a haircut? Am I a good cook? The essential feature of evaluation is that is asks how good something is.
If we’re choosing a capsicum at the vege market, we can make snap judgements. We’re probably looking for particular features of a ‘good’ capsicum, but we’re doing it almost without thinking – we’re making an evaluative judgement on an implicit basis.
Choosing a school, we’re more likely to think slowly and carefully. We’ll probably visit several schools, meet the principal and some teachers, look at their educational achievement stats, walk around and check out the vibe of the place. We might make a list of considerations, and rate each school against each consideration.
Professional evaluation is a bit like choosing a school, in that we’re doing it systematically. We’re not just gathering facts and figures, we’re also looking to explicitly identify what considerations would tell us a policy or program is good, and we’re using those considerations, together with facts and figures, to make a reasoned judgement.
Professional evaluation is different from choosing a school, though, in that we are not doing it based on our personal preferences. We have to make a warranted judgement, on an agreed basis, that people with a stake in the policy or program will find valid.
To help us make a warranted judgement, we identify an agreed set of criteria – aspects of the policy or program that describe what ‘good’ looks like. And those criteria are determined on the basis of values. In evaluation, values have been defined as:
- “principles, attributes, or qualities held to be instrinsically good, desirable, important, and of general worth” (Stufflebeam, 2001, p. 1); or
- “normative beliefs about how things should be, strong preferences for particular outcomes, or principles that individuals and groups hold to be desirable, good or worthy” (Schwandt, 2015, p. 48).
These are just fancy ways of saying that values are an expression of what matters to people.
Values are verifiable facts. They can be validated “by observation, inference or definition”. In practical terms, values are “real, although logically complex, properties of everyday things” which can be determined to “an acceptable degree of objectivity and comprehensiveness” (Scriven, 1993, p. 9).
In our evaluation practice, we put a lot of effort into working with stakeholders (e.g. people and communities who are supposed to benefit from a program, people delivering it, decision-makers) to identify explicit values-based criteria that provide an agreed and transparent basis for making evaluative judgements. These criteria are the backbone of the evaluation and the means by which it delivers findings that are valid and credible, that get used and leave people better off.
And that is why we talk about explicit values in evaluation.
Adapted from King, J. (2019). Evaluation and Value for Money: Development of an approach using explicit evaluative reasoning. (Doctoral dissertation). Melbourne, Australia: University of Melbourne.
Scriven, M. (1993). The Nature of Evaluation. New Directions for Program Evaluation, 58, 5-48.
Schwandt, T. (2015). Evaluation Foundations Revisited: Cultivating a Life of the Mind for Practice. Stanford University Press.
Stufflebeam, D.L. (2001, March). Evaluation Values and Criteria Checklist.