Explicit values in evaluation
Evaluation is about determining the value of something, whether it’s a policy, a program, a funding bid, a student’s performance, or a scientific study. In everyday life, we are all constantly evaluating things. How quickly and reliably can the public transport system get me to my destination? Which school should we send our kids to? Am I a good cook? The essential feature of evaluation is that is asks how good something is (Davidson, 2005).
If we’re choosing an avocado at the food market, we can make snap judgements. We’re probably looking for particular features of a ‘good’ avocado, but we’re doing it almost without thinking – we’re making an evaluative judgement on an implicit basis.
Choosing a school, we’re more likely to think slowly and carefully. We’ll probably visit several schools, meet the principal and some teachers, look at their educational achievement stats, walk around and check out the vibe of the place. We might make a list of considerations, and rate each school against each consideration.
Professional evaluation is a bit like choosing a school, in that we’re doing it systematically. We’re not just gathering facts and figures, we’re also looking to explicitly identify features of good quality or value, and we’re using those features as a yardstick when we look at the facts and figures, to make a reasoned judgement.
Professional evaluation is different from choosing a school, though, in that we are not doing it based on our personal preferences. We have to make a warranted judgement, on an agreed basis, that people with a stake in the policy or program will find valid.
To help us make a warranted judgement, we identify an agreed set of criteria – aspects of the policy or program that describe what ‘good’ looks like. And those criteria are determined on the basis of values. In evaluation, values have been defined as:
- “principles, attributes, or qualities held to be instrinsically good, desirable, important, and of general worth” (Stufflebeam, 2001, p. 1); or
- “normative beliefs about how things should be, strong preferences for particular outcomes, or principles that individuals and groups hold to be desirable, good or worthy” (Schwandt, 2015, p. 48).
These are just fancy ways of saying that values are an expression of what matters to people.
Values are verifiable facts. They can be validated “by observation, inference or definition”. In practical terms, values are “real, although logically complex, properties of everyday things” which can be determined to “an acceptable degree of objectivity and comprehensiveness” (Scriven, 1993, p. 9).
When a shared set of values are made explicit, they cannot be dismissed as ‘subjective’. They are inter-subjective – an agreed social construct used by a group of people for an agreed purpose. Other inter-subjective constructs that enable society to function include our democratic, legal and financial systems.
In our evaluation practice, we emphasise working with stakeholders (e.g. people and communities who are supposed to benefit from a program, people delivering it, decision-makers) to identify explicit values-based criteria that provide an agreed and transparent basis for making evaluative judgements. These criteria are the backbone of an evaluation and the means by which it delivers findings that are valid and credible, that get used and leave people better off.
And that is why we talk about explicit values in evaluation.
Adapted from King, J. (2019). Evaluation and Value for Money: Development of an approach using explicit evaluative reasoning. (Doctoral dissertation). Melbourne, Australia: University of Melbourne.
References:
Davidson, E.J. (2005). Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation. Sage.
Scriven, M. (1993). The Nature of Evaluation. New Directions for Program Evaluation, 58, 5-48.
Schwandt, T. (2015). Evaluation Foundations Revisited: Cultivating a Life of the Mind for Practice. Stanford University Press.
Stufflebeam, D.L. (2001, March). Evaluation Values and Criteria Checklist.
September, 2021 / Updated February, 2022
