Why ”by the way …”?
Before reaching this link, I have actually presented all the information there is a need to have in relation to the company, Andante – tools for thinking AB, and of myself. However, as I occassionally think and write about evaluation with no specific purpose, I would like to share some of these notes here.

First, defining ‘evaluation’

As this webpage is about evaluation, it may be appropriate to define evaluation. There are many definitions around, but I like this one and always use it.”Evaluation is the systematic inquiry into the worth or merit of an object”

This is a brief and elegant definition, and it is also the least common denominator of what researchers in the field have been able to agree upon. There are a few things to note about the definition.

First, the definition does not specify that evaluations have to be independent; that is, undertaken by independent, unbiased, experts. Evaluations can also be internal, and they can build on stakeholder participation.

Second, an evaluation is not defined by its purpose. Whether undertaken for control, learning, decision-making, or any other purpose, the study can still be called an evaluation.

Third, the definition does not specify what the object is. An evaluation can assess a project, a programme, an organisation, a policy, or even an object in the grammatical sense (such as an information toolbox, a panel discussion, or a website).

Fourth, the definition does not specify what constitutes worth or merit. This could be goal achievement, efficiency, effectiveness, relevance, beauty, durability, survival capacity, or any other quality, or combination of qualities.

Fifth, the definition does not say anything about how the evaluation is disseminated; whether it should be formally presented, open to the public, or even if it has to be presented in writing. Presumably, an evaluation process could end in a seminar, and it does not necessarily have to lead to a written final report.

Consequently praxis varies, and different organisations choose to specify more precisely what evaluation is in their context. It is, for example, common to specify that evaluations have to be undertaken by independent experts. The definition above is firm in two respects in particular; first that evaluation has to be an assessment of worth or merit. This distinguishes evaluation from research (pure and applied), which does not necessarily have to arrive at a value assessment. Second, evaluation has to be a systematic process of inquiry; meaning that it has to build on the methods of social science research. The assessment must build on a systematic collection and analysis of data.

Second, evaluation policy and the evaluation of policy

Quite a few organisations have established an evaluation policy. After a while it is a good idea to step back and consider whether it was a good policy or not, whether evaluation practice has become different, and if the policy has made a difference?

A policy ususally comes at a cost; time for reflection and decision-making, possibly costs for consultants to develop the policy, time and costs to print and communicate the policy. Could you get a policy for less than USD 50.000? Yes of course, but it is not uncommon that organisations spend much more than so on a policy. Is it worth the money?

What would characterise a ”good” policy? I would suggest some few characteristics: that the policy has a clear purpose, that it is put to use, that something happens as a consequence, and that the overall purpose that led to the policy is actually reached.

How could then a policy be made useful? First, it must be a relatively brief text. There are some elements that the text must contain, at least when it is an evaluation policy: define evaluation, establish why the organisation evaluates, direct practical evaluation in terms of the major design alternatives (external/internal, participatory or not, varieties of design, approach to evidence, organisational framework, as well as budgets and other resources).

Too often, evaluation policies express general goodwill around evaluation, but they don’t pinpoint or guide people on the choices they have to make when they evaluate. ‘The proof of the pudding is in the eating’, and so with evaluation policy. If it is a good policy or not depends on whether it is used – for the better – in practice. Many evaluation policies are tasteless and hence add little of value.