Guide to Evaluation

Evaluation, put simply, is “the process of assessing progress against a series of performance criteria in order to determine: a) the extent to which objectives have been met; b) what outputs, outcomes and impacts project or programme activities have produced; and c) at what cost. Evaluation should explore the effectiveness of procedures and delivery as well as the achievement of outcomes and impacts”.

Evaluation aims to:

• find out how far aims and objectives have been met
• examine the outputs, outcomes and impacts that have been produced
• explore effectiveness of procedures and delivery
• assess cost-effectiveness and value-for-money.

Evaluation differs from monitoring, which tracks how a project or programme is being delivered and is progressing to targets, and reviewing, which explores the management, administration and delivery of a project or programme.

What is the Purpose of Evaluation?

Evaluating projects, programmes or policies enables policy makers or project/programme managers to:

• demonstrate performance
• discover where improvements could be made to design or delivery methods
• identify good practice and lessons for the future

Above all, evaluation is intended to be a positive learning experience.

Evaluation findings impact on policy decisions and on implementation of projects and programmes.

The Welsh Assembly Government  is strongly committed to evaluation as a means of building an evidence-based approach to policy development and implementation.

What can be evaluated?

Evaluations can be conducted of projects, programmes or policies. You are most likely to be evaluating projects or programmes. A project is a discrete piece of work in a particular location - for example, a scheme aimed at raising confidence levels among the long-term unemployed in a small community. A programme is a collection of projects and activities, which are aimed towards achieving a specific common objective. An example would be the Welsh Assembly Government’s training programme.

Skillbuild, Modern Apprenticeships and Modern Skills Diploma are designed for slightly different target groups, but implementing them requires a co-ordinated approach as they are aimed towards similar goals.

A policy is likely to encompass a range of projects, programmes and activities. These may differ in type and have different beneficiaries, but would be aimed towards a general common objective. An example would be a social inclusion policy.

In social research terminology, a project, programme or policy is referred to as an intervention.

What is the Purpose of the Guide?

The aim of the guide is to give you the background knowledge needed to design and brief an evaluation and effectively oversee the process. The responsibility for evaluation lies with the Research and Evaluation Team, within the Business Improvement Resource Investment Group. There are evaluation specialists within the Research and Evaluation Team. These people can assist you if you choose to carry out an evaluation internally.

However, in most cases, external consultants will be contracted to carry out evaluations. For this reason, the processes of conducting the evaluation itself – for example, questionnaire design, administering surveys and analysing data – are not discussed in this guide.