Making Connections

Using a theory of change to develop

planning and evaluation
By Jean Ellis, Diana Parkinson and Avan Wadia
for Charities Evaluation Services
February 2011

Background to this guide

This guide was written by Jean Ellis, Diana Parkinson and Avan Wadia for Charities
Evaluation Services National Performance Programme. The National Performance
Programme is funded by Capacitybuilders National Support Service programme and
is led by Charities Evaluation Services (CES) in partnership with ACEVO, the LGBT
Consortium, nef (the new economics foundation), New Philanthropy Capital and
Voice4Change England.
Improving Support is an initiative led by Capacitybuilders that brings together
practical resources and learning to strengthen support services for voluntary and
community organisations.
CES is a company limited by guarantee. Registered in England and Wales no.
2510318. Registered charity no. 803602. Registered charity in Scotland no.
SCO40937. Registered office address: 4 Coldbath Square, London, EC1R 5HL, UK.
Copyright
Unless otherwise indicated, no part of this publication may be stored in a retrievable
system or reproduced in any form without prior written permission from Charities
Evaluation Services (CES). CES will give sympathetic consideration to requests from
small organisations for permission to reproduce this publication in whole or in part but
terms upon which such reproduction may be permitted will remain at CES discretion.
Acknowledgements
Written by Jean Ellis, Diana Parkinson and Avan Wadia for Charities Evaluation
Services
Production management by Deborah Smith
Many thanks for the detailed case examples provided by:
Mike Mandelbaum (TB Alert) and Milla Gregor (Charities Evaluation Services)
Lucy Tran (Chinese National Healthy Living Centre) and Kathryn Hinds (formerly
Kings Fund)

Charities Evaluation Servcies, 2011

ISBN 978-0-9564349-4-4

Contents
Introduction
This guide
Who this guide is for
The language used in this guide

1
1
1
1

1. What is a theory of change?

The theory of change model
Mapping the change journey
What are its benefits?

2
2
3
4

2. Using a theory of change

A robust process
More realistic planning
Developing learning
Improving communication

6
6
6
8
8

3. How to develop your theory of change

Planning the process
What are the steps?
Clarifying assumptions
Establishing timelines
Planning resources
Getting ready to use your theory of change

10
10
10
20
20
21
21

4. Using theory of change for monitoring and evaluation

Monitoring against your plan
Evaluating your theory of change
Looking for causal connections
Combining theory of change with other approaches
Developing useful monitoring and evaluation

23
23
23
25
26
27

Further information
Further reading and references
Other publications from Charities Evaluation Services

28
28
29

Glossary

30

Appendix 1: TB Alert change story map

32

Appendix 2: A theory of change for mental health advocacy: a case study

37

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Introduction
This guide
This guide is an introduction to the theory of change approach to planning,
monitoring and evaluation. It explains what it is, why to use it and how to use it. It will
help you:
understand what the theory of change is
see how it fits with other models of planning and evaluation
decide whether it is the right model for your organisation
develop your own theory of change.
Chapter 1 provides an overview of the theory of change approach and Chapter 2
explains more about how the approach is used. Chapter 3 is the core chapter: we
take you in greater detail through the several steps of developing a theory of change,
and discuss what you will need to include in it to make it most useful. Chapter 4
describes how a theory of change can serve as a dynamic and comprehensive
framework for evaluating projects and programmes.
The guide illustrates the text by building a theory of change step-by-step for a
fictitious project working with unemployed young people. We also provide links to a
real-life example of theory of change developed by TB Alert and to a case example of
an evaluation using the approach by the Chinese National Healthy Living Centre.
The guide also provides suggestions for further reading and a glossary.
Who this guide is for
Making Connections is aimed at voluntary or community sector organisations wanting
to bring about change whether at an individual, organisation, community or policy
level.
The language used in this guide
Long-term aim is used to mean the overall aim of your project or organisation. For
some organisations this may equate to a mission statement.
Specific aims are broad areas of change that you hope will bring about the longterm aim.
Outcomes are the specific changes, benefits, learning or other effects that happen
as a direct result of your activities or services.
Outputs are all the detailed activities, services and products you do or produce.
Stakeholders are the people or groups who are affected by or who can affect the
activities of an organisation, including users.
Pathways of change show both the links between the specified outcomes and the
links between different outputs and outcomes.
Assumptions refer to the underlying conditions or resources that need to exist for
planned change to occur.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

1. What is a theory of change?

The theory of change model
A theory of change is a description of a social change initiative that shows how early
changes relate to more intermediate changes and then to longer-term change. It can
form the basis of strategic planning and, as a blueprint for the work ahead and its
likely effects, it can be used for management and decision making as a project or
programme develops and progresses.
It can also reveal what should be evaluated, and when and how, so that project and
programme managers can use feedback to adjust what they do and how they do it to
achieve the best results. A theory of change methodology will also help to identify the
way people, organisations and situations change as a result of an organisations
activities or services, helping to develop models of good practice.
Theory of change concepts began to emerge from realistic evaluation techniques in
the 1970s and 1980s, driven by the need to deal more effectively with context issues
and complexity when evaluating larger social programmes. The term theory of
change emerged in the early 1990s. 1
A theory of change is particularly helpful when you want to think through all the
different elements required for a successful complex initiative. However, a theory of
change model can be a flexible tool and be useful for more straightforward projects
as well as for more complex programmes. A theory of change may be described in
outline or in greater detail and complexity. It may be stated as a narrative and in a
variety of ways visually.
The CES Planning Triangle, used widely as a starting point for project planning and
evaluation, is a basic theory of change. It is worked through by clarifying the specific
aims and outcomes that will be required to achieve long-term aims or impact. Once
specific aims have been identified, objectives (and their outputs) are agreed that will
directly link to, and will result in, desired change.
The CES website illustrates fully worked triangles for different types of projects.
A fully developed theory of change will spell out more clearly the sequence in which
outcomes are likely to happen, how early and intermediate outcomes lead to end
outcomes, and how they connect to each other. (There is an illustration of
intermediate outcomes on page 14). Sometimes outcomes are closely related, or
they may occur independently. These changes and connections are often
represented visually, for example through a chart, or a set of tables.

For realistic evaluation, see Pawson, R and Tilley, N (1997) Realistic Evaluation, Sage Publications,
London. In the US, the theory of change approach (ToC) was developed by the Aspen Institute
(Fulbright-Anderson, Kubisch, and Connell, 1998).

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Mapping the change journey

This guide provides an example of a straightforward project with unemployed young
people, and illustrates a theory of change in its different stages of development.
Diagram 1 below shows how the project can map a sequence of outcomes that will
achieve its long-term aim, To reduce youth unemployment in the local area.
Diagram 1: Developing a sequence of outcomes

In the diagram above, the arrows between outcomes show the direction of change.
However, the theory of change is mapped backwards.
Once the longer-term aim and the desired outcomes are clear, you can agree the
activities and outputs that will achieve change.
As you start thinking through the activities and outputs necessary to deliver the
results, you may identify other groups that you will need to work with to achieve the
desired outcomes, and your model will become more detailed as these preconditions
for change are added. This is shown in Diagram 2 on page 4.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Diagram 2: How change happens developing the theory

In order to be really useful as a planning tool and evaluation model, the theory of
change will also identify:

the assumptions that are being made. These could be about the effectiveness of
specific models of service delivery, or about the context in which you are working.

the resources you will need to carry out your activities

the collaboration or points of contact you will need with other agencies

the timelines attached to your activities, delivery of outputs and different levels of
change.

These different components of a theory of change are discussed further in Chapter 3,

How to develop a theory of change.
What are its benefits?
A theory of change puts emphasis on what you want to achieve rather than what you
want to do. It offers a theory of causal connections that can provide a firm basis for
deciding on particular activities, delivery models and outputs. It suggests plausible
expectations of achievable outcomes. It requires you to bring to the surface and think
about underlying assumptions and to test them.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

In the US, ActKnowledge, in partnership with the Aspen Institute Rountable on

Community Change, has developed training materials, examples, and a website
(www.theoryofchange.org) to build the capacity of organisations working in the social
change field to do better planning and evaluation by being clear about what they
expect to accomplish and why. 2 The website says that working through a theory of
change process will provide:

a clear and testable theory about how change will occur, which can give you a
framework for monitoring and evaluation, and provide the basis for both
accountability and learning
a visual representation of the change you want to see and how you expect it to
come about
a blueprint for evaluation with measurable indicators of success identified
an agreement among stakeholders about what defines success and what it
takes to get there
a powerful communication tool to capture the complexity of your initiative. 3

Chapter 2 discusses in greater detail how you can use a theory of change.

ActKnowledge is developing an online tool that will help organisations build theory of change models.

Source: http://www.theoryofchange.org/background/benefits.html

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

2. Using a theory of change

A robust process
External evaluators quite often find that those planning projects and programmes
have selected outcomes that are inappropriate to their work. The advantage of
spending time on a theory of change when planning an initiative is that the exercise
can be used to bring to the surface, and to challenge if necessary, the assumptions
about how people expect certain outcomes to be achieved.
If this process is shared between trustees, project personnel, funders and other
stakeholders, this can result in more appropriate shared expectations.
Once you have developed your theory of change, you can use it to:
plan
evaluate
communicate the journey of change for your work.
If a theory of change is not developed when a project is planned, it can be done at a
later stage, as part of checking progress, or the first stage of a more thorough
evaluation
More realistic planning
A theory of change exercise can be an ideal opportunity to get stakeholders really
involved in your work. It provides an opportunity for all those involved to assess what
they can influence and what level of change they can realistically expect to achieve
within the time and resources available.
Organisations sometimes find that there is a gap between the level at which they are
working with users and the evidence of impact that their funders want to see
demonstrated a gap between project level and higher-level outcomes. A theory of
change might help you show how your project, working at a local level and on a
relatively small scale, can contribute to the broader picture.
For example, if our illustration project wants to contribute to the longer-term aim of
reducing youth unemployment, but is working with younger children, aged 9-13
years, providing an intervention at transition to secondary school, it may feel that it
can not realistically be accountable for achieving this longer-term aim. There will be
too many other individual, family or other context factors, both positive and negative,
that will affect longer-term change. However, it would be valid for the projects theory
of change to show the links between the outcomes that it can directly influence
through its own work, and those positive longer-term changes to which it might
contribute, as shown in Diagram 3.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Diagram 3: Keeping aims realistic demonstrating the link

A theory of change will also give you a detailed picture of the steps needed to
achieve an aim and to identify what services to design or outputs to deliver, when,
and to whom, and what resources you will need. This will provide a sound basis for
writing a business plan or producing a detailed action plan about how you are going
to deliver your work.
Using a theory of change for planning a campaign
Campaigning organisations are generally clear on their strategies, tactics and their
long-term aims. These long-term aims are often expressed as policy changes,
environmental conditions or changes in the wellbeing or quality of life for specific
groups of people. Meaningful evaluation of campaigning work requires definition of
what happens between the implementation of strategies and the ultimate long-term
outcomes or impact.
In order to achieve a policy change or set an agenda, organisations may identify a
number of strategies including campaigns, collaborations, public awareness efforts,
community mobilisation efforts and so on.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Through a theory of change, campaigns can plot the different arenas of change that
they want to alter, and how they relate to each other:
Strengthened organisational capacity: skills, staffing and leadership
Strengthened alliances: level of coordination, collaboration and mission
alignment
Strengthened base of support: the grassroots, leadership and institutional
relationships and alliances
Improved policy: stages of policy change in the public policy arena, including
adoption, implementation and funding
Shift in social norms: the knowledge, attitude, values and behaviours
Changes in impact: the ultimate changes in social and physical lives and
conditions. Impact is affected not just by policy change, but by other strategies,
such as community support and changes to behaviours.
Adapted from Organizational Research Services for the Annie E. Casey Foundation (2009) Getting
Started: A Self-Directed Guide to Outcome Map Development Exercise, Seattle.

Developing learning
A theory of change can provide a framework against which you can analyse your
projects progress and its unfolding effects. Your theory will identify specific aspects
of your work, allowing research questions to be developed around them; these will
provide the basis for an evaluation. When you evaluate, it will allow you to question
what it is about a specific project or intervention or way of working that causes or
triggers change for individuals, for organisations or for policy change. It can also
question what other sets of conditions or circumstances, apart from project activities,
will be necessary for desired outcomes to occur.
Whether the evaluation is carried out internally or externally, if project personnel are
involved in establishing a theory of change, it can encourage a greater culture of
analysis and learning within the organisation.
Chapter 4 discusses using a theory of change for evaluation in greater detail.
Improving communication
Once you have established a well-thought through process and have identified the
journey of change, you will be better equipped to make your case with funders for
investing in your initiative, and will have the basis for sharing a common language
and understanding.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Using a theory of change for funding social initiatives

Theory of change models are increasingly being used by some funders:
to design their funding programmes
to help funded organisations design projects
to provide a basis for reporting by funded organisations.
A theory of change can be particularly helpful when the desired programme change
is difficult to define or quantify. It can also be a useful tool to think about the level of
impact, influence and leverage that can be reasonably expected with the financial
resources available for a programme.
A theory of change set out for the SOAR Opportunity Fund in the US shows the
theory of change for a programme of supporting quality of family, friends and
neighbour care. It identifies change at three levels:
Impact on wellbeing for children, families and communities directly served by
grants
Influence on policies, regulations and systems
Leverage in terms of public and private investment in improving outcomes for
children and families.
If funders work with funded organisations on a theory of change, it may change the
relationship with them. It will require a closer partnership and more transparency
about weaknesses and problems than most organisations are used to.
Source: Organizational Research Services for the Annie E. Casey Foundation (2004) Theory of
Change: A Practical Tool for Action, Results and Learning, Seattle.

An evaluation framed around a theory of change will provide a powerful tool for
influencing policy makers and to help contribute to further debate about good
practice.
The time you invest in developing your theory of change will depend in some
respects on the complexity of your project or programme. The process can be quite
time-consuming, potentially involving research into similar initiatives to help you
develop your theory, and bringing together a group of people with insight into your
work. Above all, it will require commitment and an organisational culture that values
learning and allows time for it. Chapter 3 describes this process through a series of
five key steps.

Charities Evaluation Services

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

3. How to develop your theory of change

Planning the process
Before you begin work on developing your theory of change, there are a number of
points you should consider:

Why do you want to develop a theory of change? Who will the target audience
be? How you will use it?
What will be the scope of your theory of change? Will it be a simple change model
or a more detailed, complex one?
How much time can your organisation commit to the process, including adequate
time to consult with your stakeholders?
Who will lead the work (a lead person or team with adequate support)?
Do you have adequate skills in house, or do you need an external consultant to
work with you to facilitate the process and provide technical support?
How will your stakeholders be involved? You will need to clarify what is expected
of them, how long the process will last, and how you will use any information
gathered.

Your developed theory of change should be:

Credible. It should be based on previous experience and insight from your

different stakeholders or relevant research where appropriate.
Achievable. You will have the necessary resources to carry out the intervention
as detailed.
Testable. The model will provide clear, specific and comprehensive indicators
that you can use to assess progress and achievements.
Supported. Your stakeholders will be involved in defining and agreeing your
theory of change and you will have a common language with which to go forward.

Your theory of change is likely to include a map, or series of maps, of your aims,
outcomes and the activities and outputs that you believe will lead to your long-term
aim. This may be accompanied by a list of assumptions and a narrative
demonstrating the linkages between different elements of the model.
What are the steps?
Your process will depend to a certain extent on whether you develop a theory of
change when you are planning your project or programme, or whether you are
developing it retrospectively for evaluation purposes. You should also be guided by
your available resources and what is realistic in terms of the change you are aiming
to achieve.

Charities Evaluation Services

10

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

The process will include the following steps:

As you move along these steps, you will need to clarify and make explicit the
assumptions that lie behind your theory. This is discussed further on page 20.
Some outcomes may be outside the direct influence of any one organisation, or any
one intervention. Once this question is thought through, you might suggest ways of
working together with other agencies that can influence those changes that will be
important to the achievement of your targeted outcomes.
Change frameworks usually go through many revisions. Outcomes may be added,
moved and deleted until a map emerges that tells a story the group can agree on.
This discussion is an important part of the process.
Step 1: Identifying the problem
Your starting point is an analysis of the problem you are trying to address and its
underlying causes. You can do this by:

using research either commissioning your own research or referring to other

peoples research

consulting with your users for example, by holding focus groups or sending out
a survey

drawing from your own previous experience and that of your stakeholders.

Charities Evaluation Services

11

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

It is important to identify the causes of the problem, as this will suggest the step-bystep changes that you want to see. For example, if one element of the problem of
youth unemployment is that young people become discouraged and de-motivated by
their experience of looking for work, your theory may suggest developing a stronger
sense of their potential place in a working environment; in order to do that young
people should gain some experience of work placements.
Step 2: Agreeing your aims
Agreeing your long-term aim
The main problem you are addressing should be mirrored in its ultimate resolution
your long-term aim. This long-term aim is the starting point for your theory of change.
Summarise your long-term aim in a clear statement. This should describe the broad
difference you want to make. In our illustration project, where the problem is youth
unemployment: Our long-term aim is to reduce youth unemployment in the
local area.
You may also want to think about how your long-term aim fits in with local strategic
priorities or funding programmes, so that you can place your work in a wider context.
This may involve looking for relevant research or identifying connections between
your work and those broader priorities.
There is more about developing aims in CES publication, First Steps in Monitoring
and Evaluation.
At this stage, you are likely to have a start point (the problem and identified needs)
and an end point (long-term outcomes), and some general ideas about the sort of
activities the project might carry out. This is illustrated in Diagram 4.
Diagram 4: First steps in a theory of change

Agreeing specific aims

You will then need to work back from your long-term aim to think about the changes
(specific aims) that need to happen in order to achieve this longer-term change.
Use your more detailed problem identification to define broad areas of change that
will help you to address the issues you have identified. Try to think about all the
different ways in which people might change:

Charities Evaluation Services

12

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

situations (for example, housing, employment)

wellbeing (for example, mental or physical heath)
behaviour (for example, involvement in crime, drug use)
attitudes and feelings (for example, how people feel about themselves or others)
skills and abilities (for example, communication skills or ability to work under
pressure)
awareness and knowledge (for example, awareness of rights)
relationships (for example, with peers, family or at work).

Think also about the ways in which situations might change:

policy (for example, changes in immigration law)

environment (for example, better access to green spaces)
services (for example, new services or services delivered in different ways)
ways of working (for example, new partnerships developed).

Your theory will show how each of your specific aims links with your long-term aim
and how the project can contribute to, if not achieve, long-term outcomes.
Identifying specific aims
In order to achieve our long-term aim, we will need to:

increase the levels of motivation and aspiration in the young people

develop young peoples job-related skills and experience
enable young people to be work ready
develop commitment by local employers to providing greater training opportunities
for young people.

Charities Evaluation Services

13

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Step 3: Mapping outcomes

Once you have identified your long-term and specific aims, the next step is to work
backwards from each of your specific aims. At this point, you are beginning to plot
the change process in much greater detail, thinking about the different steps along
the journey towards each specific aim.
Take each specific aim in turn and think about the changes needed to achieve them.
These will be intermediate outcomes. There is more about intermediate outcomes in
the CES guide Your Project and its Outcomes.
Identifying intermediate outcomes
To achieve our specific aim, To enable young people to become more work ready,
young people will:

have a better understanding of what being in work means

become more aware of how they will need to improve their own skills or
knowledge, or perhaps change their behaviour
have more knowledge and skills in work-related areas, such as looking for work or
managing money
know how to behave in the workplace
be more confident of their place in a working environment.

Some of the young people will need to improve basic skills such as literacy,
numeracy and IT skills.
Once you have identified outcomes, you can start to work out the order in which they
will need to occur. For each outcome you should think: What change will be needed
before this can happen? How does this outcome relate to other changes? These
links between different outcomes are the pathways of change.
A pathway of change
Before young people are ready to gain experience in the workplace, they will need to
have an understanding of how to behave in the workplace and to have developed
appropriate skills. Some young people will also need to have improved their basic
skills.
Change may be complex; outcomes may occur independently of each other or they
may be linked. As you chart this, try to show these relationships. The flowchart in
Diagram 5 (on page 16) illustrates how you might do this.

Charities Evaluation Services

14

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Your outcome map will show the pathways; it may be a visual diagram that depicts
relationships between initiatives and intended results. These outcomes may reflect
changes at different levels:

individual
family
community
organisations
policy.

Changes usually start at the level of the individual and if this foundation is not put in
place, long-lasting change is unlikely to happen. For example:

Change for the individual: in awareness, knowledge and skills.

Change for the family: changes in behaviour or financial status.
Change for the community: increase in neighbourly behaviour; decrease in petty
and violent crime.
Organisational change: more sharing of approaches and good practice;
collaboration.
Policy change: increased awareness of issues; specific policy changes.

Diagram 5 on the next page shows outcome mapping at the level of the young
people in the illustration employment project.

Charities Evaluation Services

15

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Diagram 5: Mapping outcomes

Charities Evaluation Services

16

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Step 4: Identifying activities and outputs

You are now ready to start thinking about what services and outputs will help you to
bring about the outcomes you have identified. For a new piece of work, this will
involve thinking creatively about the outputs that will be most effective in bringing
about your desired outcomes and when they are best delivered.
Identifying activities and outputs
In order to increase young peoples knowledge about working, we will need to run a
series of workshops and talks, focusing on:

job-searching, CV writing and interview skills

employer expectations (such as being on time, dressing appropriately, getting on
with colleagues)
travelling to work and managing money.

Remember that some outputs will involve collaboration with other agencies, and
some outcomes may only be achieved if other services are also involved in some
way, so these contact points or joint activities will need to be charted as part of your
theory.
Diagram 6 on the next page shows how outputs can be added to your flowchart at
relevant points, linking to the outcomes that they will produce.
If you are creating a theory of change for an existing project, as you plot your existing
activities and outputs, this may lead to some discussion about how well outputs are
delivering your anticipated outcomes, and it may lead in turn to some revision of what
you do, and how you do it.

Charities Evaluation Services

17

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Diagram 6: Linking outcomes and outputs

Charities Evaluation Services

18

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Step 5: Specifying indicators and targets

The final step in developing your theory is to identify indicators relating to the
different elements in it. Indicators are measurable pieces of information that will help
you to assess how your organisation is carrying out its work and achieving its aims:

Output indicators help you to assess the delivery of your services and outputs,
for example, the number of workshops you deliver.

Process indicators help you to assess how you are implementing your work, for
example, how well you collaborated with partners.

Outcome indicators help you to assess progress towards achieving your aims,
for example, the number of people who increase their skills or knowledge.

Defining indicators
Our indicators will include:

number of young people making contact with the project

number of young people drawing up action plans
level of motivation among young people to find and gain work
number of young people gaining work experience
number of young people gaining permanent jobs
type of jobs taken up by young people
the level of engagement by employers
the type of support provided by one-to-one sessions
the type of work experience gained by young people.

You may want to think about specifying targets relating to your outputs indicators,
that is, the level that you consider necessary as the precondition for achieving the
next level of change.
Specifying output targets needed for change
For young people to be in a position to gain from their work experience, they will
need to attend at least three of the introductory workshops we run.
You may also want to set targets related to your outcome indicators, such as the
number of young people attending interviews, gaining jobs, and numbers remaining
in permanent employment.
For more information on developing and using indicators, see Keeping on Track: A
Guide to Setting and Using Indicators, available from CES.

Charities Evaluation Services

19

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Clarifying assumptions
As you go through the process of defining your long-term aim and specific aims, you
will find that you have made a series of assumptions about these and about your
projects potential to achieve its specified outcomes.
Discussing what needs to be in place in order for your long-term and specific aims to
be achievable is an important aspect of developing your theory of change.
Recognising these assumptions will help you identify some of the critical factors that
will affect the success of your work. It will also help you to shape how you deliver
your services. You will want to test out the assumptions when you evaluate your
work, so it is important to document them.
There are different categories of assumption; the important ones are likely to be
about the following:

the connections between the long-term, intermediate and early outcomes

the links between programme activities and the outcomes they are expected to
produce
the contextual or environmental factors that may influence whether outcomes are
achieved.

Clarifying key assumptions

Our assumptions cover a range of different issues and include the following:

The target group of young people will respond to outreach and engage with the
project.

Young people will have sufficiently stable lives to engage productively.

Families will get involved and be supportive.

The model of peer support will be inspirational.

One-to-one work will build confidence and increase learning.

Work experience will provide appropriate skills to match available jobs.

There is sufficient buoyancy in the job market to take on more workers.

Available jobs will be permanent positions.

When you are developing the first elements of your theory and putting forward
assumptions, it may be useful to base these on the literature, existing practice or
expert knowledge. On the other hand, you may be testing a completely new way of
doing something and this should be clear in your theory.
Establishing timelines
As your theory of change takes shape, you will be thinking about when your various
activities and outputs will come on stream, and when intermediate and longer-term
outcomes are likely to happen. It is important to write in these timelines. This should

Charities Evaluation Services

20

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

shape expectations of what can be monitored and when, and will help in planning
review and evaluation timetables.
Planning resources
If you think through the activities associated with the delivery of your outputs, this will
help you to plan the resources you will need and to set a budget for the intervention.
Remember to bring to the surface assumptions about the amount of staff and
volunteer time that will be needed or about the level of skills that will be necessary
and available to you. When you evaluate, one question to consider may be whether
appropriate resources were planned and delivered.
Getting ready to use your theory of change
As you develop your theory of change, you may revise it many times testing out
your assumptions and adding, moving or deleting outcomes - until you have a theory
that everyone agrees with. The discussions involved in this are crucial as they help
people to be really clear about what they are trying to do and why.
Your theory of change may now be presented in a chart, a set of tables or a spiders
web or starburst. It will be helpful to write the theory as a narrative as well. This will
mean drawing out the connections and assumptions contained in your visual
representation and expressing your theory in normal language.
A narrative might include:

background: the context and the need

long-term outcomes or impact: the ultimate desired change

intermediate outcomes: how these outcomes are important for themselves and
how they contribute to the ultimate impact

assumptions and justifications: the facts or reasons behind the features of the
initiative

the resources you will need to carry out your activities

the collaboration or points of contact you will need with other agencies

the timelines attached to your activities, delivery of outputs and different levels of
change: the initiatives activities and programmes. 4

Adapted from ActKnowledge and the Aspen Institute Roundtable on Community Change (2003)
Theory of Change: Guided Example: Project Superwoman
(http://www.theoryofchange.org/pdf/Superwomen_Example.pdf)

Charities Evaluation Services

21

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

TB Alert: Using a theory of change as part of the project planning process

Background
TB Alert is a charity working in the UK and overseas to help fight TB. The UKs
Department of Health has given TB Alert a lead role in raising awareness of TB
among primary care health professionals and at-risk communities.
TB Alert is pushing for the adoption of a social model of health rather than the
dominant clinical public health model, and works by enabling local partnerships
between public sector (NHS and local government) and third sector organisations.
TB Alert has two Community Development Officers (CDOs) based in Manchester and
London running pilots across England. In addition TB Alert has created The Truth
About TB programme (www.thetruthabouttb.org).
TB Alert recognised the need for a structured approach to monitoring and evaluation
in order to provide evidence of outcomes and to demonstrate learning. Programme
staff worked with CES to develop planning materials and to build their self-evaluation
capacity, so that monitoring and evaluation would be part of the project throughout.
Developing the theory
CES worked with TB Alert through workshops and follow-up to develop its theory of
change.
First, a stakeholder workshop gathered stakeholders views on programme priorities,
outcomes, outputs, indicators and tools. Stakeholders included TB Alert personnel,
the Department of Health and the NHS, other health charities such as those working
in HIV, and wider voluntary sector partners.
During a second workshop a brainstorm with programme staff was developed into a
story map a large-scale theory of change, which CES was able to use as the basis
for developing a full monitoring framework and information collection tools. As TB
Alerts programme moves out of the pilot phase, the theory of change will be
reviewed, updated and improved.
TB Alert managers and CDOs were fully involved in the process building their
capacity through learning by doing. As the CDOs develop further TB awarenessraising work they should be able to use these planning methods to support their third
sector partners monitoring, evaluation and learning activities.
TB Alerts full story map is set out in Appendix 1.
This chapter has taken you through the process of developing your theory of change.
Chapter 4 describes how your theory of change can be used as a framework for
monitoring, dynamic evaluation and useful learning.

Charities Evaluation Services

22

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

4. Using theory of change for monitoring and evaluation

Monitoring against your plan
Once you have completed your theory of change, you can use it as a framework to
support your project or programme monitoring:

It will help you check progress against milestones.

You can check what really happened against your map and document lessons
learned.
Your indicators will suggest what routine data to collect.
You can use it as a basis for reports to trustees, funders and other stakeholders.

The timelines you established when planning will be important and indicate when to
expect change to occur and when to monitor it. Some outcomes may occur quite
early on, while others may take several years. You can use the theory of change as a
basis for setting out a monitoring and evaluation framework, which will provide a plan
of action for your self-evaluation, and will also indicate when you might have an
external evaluation carried out; it will also provide the starting point for briefing your
evaluators.
There is more guidance on collecting outcomes data in the CES publication
Assessing Change: Developing and Using Outcomes Monitoring Tools.
Evaluating your theory of change
Ideally, you should plan some level of evaluation early on, rather than leaving it to the
end of the project, so that lessons can be fed back into your project or programme
delivery. 5
Your central evaluation questions are likely to include the following:

Was the central theory a valid one?

What outcomes, planned or unexpected, occurred for the target group and for
others?
Were our assumptions, for example, about the enabling environment, and the
links between outputs and outcomes, borne out?
What was important in the way that we did things? What activities were valued
and gave positive results? What needed to be different?

The theory of change will be constructed around a central proposition. When you
bring your data together and evaluate, above all, this is what you will be testing, as
well as the associated assumptions that you made, assessing whether your theory
has been confirmed or how it might be reformulated.

This is called formative evaluation, where evaluation findings are used for management purposes
to improve delivery or review expectations.

Charities Evaluation Services

23

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

The central theory (or proposition) for our project working with unemployed young
people (described in Chapter 3) is shown in the box below.
Clarifying the central theory
Sustainable jobs are potentially available locally. The conditions for challenging youth
unemployment in the area can be created by developing new opportunities for young
people without jobs. Those opportunities can be stimulated by engaging local
employers in building confidence and job-related skills in young people. Young
people learn new skills most readily in a supportive, one-to-one mentoring and
coaching environment, and confidence is likely to be built with both peer and family
support.
Sometimes the theory revolves around quite specific approaches to social change.
We can take an example of a different sort of social action to illustrate this. One
project approach to resolving tensions and conflict in a community might be to use
mediation processes, focusing on specific problems and events, and seeking
agreement around changed behaviours. An alternative approach might be to build
mutual understanding of differences and cooperation through more self-directed
community conversations. These imply different theories about how change
happens, which an evaluation might test.
Testing against your central theory will involve collecting data against output and
outcome indicators in order to check whether things happened as planned and
whether expected change took place. This is illustrated in the following example of
evaluation questions for the youth unemployment project.
Testing against the central theory
How many young people found work?
How confident did they feel?
How well did they accommodate to the workplace?
What skills and experience had they developed?
How work ready did their employers think they were?

Was there anything else they needed to help them become more work ready?
Did the jobs available meet the expectations of young people?

When you collect your data, your methods should allow you to get both quantitative
and qualitative data. They should also permit you to explore things that happened
and outcomes that you had not expected or planned.
This testing will also mean evaluating the processes involved in implementing the
project and the assumptions you made about links between outputs and outcomes.

Charities Evaluation Services

24

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Testing the links between outputs and outcomes

Did employed young people who acted as mentors give the support and
encouragement anticipated?
Did local employers provide work experience that was relevant and valued by the
young people?
Were the workshops sufficiently targeted and did they meet individual needs?
What activities did participants most associate with immediate benefits from the
project?
Looking for causal connections
You will also need to formulate evaluation questions to test out other underlying
assumptions that you made. In our project with young people, these covered a
number of different elements in the project, such as the initial responsiveness of the
young people and their ability to engage, and the importance of family and peer
support. The project assumed that the work experience would provide a vital link
between the participants and more permanent jobs.
When you assess what happened against these elements in your theory, there will
be related questions. For example, how important was the type of work experience
and the quality of support offered by employers to the development of young peoples
motivation and confidence?
Participants own experience, attitude, previous education history and skills may also
be important factors that can affect change. So the evaluation might see what the
data suggests about the possible link between those individual circumstances and
results.
Finally, the evaluation should take into account the assumptions that were made
about the external environment. In our employment project, there was an underlying
assumption that sustainable jobs were available if young people were more work
ready and could take them up. The evaluation will need to look at the kind of jobs
that young people found and the type of local jobs available for young people more
generally and put this in context. Had local enterprise been developing, or were
businesses closing or reducing their workforce? Were there wider government
policies creating an effect which interacted with the project?
There will be some factors that were within the control of the project and others that
were not; these should be identified as they may be critical when you reformulate
your theory of change as a result of your evaluation.

Charities Evaluation Services

25

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

The Chinese National Healthy Living Centre: Using a theory of change for
evaluation
The Kings Fund, in its Partners for Health programme, 2005-2010, developed an
approach founded in realistic evaluation. The programme put considerable resources
into training organisations in evaluation through a two-day workshop during the
application process, emphasising the importance of establishing research questions
and identifying user pathways towards outcomes. Funded organisations were
encouraged to collect information about context as well as outcomes, so that
evidence could be used by other health providers.
The Chinese National Healthy Living Centre was funded for three years to provide
mental health advocacy to Chinese users. The Centre developed a theory of change
using three categories:

Contexts (including individual social circumstances)

Mechanism (including project resources)
Outcomes (the anticipated changes).

There were two components to the evaluation: a quantitative component derived

from case notes and a qualitative case-study component consisting of interviews with
clients, clients relatives and care providers to address specific research questions.
The contexts, mechanism and outcomes were revisited in the light of the research
data, and as a result the theory of change was refined and re-written.
A detailed case study of the theory of change process adopted by the Chinese
National Healthy Living Centre and its evaluation findings is attached as Appendix 2.
Combining theory of change with other approaches
Evaluating against your theory of change will allow you to trace cause and effect
(although not to offer conclusive proof) as you assess whether the theory held good,
testing against each part of the change journey that you mapped. You will not be able
to generalise from your findings, but you will be building knowledge about how
change happens.
The quantitative data you obtain about your outcomes may be an important part of
your evaluation findings as well as the story that is told about pathways to change
and the analysis of process and testing of assumptions. If the project or programme
is dealing with a sufficient number of participants and has related quantitative

Charities Evaluation Services

26

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

outcome data, you may also be able to use statistical analysis and test for the
significance of the perceived results. 6
If your outcomes data is solid, you can also combine using a theory of change with
an approach that assesses social value, such as social return on investment (SROI).
SROI was developed from traditional cost-benefit analysis and from social
accounting. The methodology puts a monetary value on the social and environmental
benefits of an organisation relative to a given amount of investment. An analysis of
inputs, outputs, outcomes and impacts leads to the calculation of a monetary value of
those impacts, and finally to an SROI ratio or rating. For example, an organisation
might have a ratio of 4 of social value created for every 1 spent on its activities.
A number of different models have been used to assess social value, but the SROI
methodology puts stakeholder involvement at its heart, and also acknowledges the
complex interaction between different interventions and contexts in creating
outcomes. Although theory of change focuses on how an intervention works and in
what circumstances and, by contrast, the end point of an SROI is a monetary
calculation, the two approaches can work together.
Developing useful monitoring and evaluation
If you monitor and evaluate your project or programme using a theory of change you
will have a framework against which to gather data and look for explanations for what
occurred as the initiative unfolded. This will help you to be accountable to those who
have invested in your work. It will also allow you to be accountable to other
stakeholders, including those whom the initiative is intended to benefit. Monitoring
and evaluation at specific implementation points will also help you to manage your
work and adjust what you are doing to get the best outcomes.
Incorporating an analysis of process and implementation against your theory of
change blueprint will help you to better interpret your project or programme results.
Voluntary and community organisations are increasingly looking for ways to capture
the value of their work. Understanding how a service or practice model is delivered
and achieves its positive outcomes is essential for its replication; using a theory of
change approach in your planning and evaluation can help this understanding. This
will make your evaluation more useful for others implementing similar initiatives or
developing policy and, more generally, allow you to contribute to a wider body of
knowledge.

These tests can be done with some quantitative data expertise and computer software, such as
SPSS. See for example Bryman, R and Cramer, D (2005) Quantitative Data Analysis for SPSS 12
and 13, Routledge.

Charities Evaluation Services

27

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Further information
You can find further information on the theory of change on the following websites:
ActKnowledge: www.actknowledge.org
ActKnowledge has a website (www.theoryofchange.org) devoted to the use and
practice of the theory of change methodology. Theory of Change Online will soon be
available to provide online support for building a theory of change.
The Aspen Institute Roundtable on Community Change: www.aspenroundtable.org
Further reading and references
Author and title

Link

Anderson, A (2005) The

Community Builders Approach
to Theory of Change: A Practical
Guide to Theory Development,
The Aspen Institute, Washington
DC

www.ccitoolsforfeds.org/doc/commbuildersapproa
ch.pdf

Fulbright-Anderson, K, Kubisch,
AC and Connell, JP (eds) (1998)
Approaches to Evaluating
Community Initiatives Volume 2:
Theory, Measurement and
Analysis, The Aspen Institute,
Washington DC

http://www.aspeninstitute.org/publications/newapproaches-evaluating-community-initiativesvolume-2-theory-measurement-and-analysi

Mackinnon, A and Arnott, N

(2006) Mapping Change: Using
a Theory of Change to Guide
Planning and Evaluation, for
Grantcraft

www.grantcraft.org/dl_pdf/theory_change.pdf

Organizational Research
Services (2004) Theory of
Change: A Practical Tool for
Action, Results and Learning,
prepared for the Annie E. Casey
Foundation

www.organizationalresearch.com/publications/aecf
_theory_of_change_manual.pdf

Pawson, R and Tilley, N (1997)

Realistic Evaluation, Sage
Publications, London

http://www.uk.sagepub.com/books/Book205276

Charities Evaluation Services

28

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Other publications from Charities Evaluation Services

These CES publications provide further guidance on monitoring and evaluation
methodology:
Guide

Description

Assessing Change:
Developing and Using
Outcomes Monitoring
Tools (2010)

Practical handbook to help you identify and develop ways to

collect information on the outcomes of your work.
Free to download at www.ces-vol.org.uk/assessingchange

Discussion papers

CES' series of papers on evaluation covers seven important

topics, including Outcomes monitoring and Assessing impact

First Steps in
Monitoring and
Evaluation (2002)

A basic guide for organisations who are looking at

monitoring and evaluation for the first time.
Free to download at www.ces-vol.org.uk/firststepsme

Keeping on Track: A
Guide to Setting and
Using Indicators
(2008)

Guide to help third sector organisations improve their skills

in setting output, outcome and process indicators

Next Steps: Monitoring Practical guide to help organisations develop their

and Evaluation on a
monitoring and evaluation approach and improve their
Shoestring (2011)
effectiveness on a limited budget.
Free to download at www.ces-vol.org.uk/shoestring
Practical Monitoring
and Evaluation: A
Guide for Voluntary
Organisations, 3rd
edition (2009)

Comprehensive guide to monitoring and evaluation in the

voluntary sector, now in its third section

Using ICT to Improve

your Monitoring and
Evaluation (2008)

This workbook covers the steps and issues you need to

consider in developing appropriate computer systems that
will help you monitor and evaluate your work

Your Project and its

Outcomes (2007)

A booklet for trustees, staff and volunteers who are involved

in planning, monitoring and evaluating their project. The aim
of the booklet is to help you describe the outcomes you want
your project to achieve
Free to download at www.ces-vol.org.uk/yourproject

For full details of our publications and prices, please visit

www.ces-vol.org.uk/publications

Charities Evaluation Services

29

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Glossary
Term

Description

Assumptions

The underlying conditions or resources that need to exist for

planned change to occur

Backwards mapping

The process of starting with a long-term aim and working

backwards to the earliest changes that need to occur

Evaluation

The process of using monitoring and other information to

make judgements on how an organisation, project or
programme is doing

Impact

There are different definitions of impact. It is often seen as

the change, effect or benefit that results from the services or
activities at a broader or higher level than an outcome.
Others use it to mean the same as outcome

Indicators

Signs or clues that you collect information on and that help

you to assess how your organisation is carrying out its work
and achieving its aims.

Intervention

Project-level or programme-level activity carried out in order

to bring about specified outcomes

Long-term aim

The overall aim of your project or organisation. For some

organisations this may equate to your mission statement

Monitoring

The routine, systematic gathering and recording of

information for the purpose of checking an organisations or
projects progress against its plans.

Narrative

A summary of a project or programme theory of change that

explains the pathways of change, highlights some of the
major assumptions and tells the story about how outcomes
and long-term change will be achieved

Outcomes

The actual changes, benefits, learning or other effects that

happen as a result of services and activities provided by an
organisation or project

Outputs

The activities, services and products provided by an

organisation

Pathways of change

Pathways of change show the links between the specified

outcomes and the links between different outputs and
outcomes

Pilot

A way of testing out the effectiveness of a new system by

applying it to a small group and getting feedback on the
process

Charities Evaluation Services

30

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Process

The method, or step-by-step description, of how a task or

activity is to be done

Social Return on
Investment (SROI)

SROI is a framework for understanding, measuring and

managing outcomes and impacts. It is based on involving
stakeholders in determining the relevant outcomes and puts
financial values on the significant changes identified by
stakeholders.

Specific aims

Broad areas of change that you hope will bring about your
long-term aim

Stakeholders

The people or groups who are affected by or who can affect

the activities of an organisation. This can include staff,
volunteers, users, customers, suppliers, trustees, funders,
commissioners, donors, purchasers, investors, supporters
and members

Target group

The main group or groups you are working with and the
people your organisation or project is aiming to benefit

User

A beneficiary of an organisation or project

Charities Evaluation Services

31

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Appendix 1: TB Alert change story map

Introduction
TB Alert works in the UK and overseas to help fight TB. The UKs Department of
Health has given TB Alert a lead role in raising awareness of TB among primary care
health professionals and at-risk communities.
TB Alert works in the UK by enabling local partnerships between public and third
sector organisations to deliver awareness-raising work. In addition TB Alert has
created 'The Truth About TB' programme (www.thetruthabouttb.org).
TB Alert recognised the need for a structured approach to monitoring and evaluation
in order to provide evidence of outcomes and to demonstrate learning. Programme
staff worked with Charities Evaluation Services (CES) to develop a theory of change
as part of their planning processes and to build self-evaluation capacity. First, a
stakeholder workshop gathered views on programme priorities, outcomes, outputs,
indicators and tools. During a second workshop, a full story map was developed,
which CES was able to use as the basis for developing a full monitoring framework
and information collection tools.
As TB Alert's programme moves out of the pilot phase, the theory of change will be
reviewed and updated. It is a living document that shows both the intended outcomes
and the programme's developing practice.
Using the map
This map shows the story of the programme in its four phases of promotion, induction
and training, local delivery and strategic engagement. Each phase has its own
outputs and outcomes, which can be tested through monitoring and evaluation, so
that the programme's effectiveness can be assessed and demonstrated.
The story is rooted in the needs originally identified by TB Alert, including addressing
low levels of awareness and barriers to accessing health services. TB Alert aspires
for the programme to have long-term positive impact, including reduced TB incidence
and reduced cost to the NHS.
For more information on developing a Theory of Change map, see
www.ces-vol.org.uk/TOC.

Charities Evaluation Services

32

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Visual map

Charities Evaluation Services

33

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Charities Evaluation Services

34

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Story map summary

Target group A: Individuals at-risk of/with TB

Need

Pre-programme

People present late for TB diagnosis (or do not present

for BCG/screening) due to low levels of awareness.
High stigma.
Barriers to accessing health services.

Promotion
Induction and training
phase
Programme
TSO capacity building
phase
and delivery phase
outcomes
Strategic
engagement between
third and public
sectors

Impact

Post-programme

Awareness starts to increase.

Awareness increases more, leading to early
presentation and reduced stigma.
Awareness, early presentation and reduced stigma
stabilise or improve. Individuals have better access to
better services.
People present earlier for diagnosis (or present for
BCG/screening), TB management improves; incidence
and stigma reduce in the local area, barriers to health
services are lowered.

Target group B: Public sector organisations

Need

Pre-programme

A lack of locally owned public/third sector partnerships

working on TB issues.
Barriers to accessing health services for at-risk
communities.
TB incidence increasing.

Promotion

Public sector organisations are aware of the programme

and take an interest.

Induction and training

phase

Awareness and interest increases, along with some

partnership working and network activity with the
voluntary sector. Public sector partners begin to engage
in project delivery, eg, through seminars.

Programme
phase
TSO capacity building
outcomes
and delivery phase

Impact

Awareness, interest and partnership working continue to

build. Local networks of public sector and TSOs are
strengthened.

Strategic
engagement between
third and public
sectors

Networks are established and some partnerships

become formal. Public sector partners can see that the
work is of high-quality and effective; this leads to their
commissioning TB work on a similar model.

Post-programme

Locally owned public/third sector partnerships working

on TB issues, funding provided, reduced barriers to
access, incidence reduces and management improves,
cost of TB reduces.

Charities Evaluation Services

35

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Target group C: Third sector organisations

Need

Pre-programme

Low awareness of TB symptoms, diagnosis, services and

support.
High stigma.
A lack of locally-owned public/third sector partnerships
working on TB issues.

Promotion

Voluntary organisations start to hear about TB Alert through

media, direct contact and word-of-mouth, as a result their
understanding of TB Alert's work increases, and this results in
further demand for training and other materials.

Induction and
training phase

TB Alert delivers awareness raising seminars and training for

trainers, which increases TB awareness and understanding
amongst staff. Their understanding of TB Alert's work
increases.
Some organisations start to carry out awareness-raising work
using resource packs.
Some voluntary sector organisations seek further
involvement, by applying for funds from and developing
projects with TB Alert.
These projects receive a range of support from TB Alert,
which results in well-run projects focused on TB, which
achieve their outputs and outcomes.
Support from TB Alert also builds the reputation of these
projects with TB stakeholders, leading to more development
and partnerships.

TSO capacity
building and
delivery phase

Local awareness projects deliver TB-focused interventions,

with ongoing support from TB Alert, resulting in improved
awareness and reduced stigma around diagnosis and
treatment.
Other TSOs continue with basic awareness-raising activities,
supported by TB Alert. These activities result in improved
awareness and reduced stigma, resulting in increased early
presentation of TB cases in the intervention areas.
The high quality project and monitoring process design results
in a decent evidence base regarding the projects' changes
stories and outcomes.

Strategic
engagement
between third
and public
sectors

Projects, with TB Alert's support, build their credibility and

relationships and start to form networks with the local public
sector, getting involved in both policy and delivery.
Their influence increases the voice of TB patients in public
sector decision-making, improving policy at the local and
national level.

Postprogramme

Better TB management and lower stigma, locally owned

public/third sector partnerships working on TB issues, funding
for TB work

Programme
phase
outcomes

Impact

Charities Evaluation Services

36

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Appendix 2: A theory of change for mental health advocacy: a case

study
Partners for Health in London, a Kings Fund one-year grants programme running in
2005-2010, was designed to produce good quality evidence from community
interventions. There were three themes to the programme: mental health advocacy,
sexual health and end-of-life care.
Service teams were introduced to a realist evaluation framework 7 and supported to
collect and analyse data to answer specified research questions. Realistic evaluation
includes the identification of a theory of change at the outset, and the evaluation then
tests out whether there is any evidence to suggest that the theory is supported.
The Chinese National Healthy Living Centre (CNHLC) was funded for three years in
June 2006, to provide mental health advocacy to Chinese users. There is evidence
for under-representation of Chinese people across the NHS, including in mental
health services. This could suggest that Chinese people experience better mental
health than the general population or that specific barriers to services exist. This
project was designed to improve access to services for Chinese people with mental
health problems. The evaluation officer, Lucy Tran, segmented the population into
three groups: new arrivals, first generation settlers and students. It was thought
these groups would respond to the intervention differently, and it would be useful to
find out about the differing needs and contexts.
She developed the theory using three categories:
Contexts
Mechanism
Outcomes
Taken together they become CMO configurations and are used to explain why the
service provider thinks the project will work.
CMO configurations

Contexts this includes factors describing the social circumstances of the

individual and their situation that may affect the project and the ability of the user
to access and use the service
Mechanism this encompasses the resources that the organisation brings to the
problem identified and the reasons why the user would use the service
Outcomes this indicates the changes that the organisation would expect to see
as a result of their work.

Described in Pawson and Tilley (1997) Realistic Evaluation, Sage, London.

Charities Evaluation Services

37

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

The evaluator set out her first change theory in the table below:
Contexts
First
generation
settlers

New arrivals

Mechanisms

Need for support

identified by family
or the community
and met by
Cultural conflict
provision of an
between generations
advocate with
Isolation among the
Chinese cultural
elderly
background and
language skills
Lack of English

Mainly Cantonesespeakers from Hong

Kong

Mainly Mandarinspeaking from China

Isolation from family
(in homeland) and
community
Culture shock
Welfare issues
Lack of knowledge
of health and social
care system

Need for support

identified by
service providers
and met by
provision of an
advocate with
Chinese cultural
background and
language skills,
and provision of
practical support

Lack of English
Overseas
students

Charities Evaluation Services

Outcomes

From China, Hong

Kong, Malaysia,
Singapore

Need for support

identified by
student support
services and met
Initial isolation
by provision of an
Culture shock
advocate with
Chinese cultural
Pressures to achieve
background and
(cultural and
language skills
financial)

38

Improved access
to services

Improved
communication
and
understanding
between clients
and health
professionals

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

This initial theory of change at the start of the project, showed in the table above, was
expressed in the following:
Mental health advocacy, delivered by a Chinese advocate with Mandarin
and Cantonese language skills, to first generation settlers, new arrivals
and students, will improve access to services, and communication and
understanding between clients and health professionals.
She decided to ask these three research questions to understand the project better:
1. What issue does the client want to tackle?
Lucys early assumptions were that clients may not want to admit to having
mental health problems and may concentrate on physical symptoms. She wanted
to track what they wanted to discuss to see if this happened. She also thought
that each group would have slightly different concerns.
2. Does the advocate support the client in discussing illness with the family or
community?
Family and community are considered very important to this group and there is a
strong notion of face, or what is socially acceptable to the group. Mental illness
carries a great deal of stigma and Lucy thought that families may present a barrier
to clients seeking help. She wanted to find out if the advocate had an educational
role within the wider community and with families to enable clients to gain greater
access to services.
3. What is the kind of cultural explanation done by the advocate in meetings
with the health professional?
Lucy thought that cultural explanation, more than the language skills of the
advocate, may play an important part in the advocates role.
Evaluation methods
There were two components to the evaluation:
1. A quantitative component derived from the case notes kept by the advocate:
number of clients; client demographics; referral routes.
2. A qualitative case-study component consisting of interviews with clients, clients
relatives and care providers to address the research questions posed by the
process evaluation.
Between the start of the project in December 2006 and July 2009 when data was
collated, a total of 40 clients accessed the service. For the qualitative component,
eight client cases were selected and a total of 14 transcripts were analysed, with the
data set consisting of a mixture of interviews with clients, their family members, care
providers and an advocate account.

Charities Evaluation Services

39

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Evaluation findings
Did the project meet its intended outcomes?
Overall, the project was highly successful in providing effective, culturally-sensitive
support to its users. 8 The intended outcomes were:

Improved communication and understanding between clients and health

professionals.
Improved access to services for clients.

The findings demonstrated a clear role for the Chinese mental health advocate in
improving communication and understanding between health professionals and
patients not only in providing linguistic and cultural support during meetings but
also in providing an independent, external link through which information could be
passed in both directions. Contrary to the conventional role of an advocate in working
on behalf of a single client, the advocate on many occasions was also advocating for
service providers, for example, in reinforcing their messages.
The project also achieved the second outcome of improving access to services.
Through the provision of an advocate of the same cultural and linguistic background,
the project hoped to break down any barriers to mainstream services that might be
perceived by the Chinese community. Indicators for this might be:

an increase in self-referral rates and referrals by family members as a measure of

change in the perceptions of Chinese community members
number of referrals made to other services by the advocate.

In the first eight months, only 9 per cent of referrals were made by family or clients
themselves. This increased to 60 per cent and 69 per cent in the second and third
quarters of the project, perhaps suggesting that members of the community were
more willing to seek help, although the increase might also be attributed to increased
awareness of the service.
Family or self-referrals were mainly associated with less severe illness and emotional
problems. In these cases, the advocate accompanied clients to GPs and made
referrals to mental health services and the CNHLCs Chinese-speaking counselling
service.
All referrals made to the project by statutory services were for clients with severe
diagnoses. In these cases, the advocate improved patient compliance and
engagement with mental health services.
For service providers, language support was also highly valued but the advocate
added an additional dimension to the role of the interpreter, providing an independent

The term user here refers to both the statutory service providers and clients who accessed the
project.

Charities Evaluation Services

40

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

voice. Service providers often found it difficult to engage with Chinese patients and in
these circumstances, the advocate was often working to support the provider. The
advocate also acted as a link person, relaying client past history and cultural contexts
to the provider and relevant information to the client.
Research question 1: What issues did clients want to tackle?
The evaluation revealed differences between the issues that clients in the first
generation and new migrant groups faced. For new migrants, practical issues such
as immigration and welfare and language issues were the primary concerns. For first
generation settlers, these were also issues but the range of issues was much
broader. The differences were not clear cut, but were likely to be associated with
length of residence in the UK, circumstances of the migration (with new migrants
tending to have migrated under irregular circumstances), immigration status and
diagnosis.
For some clients, the benefits of having an advocate were not initially recognised,
and they appreciated only the language support provided. The continuity provided by
the advocate became valued as the advocate-client relationship progressed and in
some cases clients also developed emotional dependence on the advocate.
In the original CMO configurations, isolation from family was a characteristic of the
new migrant group and this was borne out in the project findings. Social isolation and
lack of support from friends and family was a significant issue among new migrants.
The advocates input was crucial in enabling engagement with mental health services
and supporting the basic health and welfare needs of this group. However, the
broader support needs of this group were not met.
The evaluation also found that clients who self-referred or were referred by family
expressed their difficulties in psychological or emotional terms (rather than focusing
purely on physical symptoms). The majority of clients who had clinical diagnoses
accepted their diagnosis.
Research question 2: Did the advocate support the client in discussing illness with
the family or community?
In posing this question, assumptions were made about the stigma of mental illness
and its impact on the clients relationship with family members and friends within the
context of strong family identity and values.
It has been suggested that the close-knit family structure of the Chinese community
offers protection against mental ill health. While the findings of this project neither
support nor refute this hypothesis, they show that the family situation and cultural
influences sometimes contribute to mental illness. In these cases, the cultural identity
of the advocate was an important asset in the mediatory role of the advocate, both
between client and family, and between health professionals and family.
Stigma associated with mental illness was not captured in the evaluation. Clients who
approached the project themselves or who were referred by family were more likely

Charities Evaluation Services

41

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

to be suffering from common mental disorders or psychological distress which carry

less stigma than severe mental disorders.
Research question 3: What is the kind of cultural explanation done by the advocate
in meetings with health professionals?
Cultural explanations for client behaviour or reasoning were generally provided to
health professionals outside of meetings with clients, with clients consent. The type
of culture-related information that the advocate provided included:

client history and its cultural context

client beliefs around medication and the use of Chinese medicine
Chinese family structures and values.

However, the cultural input of the advocate during meetings was less obvious. The
advocate felt that the process of interpretation itself often went beyond the linguistic
and carried cultural nuances that were difficult to capture.
For clients, there was not an explicit need for cultural explanation but the cultural
identity of the advocate was a crucial factor in enabling them to access the service.
Based on the findings, the conjectured CMO configurations proposed at the start of
the project were revised and show the mechanisms and outcomes of the advocacy
process for each client group. CMO configurations for health and social care
professionals were also created based on the findings.

Charities Evaluation Services

42

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Contexts

Mechanisms

Outcomes

Mainly Cantonesespeakers from Hong

Kong
Cultural conflict
between generations
Isolation among the
elderly
Lack of English
Family issues

Self- and family

referral

Improved access
to services
Improved
communication
and
understanding
between clients
and health
professionals

New
arrivals

Mainly Mandarinspeaking from China

Isolation from family (in
homeland) and
community
Culture shock
Welfare issues
Lack of knowledge of
health and social care
system
Lack of English

Service provider
referral
Language skills
and practical
support for the
client
Language and
advocacy skills to
engage clients

Improved
compliance and
better
engagement with
services
Improved
communication
and
understanding
between clients
and health
professionals

Overseas
students

From China, Hong

Kong, Malaysia,
Singapore
Initial isolation
Culture shock
Pressures to achieve
(cultural and financial)

Self, family and

service provider
referral
Cultural identity
and emotional
support
Cultural
explanation for
professionals and
language support

Improved
communication
and
understanding
between clients
and health
professionals

First
generation
settlers

Charities Evaluation Services

Cultural identity
and language skills
of advocate
Cultural
explanation for
professionals and
language support

43

Making Connections: Using a Theory of Change to Develop Planning and Evaluation

Contexts

Mechanisms

Outcomes

Mental and physical health

professionals, eg,
psychiatrists, GPs,
community psychiatric
nurses, social workers

Cultural explanation
Improved quality of
interpretation
Continuity
Befriending

Greater understanding
of user
Greater confidence in
their impact on user

Social care services, eg,

social workers, housing
officers
Other agencies, eg,
benefits agencies, legal
services

Interpretation
Continuity
Befriending

Greater understanding
of user
Greater confidence in
ability to improve the
users situation

Revised theory of change

As a result of the evaluation, the theory of change can be refined and re-written:
Mental health advocacy, delivered by a Chinese advocate with Mandarin
and Cantonese language skills, to first generation settlers, new arrivals
and students will improve access to health, social welfare and
community services, and improve compliance to medication and
understanding of the treatments for service users.
Mental health advocacy, delivered by a Chinese advocate with Mandarin
and Cantonese language skills to first generation settlers, new arrivals
and students will improve the cultural understanding of this community,
strengthen continuity of provision, and increase confidence in the ability
to improve the users situation for service providers.

____________________________________
Extracted with permission from the final report to the Kings Fund: Evaluation of a Chinese
Mental Health Advocacy and Support Project, November 2009, by Lucy Tran.
http://www.kingsfund.org.uk/current_projects/partners_for_health_in_london/case_studies/ch
inese_national.html

Charities Evaluation Services

44

Charities Evaluation Services

4 Coldbath Square
London EC1R 5HL
020 7713 5722
[email protected]
www.ces-vol.org.uk