DISCUSSION  PAPER  

  A Concept Note on Child

Protection Systems Monitoring

and Evaluation

John D. Fluke
Child Protection Research Center, American Humane Association

Fred Wulczyn
Chapin Hall Center for Children at the University of Chicago  

July  16,  2010  

This is a working document that was commissioned by UNICEF. It has been prepared to facilitate the
exchange of knowledge and to stimulate discussion. The findings, interpretations and conclusions
expressed in this paper are those of the authors and do not necessarily reflect the policies or views of
UNICEF.
 1  
 
  

Introduction    
 
The  overall  goal  of  this  concept  note  (note)  is  to  provide  support  for  planning  discussions  for  the  Child  Protection  
Monitoring  and  Evaluation  Reference  Group  (CPMERG)  by  providing  a  brief  explication  of  the  prospects  for  monitoring  
and  evaluation  (M&E)  efforts  in  the  area  of  child  protection  systems.  The  objectives  outlined  for  this  in  the  Terms  of  
Reference  (ToR)  for  the  note  were  as  follows:  
 
x To  identify  major  issues  and  challenges  related  to  monitoring  and  evaluation  of  child  protection  systems  
 
x To  propose  initiatives  that  the  CPMERG  could  undertake  to  address  these  issues  and  challenges,  including  in  the  
short  term  (over  the  next  two  years)  and  in  the  longer  term  
 
In  taking  on  this  task  the  authors  feel  it  is  important  to  be  clear  that  in  the  preparation  time  available,  the  obvious  
complexities  of  the  topic,  and  the  brevity  of  this  note  that  it  is  only  possible  to  scratch  the  surface  of  existing  and  
potential  M&E  efforts,  as  well  as,  the  relevant  documents.    It  is  also  important  to  recognize  that  the  scope  is  limited  to  a  
discussion  of  M&E  in  relation  to  child  protection  systems,  and  that  even  this  limitation  defines  a  broad  scope  that  we  
can  only  begin  to  summarize.  That  said,  our  hope  is  that  this  document  does  stimulate  discussion,  and  will  help  to  
provide  some  focus  on  topics  that  require  much  more  in  depth  consideration.  
 
Background  
 
The  systems  approach  for  child  protection  efforts  recently  emerged  through  discussions  amongst  a  range  of  global  child  
protection  advocates  based  on  concerns  that  child  protection  efforts  had  primarily  been  addressing  single  issues  such  as  
manmade  and  natural  disasters,  trafficking,  street  children,  and  so  forth.1  This  concern  is  summarized  below:  
 
Since  the  entry  into  force  of  the  Convention  on  the  Rights  of  the  Child,  legal  reforms  for  the  most  part  have  
addressed  child  protection  concerns  on  an  issue-­‐by-­‐issue  basis,  rather  than  adopting  a  comprehensive  approach.  
In  addition  to  thoroughly  reviewing  legislative  frameworks  ʹ  and  reforming  and  amending  them  as  necessary  ʹ  
governments  need  to  adopt  national  budgets,  policies,  practices  and  monitoring  mechanisms  through  a  human  

                                                                                                                     
1
 Save  the  Children  (2006).  Why  effective  national  child  protection  systems  are  needed:    Save  the  Children's  key  recommendations  in  response  to  
the  UN  Secretary  General's  Study  on  Violence  Against  Children.    United  Kingdom:    Save  the  Children.    

UNICEF,  (2008).  UNICEF  Child  Protection  Strategy.

2  
 
 rights-­‐based  approach.  Governments  also  need  to  encourage  public  discussion  of  child  protection  issues,  
because  legislation  alone  will  not  have  sufficient  impact  unless  awareness  is  raised  and  attitudes  are  changed.  
(UNICEF,  2009,  p  2).2  
 
Historically,  analysis  and  programming  in  child  protection  have  focused  on  particular  issues  or  specific  groups  of  
vulnerable  children,  sometimes  referred  to  as  a  vertical  approach.  Issues  receiving  attention  in  recent  years  include  
violence  against  children,  alternative  care,  justice  for  children,  children  affected  by  armed  forces  and  groups,  trafficking,  
sexual  exploitation,  child  labor  and  child  separation.  However,  many  children  are  vulnerable  to  multiple  child  protection  
violations.  While  vertical,  issue-­‐focused  programming  can  be  very  effective  in  serving  the  specific  cohort  of  children  
targeted,  it  can  result  in  protection  gaps,  lack  of  coherent  referral  systems  and  insufficient  attention  to  early  
intervention,  family  support  mechanisms  and  prevention  efforts.    
 
In  contrast,  child  protection  systems  are  seen  as  a  comprehensive  and  sustainable  approach  to  preventing  and  
responding  to  child  protection  issues.  They  comprise  the  set  of  laws,  policies,  regulations  and  services  required  across  all  
social  sectors  ʹ  especially  social  welfare,  education,  health,  security  and  justice  ʹ  to  respond  to  and  prevent  protection-­‐
related  risks.  This  approach  does  not  negate  the  importance  of  addressing  child  protection  issues,  but  rather  ensures  
that  these  issues  are  placed  within  a  holistic  service  structure.  Vertical  concerns  in  this  view  can  become  an  entry  point  
to  strengthen  child  protection  systems  more  broadly.  Such  systems  seek  to  protect  all  children  and  to  unite  all  
stakeholders  behind  a  common  set  of  goals,  creating  a  long  term  response  that  is  robust,  properly  coordinated  and  
adapted  to  evolving  problems.  
 
Since  the  notion  of  a  child  protection  systems  approach  began  to  gain  traction  a  number  of  efforts  have  taken  place  on  
the  part  of  UNICEF  and  other  related  internationally  based  Non-­‐governmental  Organizations  (NGOs)  in  order  to  help  
clarify  what  is  meant  by  a  systems  approach  and  how  such  an  approach  can  be  infused  into  global,  national,  and  local  
ĞĨĨŽƌƚƐĂŝŵĞĚĂƚĂĚĚƌĞƐƐŝŶŐĐŚŝůĚƌĞŶ͛ƐƌŝŐŚƚƐĂŶĚŝŵƉƌŽǀŝŶŐƚŚĞŝƌǁĞůů-­‐being.  The  need  for  an  M&E  framework  focused  on  
the  systems  approach  is  a  key  part  of  the  process  needed  to  understand  and  improve  such  systems.  
 
Child  Protection  Systems:    An  Overview  
 
All  systems  including  child  protection  systems  consist  of  interactive  parts  that  operate  together  within  some  discernable  
boundary  or  scope.  It  is  important  to  be  clear  that  the  idea  of  a  systems  approach  does  not  prescribe  the  scope  or  the  
components  of  a  child  protection  system  in  the  context  of  a  given  setting.  Another  assumption  is  that  regardless  of  the  

                                                                                                                     
2
UNICEF  (2009).  Progress  for  Children;  a  Report  Card.  New  York:  UNICEF.

3  
 
state  of  the  system,  a  child  protection  system  exists  wherever  children  are  being  raised.3  Through  the  UN  consensus  
represented  by  the  Convention  on  the  Rights  of  the  Child  (CRC),  the  broad  goals  of  a  systems  approach  in  child  
protection  must  address  whether  children  are  effectively  protected  from  violence,  abuse,  exploitation,  and  other  forms  
of  maltreatment.    
 
Given  this  high  level  view  of  child  protection  systems  there  are  also  certain  dynamics  of  these  systems  that  may  help  to  
guide  discussion  from  an  M&E  perspective.    First,  dynamics  exist  between  the  status  of  children  (measured  as  
outcomes),  child  protection  goals,  and  the  child  protection  system  in  relation  to  change,  including  social  change.    
Further,  there  are  boundaries  or  embeddedness  between  or  among  the  components  of  the  child  protection  system  and  
other  systems  such  as  health  and  education,  as  well  as,  externalities  and  emergencies  that  influence  the  child  protection  
systems.  This  means  that,  observing,  understanding,  and  determining  the  impact  of  these  events  and  structures  on  the  
status  and  well-­‐being  of  children  is  undoubtedly  one  of  the  more  important  functions  of  an  M&E  effort.  Finally,  over  
time  all  systems  respond  to  the  gap  between  their  goals  and  their  attainment.  One  complexity  of  child  protection  
systems  is  that  the  goals  may  be  difficult  to  identify  from  the  M&E  perspective  as  they  may  not  only  be  responding  to  
both  explicitly  stated  policy  goals,  but  also  to  the  cultural  context  and  interpretation  of  those  goals  which  may  be  
implicit.4  
 
Defining,  Mapping  and  Assessing  Child  Protection  Systems    
 
Figure  1  provides  a  schematic  of  a  child  protection  system  in  relation  to  levels  and  components  that  are  consistently  
identified  as  part  of  such  a  system  in  the  literature.    The  components  of  the  system  are  tied  to  a  continuum  of  levels  that  
consist  of  actors  and  contexts  along  one  dimension  and  more  formal  to  informal  manifestations  of  the  components  on  a  
second  dimension.    While  more  formal  systems  run  the  gamut  from  national  agencies  to  systems  established  locally  and  
operated  professionally  by  NGOs,  more  informal  child  protection  systems  are  most  often  those  that  operate  between  
individuals  and  family  members,  but  may  also  operate  more  broadly  at  a  community  level.  Both  informal  and  more  
formal  systems  can  emerge  from  and  are  impacted  by  social  or  cultural  norms  related  to  the  care  and  protection  of  
children.    
 

                                                                                                                     
3
 Wulczyn,  F.,  Daro,  D.,  Fluke,  J.,  Feldman,  S.,  Glodek,  C.,  &  Lifanda,  K.  (2009).  Adapting  a  Systems  Approach  to  Child  Protection:  Key  Concepts  and  
Considerations.  Chicago:  Chapin  Hall.  
4
 Ibid.

4  
 
F igure 1. C hild Protection Systems: Actors, Context, and Components5

Aside  from  helping  to  explain  what  a  child  protection  system  is,  another  objective  of  such  schematics  is  to  provide  a  
blueprint  for  mapping  exercises  within  country  specific  contexts.  The  expectation  is  that  the  mapping  exercise  can  
provide  a  structure  on  which  to  hang  the  detail  of  the  actual  system  of  child  protection  in  a  given  setting,  to  help  assess  
the  status  of  a  system,  and  to  prioritize  ongoing  efforts  to  develop  the  system  further  in  support  of  its  goals,  including  
refining  the  goals  themselves.      
 
One  such  mapping  process  was  recently  developed  by  UNICEF  and  their  partners  Maestral  International6  that  
incorporates  seven  key  elements  thought  to  contribute  to  describe  the  basic  range  of  child  protection  systems  
components  including:  1)  Laws,  Policies,  Standards  and  Regulations;  2).  Cooperation,  Coordination  and  Collaboration;  3).  
Capacity  Building;  4).  Service  and  Service  Delivery  Mechanisms;  5).  Communication,  Education  and  Mobilization  for  
Change;  6).  Financial  Resources;  and  7).  Accountability  Mechanisms.  This  toolkit  has  now  been  piloted  in  several  
countries,  and  other  efforts  of  a  similar  nature  are  underway,  however,  the  value  and  utility  of  the  mapping  and  
assessment  have  yet  to  be  determined  (see  looking  ahead  section  below).  We  now  turn  to  the  complexities  of  the  M&E  
enterprise  itself  and  the  issues  and  challenges  associated  with  how  it  may  contribute  the  systems  approach.  
 

                                                                                                                     
5
Ibid.
6
 hE/&͘;ϮϬϬϵͿ͘ŚŝůĚWƌŽƚĞĐƚŝŽŶ^LJƐƚĞŵƐDĂƉƉŝŶŐĂŶĚƐƐĞƐƐŵĞŶƚ͗hƐĞƌ͛Ɛ'ƵŝĚĞ͘EĞǁzŽƌŬ͗hE/&͘  

5  
 
Monitoring  and  Evaluation:    Main  Issues  and  Challenges    
 
M&E  efforts  can  be  viewed  as  a  component  part  of  what  supports  and  drives  a  child  protection  system  toward  goal  
attainment.  A  child  protection  system  impacts  multiple  outcomes.  If  a  given  system  is  successful,  the  attainment  of  
outcomes  can  be  viewed  collecƚŝǀĞůLJĂƐŵĞĞƚŝŶŐƚŚĞŐŽĂůƐŽĨĂĚĚƌĞƐƐŝŶŐĐŚŝůĚƌĞŶ͛ƐƌŝŐŚƚƐĂŶĚŝŵƉƌŽǀŝŶŐƚŚĞŝƌǁĞůůďĞŝŶŐ͘
On  the  other  hand,  the  complexity  of  child  protection  systems  means  that  M&E  efforts  are  not  simple  and  that  they  
must  be  multi-­‐faceted.  
 
M&E  is  crucial  for  examining  systems  for  multiple  reasons.  The  obvious  first  priority  is  to  address  accountability  and  help  
determine  the  status  of  children  and  whether  outcomes  for  children  are  changing  (i.e.,  improving  or  not).  The  second  is  
to  help  identify  what  elements  and  dynamics  of  a  system  provide  the  most  leverage  for  goal  directed  change.  Less  
obviously,  and  as  a  minimally  organized  process,  M&E  can  be  viewed  abstractly  as  one  of  the  key  ingredients  of  the  
system  since  as  a  source  of  information  it  is  also  used  by  the  system  to  modify  its  behavior  in  the  form  of  feedback.  
Thus,  like  the  assumption  regarding  the  implicit  existence  of  child  protection  systems  themselves,  the  actors  in  the  
system  interpret  M&E  information  for  feedback  purposes.  As  the  systems  become  more  complex  and  formal  the  issues  
of  data  quality,  validity,  and  interpretation  become  increasingly  complex  as  well.  Because  of  this  role  in  the  feedback  
dynamics  of  the  system,  attention  to  the  relative  accuracy,  validity  and  interpretation  of  the  M&E  information  and  
knowledge  is  a  critical  aspect  of  the  systems  approach.    
 
While  the  section  above  provides  some  context  for  the  systems  approach  to  child  protection,  the  role  of  a  MERG  in  this  
area  is  much  broader.  Given  the  recent  and  burgeoning  status  of  the  systems  approach,  the  M&E  concerns  that  bear  
specifically  on  the  approach  are  likely  to  be  emergent.  With  this  in  mind  the  challenges  in  defining  a  productive  role  for  a  
CPMERG  with  respect  to  child  protection  systems  are  also  addressed  somewhat  broadly  in  this  section,  at  least  within  
the  limitations  of  the  note.  
 
Definitional  Challenges  
 
Definitions  are  a  typical  starting  point  for  M&E  planning  and  generally  a  challenging  one,  and  unfortunately  space  in  this  
note  limits  the  specifics  that  can  be  addressed  here.  One  potentially  useful  observation  given  much  of  the  material  
reviewed  for  this  note  is  that  there  are  many  terms  for  which  good  operational  definitions  are  needed  in  order  to  
facilitate  communication  within  the  CPMERG  itself  in  order  to  minimize  miscommunication.  That  said,  too  much  focus  
on  definitions,  especially  where  consensus  is  not  possible  may  hinder  progress.  The  terminology  that  might  be  reviewed  
includes  the  following  categories:  
 

6  
 
 Surveillance  related  terms:  These  not  only  include  relatively  standard  terms  from  epidemiology  such  as  
prevalence  and  incidence,  but  require  some  customization  specific  to  various  child  protection  phenomenon.  For  
example,  incidence  of  child  maltreatment  may  involve  measures  of  self  reporting  or  be  based  on  official  sources  
of  children  coming  to  the  attention  of  the  appropriate  authorities.  Both  may  meet  definitional  requirements  for  
maltreatment  incidence,  but  are  distinct  measures  with  very  different  interpretations  and  uses.    
 
Child  protection  service  continuum  terms:  It  is  not  clear  that  a  shared  understanding  of  key  terms  like  
promotion,  prevention,  response,  and  intervention  exists.  
 
Evaluation  related  terms:  These  include  outcomes,  inputs,  indicators,  quality  indicators,  baselines  and  so  forth.  
All  these  need  clarification,  sub-­‐classification  (e.g.,  short  term  vs.  long  term  outcomes),  and  child  protection  
contextualization.  
Unit  of  analysis  related  terms:  These  include  typical  subjects  of  child  protection  M&E  efforts  such  as  children,  
caregivers,  families,  communities,  local  authorities  and  so  forth.    
 
Systems  related  terms:  The  terminology  here  includes  feedback,  dynamics,  structures,  components,  and  the  like.  
 
One  or  more  frameworks:    In  this  context,  a  framework  would  describe  how  actions  within  the  system  are  
connected  to  outcomes  so  that  corrective  actions  (pathways)  can  be  tested.  
 
The  utility  of  appropriate  definitions  is  apparent;  considering  how  much  attention  is  needed  with  respect  to  definitions  
may  also  be  an  appropriate  exercise  in  as  much  as  general  resolution  is  rarely  precise.  
 
Goals,  standards,  and  baselines    
 
Monitoring  can  be  thought  of  as  an  exercise  in  gap  analysis  between  the  current  set  of  conditions  and  the  desired  goals.  
Gap  analysis  implies  having  a  sense  for  what  would  be  trueͶa  standardͶif  the  effort  met  all  goals.  According  to  the  
National  Institutes  of  Health,  a  baseline  is  the  time  point  just  before  an  intervention  when  starting  measurements  are  
taken.7    Used  in  conjunction  with  standards,  a  baseline  may  also  point  to  process  and  quality  improvements  that  are  
needed  in  the  first  place.8  
 

                                                                                                                     
7
 See  http://www.niaid.nih.gov/factsheets/Glossary.htm  
8
 Wulczyn,  F.,  Orlebeke,  B.,  and  Haight,  J.  (2009).  Finding  the  Return  on  Investment:  A  Framework  for  Monitoring  Local  Child  Welfare  Agencies.  
Chicago:  Chapin  Hall.  
 
7  
 
An  obvious  starting  point  for  setting  goals  and  standards  is  Article  19  of  the  Convention  on  the  Rights  of  Children.  Setting  
goals  and  standards  in  the  area  of  child  protection  systems  is  complicated  for  several  reasons.  Among  these  is  the  need  
to  formulate  meaningful  standards  at  various  levels  including  local  or  community  levels,  cultural  levels,  the  country  level,  
and  at  the  multi-­‐national  level.  Ideally,  standards  would  have  some  coherence  across  levels.    
 
Another  complication  is  the  technical  process  through  which  goals,  standards,  and  baselines  are  likely  to  be  set.  For  
example,  in  part  owing  to  the  issue  oriented  history  of  child  protection  work,  there  may  be  a  tendency  to  focus  on  key  
themes  in  child  protection9  that  do  not  address  the  full  range  of  the  child  protection  system.  In  the  same  vein,  the  
knowledge  and  technology  for  M&E  associated  with  specific  child  protection  themes  may  be  differentially  developed  
leading  to  a  focus  on  themes  as  a  straightforward  way  to  make  progress.  It  may  be  worthwhile  to  try  and  insure  that  
attention  to  the  development  of  M&E  and  attendant  resources  are  not  too  lopsided  in  favor  of  methods  and  approaches  
that  are  better  known  and  understood.  
 
Another  complicating  aspect  of  the  process  for  setting  goals,  standards,  and  baselines  is  the  participation  of  key  
constituents,  and  especially  children.  Further  complicating  this  participation  process  is  the  degree  to  which  participation  
by  some  constituents  is  accepted  by  governments,    local  authorities,  or  community  members  and  the  degree  to  which  
some  stakeholders  are  constrained  from  participating  due  to  concerns  about  their  governments  or  because  they  are  
marginalized  or  socially  excluded.10  
 
Measurement    
 
A  primary  orientation  of  the  paper  Adapting  a  Systems  Approach11  ŝƐŽŶŝŶƐƵƌŝŶŐƚŚĂƚĐŚŝůĚƌĞŶ͛ƐƌŝŐŚƚƐĂƌĞĂĚĚƌĞƐƐĞĚĂŶĚ
that  child  well-­‐being  is  improved  as  part  of  the  goal  setting  framework  of  the  child  protection  systems  approach.  The  
implication  is  that  child  outcomes  and  their  measurement  are  a  fundamental  aspect  of  what  is  needed  from  an  M&E  
perspective.  Obviously  the  formulation  and  testing  of  appropriate  and  valid  measurement  of  well-­‐being  related  
outcomes  is  extremely  challenging.    
 

                                                                                                                     
9
 Examples  of  themes  include  (i)  birth  registration;  (ii)  child  labor;  (iii)  harmful  cultural  practices  (e.g.,  female  genital  mutilation/cutting,  child  
marriage,  discrimination);  (iv)  physically,  sexually,  and  psychologically  abused  and  neglected  children;  (iv)  children  without  adequate  family  care  
or  alternative  care,  (vi)  child  mobility  and  child  trafficking;  (vii)  commercial  sexual  exploitation;  (viii)  children  and  justice;  and  (ix)  child  protection  
in  emergencies/armed  conflict.  
10
 Wessells,  M.  (2009).  What  Are  We  Learning  About  Protecting  Children  in  the  Community?  United  Kingdom:  Save  the  Children.  
 
11
 Wulczyn,  F.,  Daro,  D.,  Fluke,  J.,  Feldman,  S.,  Glodek,  C.,  &  Lifanda,  K.  (2009).  Adapting  a  Systems  Approach  to  Child  Protection:  Key  Concepts  and  
Considerations.  Chicago:  Chapin  Hall.  

8  
 
In  the  international  context  apparently  successful  efforts  have  been  made  to  address  the  measurement  of  child  
outcomes  as  they  pertain  to  specific  related  sectors  (i.e.,  health,  education)  or  issues/themes  (i.e.,  HIV/AIDS,  Children  in  
Care,  etc.).  In  contrast  while  efforts  have  been  made  to  develop  child  outcome  measures  for  child  protection,  these  
appear  to  be  less  developed  at  this  stage,  their  utilization  limited,  and  experience  with  these  measurement  approaches  
is  not  common  at  least  among  low  and  middle  income  countries.12  
 
Further  complicating  the  measurement  of  outcomes  is  the  concern  that  child  well-­‐being  is  culturally  specific.  Does  this  
imply  that  measurement  must  also  be  culturally  specific  rendering  the  possibility  of  comparisons  between  or  even  
within  countries  that  much  more  complicated,  or  are  there  features  of  child  development  or  other  aspects  of  well-­‐being  
that  can  be  considered  more  broadly  in  common?  ůĞĂƌůLJƚŚĞZƉƌŽǀŝĚĞƐƚŚĞďĂƐŝƐŽĨĂƵŶŝǀĞƌƐĂůǀŝĞǁŽĨĐŚŝůĚƌĞŶ͛Ɛ
rights,  but  this  may  not  lead  directly  to  an  understanding  of  child  well-­‐being  in  relation  to  these  rights.  If  there  are  
common  outcome  measures  of  well-­‐being,  does  this  imply  that  homogeneity  of  expression  and  the  degree  of  
homogeneity  is  sufficient  to  support  the  development  of  standards?  
 
 Surveillance  measurement  that  pertains  to  child  protection  appears  more  systematically  developed  and  
methodologically  advanced  than  other  M&E  efforts  for  child  protection.  For  example  the  UNICEF  Multiple  Indicator  
Cluster  Survey  (MICS)13    and  the  Demographic  and  Health  Survey  (DHS)14  both  collect  common  information  at  the  
household  level  regarding  the  status  of  children  specific  to  child  protection  including  child  child  labour,  child  marriage,  
birth  registration,  female  genital  mutiliation/cutting,  domestic  violence,  child  disability  and  child  discipline  across  low  
and  middle  income  countries.    While  these  data  have  important  limitations  such  as  not  covering  the  range  of  child  
protection  themes  and  being  limited  to  children  residing  in  households,  they  are  a  very  valuable  tool  in  the  M&E  
repertoire.  Because  these  are  ongoing  data  collection  programs,  the  potential  exists  to  utilize  these  data  to  help  monitor  
gaps  in  relation  to  baselines  atregional    country,  and  multi-­‐national  levels.    Furthermore,  the  data  from  these  programs  
permit  the  use  of  multi-­‐variate  analyses  that  can  be  exploited  to  identify  key  strategies  and  develop  programs  that  are  
grounded  in  evidence  and  the  size  of  the  impact  of  a  potential  system  of  interventions  can  be  estimated.    In  the  context  
of  child  protection  systems  the  MICS  and  DHS  programs  could  prove  most  useful  and  perhaps  even  serve  as  a  source  of  
outcome  measurement  in  terms  of  monitoring  the  promotion  and  prevention  end  of  child  protection  system  continuum  
and  in  helping  to  identifying  high  impact  prevention  needs.  Data  collection  tools  and  data  collection  programs  with  

                                                                                                                     
12
 See  for  example  the  following  documents:  UNICEF.  (undated).  Draft  ToR  on  the  Child  Protection  MERG;  Wessells,  M.  (2009).  What  Are  We  
Learning  About  Protecting  Children  in  the  Community?  United  Kingdom:  Save  the  Children.  Save  the  Children  (2008).  Menu  of  Outcome  
Indicators.  United  Kingdom:  Save  the  Children.  
13
 http://www.childinfo.org/  
14
 http://www.measuredhs.com/start.cfm  

9  
 
similar  scope  are  needed  to  address  other  M&E  needs  in  child  protection.  Further  attention  also  needs  to  be  given  to  
measurement  of  changes  in  attitudes  and  social  norms  that  underlie  various  forms  of  violence,  exploitation  and  abuse.      
 
Other  country  specific  and  even  regional  efforts  (e.g.,  Balkan  Area  Epidemiologic  Survey  of  Child  Abuse  and  Neglect)  are  
also  underway  to  develop  a  range  of  data  gathering  programs  addressing  both  surveillance  and  system  level  
intervention  processes  in  child  protection  that  could  potentially  be  coordinated  to  a  greater  extent.    One  example  of  
such  coordination  is  the  International  Society  for  the  Prevention  of  Child  Abuse  and  Neglect  Working  Group  on  Child  
Maltreatment  Data  Collection  (ISPCANWG)15  which    is  attempting  to  maintain  current  information  on  national  child  
maltreatment  data  collection  programs.  
 
Measurement  of  inputs  including  financial  inputs,  workforce  development  including  recruitment  and  training,  legal  and  
policy  development,  numbers  and  characteristics  of  service  recipients,  and  so  forth  also  require  attention.  Experience  by  
the  authors  with  multiple  systems  in  high  income  countries  has  shown  that  the  implementation  of  these  measurements  
is  exceedingly  challenging,  and  is  likely  to  be  especially  complex  in  settings  that  are  informal.  Much  of  the  M&E  work  in  
this  area  in  high  income  countries  relies  on  the  development  of  administrative  data  systems  and  consequently  some  
degree  of  ongoing  governmental  support.  In  low  income  countries  some  of  the  same  difficulties  experienced  by  high  
income  countries  can  be  anticipated,  although  perhaps  more  so,  including  difficulties  in  sustaining  long  term  
infrastructure  development,  sufficiency  of  trained  personnel,  local  cooperation  with  regional  or  national  governments,  
and  of  course  adequate  financial  resources.    
 
Another  fundamental  measurement  issue  when  considering  systems  is  the  degree  to  which  the  outcomes  can  
reasonably  be  attributed  to  the  performance  of  the  system.    The  overall  quality  of  the  data  is  a  concern  here  and  apart  
from  not  inconsiderable  issues  concerning  the  reliability  and  validity  of  measurement  the  sustainability  of  the  data  
collection  program  over  time  with  a  high  degree  of  consistency  is  also  a  critical  part  of  the  infrastructure  needed.        
 
Finally,  consideration  of  contexts  in  which  countries  often  find  themselves  may  prevent  or  interrupt  the  M&E  process.  
These  include  countries  addressing  emergencies,  fragile  states  that  are  burdened  by  other  critical  priorities,  and  states  in  
political  or  social  transition.  Although  ideally  the  existence  of  a  child  protection  system  would  help  to  facilitate  effective  
responses  to  these  types  of  events  and  particularly  emergencies  and  would  help  to  prevent  violations  of  rights  or  other  
child  protection  issues  from  arising,  it  is  also  possible  for  weak  systems  that  the  events  may  cause  the  unraveling  of  both  
the  informal  and  more  formal  child  protections  systems  in  a  country  or  region.  
 

                                                                                                                     
15
http://www.ispcan.org/CMData/index.html

10  
 
Links  to  Other  Work  
 
Monitoring  and  evaluation  involve  activities  that  are  universal  ʹ  observations  have  to  be  made,  data  have  to  recorded  
and  analyzed,  and  the  findings  must  be  interpreted.  These  linked  actions  are  fundamental  to  knowledge  development.    
The  purpose  to  which  monitoring  and  evaluation  are  applied  is  more  context  sensitive  and  frame  answers  to  the  how,  
what,  and  why  of  a  particular  monitoring  and  evaluation  project.    For  example,  monitoring  system  performance  requires  
a  different  skill  set  when  compared  with  monitoring  the  health  and  well-­‐being  of  children  within  the  rights  perspective.    
Yet,  observation,  recording,  analysis,  and  interpretation  are  essential  in  both  instances.  
 
The  link  between  monitoring,  evaluation,  and  the  systems  perspective  can  be  traced  back  through  to  the  CRC.    In  one  
sense,  the  CRC  provides  a  scope  of  work  that  answers  a  relatively  simple  question  ʹ  what  is  the  subject  matter  of  
concern?    What  does  one  want  to  monitor  and  evaluate,  and  toward  what  end?    For  example,  Article  19  of  the  CRC  
ĚŝƌĞĐƚƐ͞^ƚĂƚĞƐWĂƌƚŝĞƐ͘͘͘ƚŽƉƌŽƚĞĐƚĐŚŝůĚƌĞŶĨƌŽŵĂůůĨŽƌŵƐŽĨƉŚLJƐŝĐĂůŽƌŵĞŶƚĂůǀŝŽůĞŶĐĞ͕ŝŶũƵƌLJŽƌĂďƵƐĞ͕ŶĞŐůĞĐƚŽƌ
negligent  treatmenƚ͕ŵĂůƚƌĞĂƚŵĞŶƚŽƌĞdžƉůŽŝƚĂƚŝŽŶ͕ŝŶĐůƵĚŝŶŐƐĞdžƵĂůĂďƵƐĞ͘͟dŚĞƐĞĂƌĞĨƵƌƚŚĞƌĚŝƌĞĐƚĞĚƚŽƉƵƌƐƵĞ
legislative,  administrative,  social,  and  educational  measures  deemed  appropriate,  including  the  development  of  social  
programs  to  support  children  and  those  who  care  for  them.    Finally,  Article  19  goes  on  to  call  for  other  forms  of  
ƉƌĞǀĞŶƚŝŽŶĂƐǁĞůůĂƐƉƌŽĐĞĚƵƌĞƐĨŽƌ͞ŝĚĞŶƚŝĨŝĐĂƚŝŽŶ͕ƌĞƉŽƌƚŝŶŐ͕ƌĞĨĞƌƌĂů͕ŝŶǀĞƐƚŝŐĂƚŝŽŶ͕ƚƌĞĂƚŵĞŶƚ͕ĂŶĚĨŽůůŽǁ-­‐up  of  
ŝŶƐƚĂŶĐĞƐĐŚŝůĚƌĞŶŵĂůƚƌĞĂƚŵĞŶƚ͘͟  
 
The  charge,  if  you  will,  is  furtŚĞƌĐůĂƌŝĨŝĞĚŝŶhE/&͛ƐŽǁŶĐŚŝůĚƉƌŽƚĞĐƚŝŽŶƐƚƌĂƚĞŐLJǁŚŝĐŚƐƚĂƚĞƐƚŚĂƚ͞΀W΁ƌĞǀĞŶƚŝŶŐĂŶĚ
ƌĞƐƉŽŶĚŝŶŐƚŽǀŝŽůĞŶĐĞ͕ĞdžƉůŽŝƚĂƚŝŽŶĂŶĚĂďƵƐĞŝƐĞƐƐĞŶƚŝĂůƚŽĞŶƐƵƌŝŶŐĐŚŝůĚƌĞŶ͛ƐƌŝŐŚƚƐƚŽƐƵƌǀŝǀĂů͕ĚĞǀĞůŽƉŵĞŶƚĂŶĚ
well-­‐being.͟  The  vision  and  approach  of  UNICEF  is  to  create  a  protective  environment,  where  girls  and  boys  are  free  
from  violence,  exploitation,  and  unnecessary  separation  from  family;  and  where  laws,  services,  behaviors  and  practices  
ŵŝŶŝŵŝnjĞĐŚŝůĚƌĞŶ͛ƐǀƵůŶĞƌĂďŝůŝƚLJ͕ĂĚĚƌĞƐƐŬŶŽǁŶƌŝƐŬĨĂĐƚŽƌƐ͕ĂŶĚ  ƐƚƌĞŶŐƚŚĞŶĐŚŝůĚƌĞŶ͛ƐŽǁŶƌĞƐŝůŝĞŶĐĞ͘dŚŝƐĂƉƉƌŽĂĐŚŝƐ
human  rights-­‐based,  and  emphasizes  prevention  as  well  as  the  accountability  of  governments.  It  enhances  aid  
effectiveness  by  supporting  sustained  national  capacity  for  child  protection.  Finally,  it  reĨůĞĐƚƐĐŚŝůĚƌĞŶ͛ƐŽǁŶƌŽůĞƐĂŶĚ
resilience  as  agents  of  change  and  actors  in  strengthening  the  protective  environment.  
 
^ƵďƐĞƋƵĞŶƚǁŽƌŬŝŶƚŚĞĂƌĞĂ͕ŝŶĐůƵĚŝŶŐŚĂƉŝŶ,Ăůů͛ƐƉĂƉĞƌƚŝƚůĞĚAdapting  a  Systems  Approach,  the  mapping  and  
assessment  tool  developed  by  UNICEF  and  Maestral  International,  together  with  other  regionally  designed  papers,  tools,  
and  frameworks,  all  play  a  role  in  bringing  needed  clarity  (i.e.,  operational  definition  to  use  a  measurement  term)  to  the  
vision  set  forth  in  the  CRC  and  other  foundational  documents.16    For  example,  the  Toolkit  to  Map  and  Assess  Child  

                                                                                                                     
16
 See,  for  example,  the  following  documents.  Save  the  Children  (2006).    Why  effective  national  child  protection  systems  are  needed:    Save  the  
Children's  key  recommendations  in  response  to  the  UN  Secretary  General's  Study  on  Violence  Against  Children.    United  Kingdom:    Save  the  
11  
 
Protection  Systems  addresses  content  (what  data  to  collect),  process  (how  to  collect  the  data),  record  keeping,  informants  
(who  to  ask),  and  definitions.    These  are  all  steps  in  a  process  intended  to  produce  actionable  knowledge  about  the  well-­‐
being  of  children,  the  extent  to  which  their  rights  are  being  upheld,  and  the  extent  to  which  underlying  systems  are  
performing  their  functions  within  the  context  of  a  given  child  protection  system.  
 
If  there  is  one  element  missing  from  these  documents,  one  would  have  to  say  that  on  balance  the  documents  outline  
what  might  be  called  the  data  collection  phase  of  monitoring  and  evaluation.    Much  less  time  ŝƐƐƉĞŶƚŽŶƚŚĞ͚ŵĞƚŚŽĚƐ͛
by  which  the  data  are  analyzed  (i.e.,  the  ways  in  which  meaning  is  derived  from  the  data  collected).    We  will  consider  
this  set  of  issues  as  well  as  others  in  the  section  on  looking  ahead  that  follows.  
Looking  Ahead    
 
In  this  section  we  identify  a  set  of  basic  ideas  that  the  CPMERG  may  want  to  take  on  within  both  a  short  and  long  term  
timeline.    Our  suggestions  are  guided  by  our  experience  developing  data  systems,  designing  monitoring  systems,  and  
evaluating  programs  in  a  variety  of  circumstances  and  context.    
Here  are  some  ideas  to  start  with.  
 
o Definitions:    as  described  above,  the  CPMERG  may  be  helpful  in  clarifying  the  range  of  terms  related  to  
child  protection  M&E  work.  A  systematic  process  to  review  terms  of  art  and  achieve  some  degree  of  
consensus  would  help  to  unify  the  M&E  efforts;  however,  this  activity  should  be  time  limited  and  it  must  
be  recognized  that  even  basic  agreement  on  all  terminology  will  not  be  possible.    
 
o Case  studies:    Case  studies  of  existing  monitoring  and  evaluation  efforts  would  provide  a  template  of  
ĞdžƉĞƌŝĞŶĐĞƌĞůĂƚŝǀĞƚŽŚŽǁŵŽŶŝƚŽƌŝŶŐĂŶĚĞǀĂůƵĂƚŝŽŶ͚ƉƌŽũĞĐƚƐ͛ĞǀŽůǀĞŽǀĞƌƚŝŵĞ͘&ŽƌĞdžĂŵƉůĞ͕ŵĂŶLJ
countries  now  have  administrative  data  systems  but  many  do  not.    While  administrative  data  systems  do  
not  provide  all  the  data  one  needs  to  monitor  and  evaluate  child  protection  systems,  such  systems  do  
form  a  backbone  of  sorts  without  which  the  structures  needed  to  monitor  and  evaluate  are  incomplete.    
That  said,  the  development,  maintenance,  and  use  of  administrative  data  systems  is  an  area  fraught  

                                                                                                                                                                                                                                                                                                                                                                                                                                               
Children.    Save  the  Children  (2008).    A  'Rough  Guide'  to  Child  Protection  Systems.    London:    Save  the  Children.    Save  the  Children  (2009).    Child  
protection  systems  in  emergencies:    A  review  of  current  thinking  and  experience.    United  Kingdom:    Save  the  Children    UNICEF  (2008).    UNICEF  
Child  Protection  Strategy.    Annual  Session,  2008.    UNICEF  EAPRO  (2009).    East  Asia  and  Pacific  Child  Protection  Programme  Strategy  Toolkit.    East  
Asia  and  Pacific  Regional  Office.    Bangkok,  Thailand:    UNI&͘tŽƌůĚ,ĞĂůƚŚKƌŐĂŶŝnjĂƚŝŽŶ͘;ϮϬϬϳͿ͘ǀĞƌLJďŽĚLJ͛ƐƵƐŝŶĞƐƐ͗^ƚƌĞŶŐƚŚĞŶŝŶŐ,ĞĂůƚŚ
Systems  to  Improve  Health  Outcomes.    A  Framework  for  Action.    World  Health  Organization.    Australian  Research  Alliance  for  Children  and  Youth  
(2008).    Inverting  the  pyramid:    Enhancing  systems  for  protecting  children.    Allen  Consulting  Group:    Melbourne.    Inter-­‐ŵĞƌŝĐĂŶŚŝůĚƌĞŶ͛Ɛ
Institute  (2003).  Basic  Prototype:  National  Child  System.  INN/OAS:  Montevideo,  Uruguay.    United  Nations  High  Commission  on  Refugees  (2009).    
Summary  note:    Inter-­‐agency  expert  consultation  on  child  protection  systems.    19  and  20  January  2009.    Centre  for  Humanitarian  Dialogue:    
Geneva,  Switzerland.    UN  General  Assembly,  Convention  on  the  Rights  of  the  Child,  20  November  1989,  United  Nations,  Treaty  Series,  vol.  1577,  
p.  3,  available  at:  http://www.unhcr.org/refworld/docid/3ae6b38f0.html  [accessed  27  October  2009].          

12  
 
 with  challenges  and  the  potential  for  expensive  mistakes.  
 
ĂƐĞƐƚƵĚŝĞƐŽĨĨĞƌĂǁĂLJƚŽŚŝŐŚůŝŐŚƚƚŚĞ͚ůĞƐƐŽŶƐůĞĂƌŶĞĚ͛ŝŶƚŚŽƐĞĐŽŶƚĞdžƚƐǁŚĞƌĞƚŚĞĚĞǀĞůŽƉŵĞŶƚŽĨ
monitoring  and  evaluation  systems  are  more  advanced.  Also,  possible  are  the  formation  of  comparative  
case  studies  on  child  protection  systems  which  could  highlight  key  features  of  particular  sets  of  
countries,  or  provide  input  for  regional  planning.  
 
o Mapping  and  assessment:  The  process  of  mapping  and  assessment  of  child  protection  systems  is  already  
being  supported,  however,  an  assessment  of  the  utility  of  the  work  has  not  been  undertaken.  Perhaps  
specific  systematic  attention  to  the  evaluation  of  the  mapping  and  assessment  processes  would  help  
inform  the  improvement  of  the  toolkits  and  provide  some  sense  of  how  the  data  gathered  during  these  
processes  could  be  used  on  an  ongoing  basis.  The  key  to  a  successful  process  would  be  the  development  
of  evaluation  dimensions  and  criteria  in  advance  to  insure  the  process  is  rigorous.    
 
o Exploit  existing  data:  Despite  their  limitations,  a  clear  example  here  is  that  data  such  as  MICS  and  DHS  
are  underutilized  in  the  M&E  work  on  child  protection  systems.  Several  questions  could  be  envisioned  
with  respect  to  MICS:  1)  what  extent  do  child  protection  systems  exist  and  what  characterizes  the  
systems  in  the  MICS  countries?  2)  Is  it  possible  to  formulate  a  study  design  in  one  or  two  countries  that  
employs  the  MICS  data  to  set  baselines  and  then  subsequent  iterations  of  MICS  to  assess  prevention  
program  effectiveness  on  multiple  child  protection  surveillance  measures?;  and  3)  What  would  be  the  
benefits,  challenges,  and  limitations  of  such  studies?  
 
Investment  in  the  use  of  existing  data  will  help  to  reinforce  and  improve  the  infrastructure  that  must  be  
developed  to  support  the  complex  M&E  process  needed  for  child  protection  systems.  There  are  other  
sources  of  existing  data  both  quantitative  and  qualitative  that  need  to  be  inventoried,  assessed  for  
utility  and  data  quality,  and  where  appropriate  utilized.  
 
o Ethics  and  the  politics  of  information  ʹ  standards  for  the  release  of  (any)  data  (publication  and  public  
use  access):    One  of  the  main  concerns  once  data  are  collected  has  to  do  with  access  to  the  data.    Access  
to  information  provides  for  broad  participation  in  how  meaning  is  derived  from  a  given  set  of  data.    A  
crucial  step,  one  that  frames  the  enterprise  over  the  long  term,  has  to  do  with  standards  for  public  
access  to  data.    What  set  of  rights  and  obligations  fall  to  those  who  gather  the  data,  those  who  analyze  
the  data  in  the  event  the  two  parties  are  not  one  and  the  same,  and  the  types  of  data  covered  by  the  
policy  (administrative  data,  survey  data,  etc.)?  

13  
 
  
Having  clear  policy  in  this  area  may  have  the  ancillary  benefit  of  increasing  participation  if  informants  
ŬŶŽǁƚŚĂƚĂĐĐĞƐƐƚŽƚŚĞĚĂƚĂŝƐ͚ŽƉĞŶ͛͘KƉĞŶŝƐ͕ŽĨĐŽƵƌƐĞ͕ƐƵďũĞĐƚƚŽĚĞĨŝŶŝƚŝŽŶŝŶƐŽĨĂƌĂƐĐŽŶĨŝĚĞŶƚŝĂů
information  is  often  involved.  
 
o Incentive  based  interventions/transfers:  Transfer  approaches  appear  to  offer  the  possibility  of  concrete  
methods  for  community  interventions  for  specific  interventions  in  child  protection  issues  and  are  being  
considered  for  child  protection  system  formation  and  sustainability.17  Some  review  of  the  literature  and  
development  of  evaluation  design  for  social  transfer  approaches  to  child  protection  could  be  explored  and  
assessed  for  its  utility  in  systems  development  and  the  reinforcement  of  more  informal  systems.  This  could  
be  an  ideal  area  for  M&E  related  efforts  given  the  possibilities  of  integrating  cost  and  outcomes  data.  
 
o Indicator  development:  There  are  ongoing  processes  associated  with  indicator  development  in  
ĐŚŝůĚƌĞŶ͛ƐƐĞƌǀŝĐĞƐ͕ĐŚŝůĚǁĞůů-­‐being,  etc.  The  draft  ToR  for  the  CPMERG  states  indicator  development  as  
one  possible  function  of  the  group  includinŐ͘͘͘͞ƚĞĐŚŶŝĐĂůƌĞǀŝĞǁĂŶĚĂĚǀŝĐĞŽŶDΘŝŶĚŝĐĂƚŽƌƐĨŽƌ
national,  inter-­‐country  and  global  reporting,  and  other  information  gaps  and  outlining  an  agenda  to  
address  them,  which  takes  into  account  field  settings  and  realities.͟18      
 
The  scope  of  indicator  development  is  quite  broad  since  indicators  are  needed  to  understand  system  
functions  and  outcomes.  The  most  important  indicators  are  likely  to  be  outcome  indicators,  both  short  
and  long  term,  for  children.  However,  because  of  the  previously  mentioned  challenges  associated  with  
attributing  outcomes  to  systems,  indicators  for  the  inputs,  processes,  resources,  and  dynamics  of  the  
child  protection  systems  are  also  needed.      
 
o  ͞EĞƵƌŽŶƐƚŽŶĞŝŐŚďŽƵƌŚŽŽĚƐ͟ʹ  child  development:    In  the  literature  covering  child  rights  and  the  
systems  approach  one  might  argue  that  some  amount  of  work  has  to  be  expended  on  the  critical  
question  of  child  well-­‐being.    The  right  to  survival,  development,  and  well-­‐being  are  well  established.    The  
questions  are:  what  do  we  mean  by  well-­‐being  and  how  do  long  term  secular  improvements  in  child  well-­‐
being  relate  to  what  systems  can  and  cannot  do  relative  to  risk  and  protective  factors  that  affect  how  
children  grow  up.  
 

                                                                                                                     
17
 Gregson,  K.,  &  Yates,  R.  (2010).  UNICEF  UPDATE  (unpublished  slide  set).  
18
 UNICEF.  (undated).  Draft  ToR  on  the  Child  Protection  MERG  

14  
 
 A  paper  which  clarifies  the  interaction  between  bioͬŐĞŶĞƚŝĐ͚ŝŶĚŝĐĂƚŽƌƐ͛  and  the  more  malleable  aspects  
of  well-­‐being  is  an  important  contribution  to  the  large  debate  not  to  mention  the  implications  for  how  
one  conducts  monitoring  and  evaluation.  
 
o Methods  paper:    As  noted  earlier,  one  can  divide  monitoring  and  evaluation  into  two  discrete  phases,  
although  the  intent  in  doing  so  is  not  to  suggest  a  linear  process  when  it  is  clear  the  process  is  not  linear.  
 
The  first  phase  is  data  capture  (e.g.,  data  collection);  the  second  is  data  analysis.    Each  phase  has  
separate,  embedded  processes.      Nevertheless,  data  collection  and  data  analyses  require  coordination  
or  choreography  between  the  types  of  questions  one  wants  to  answer,  the  data  collected,  and  the  
means  used  to  answer  those  questions.    A  methods  paper  could  incorporate  a  compilation  and  synthesis  
of  information  on  measurements  and  indicators  that  would  be  applicable  for  child  protection  M&E.  
Evaluation  in  particular  often  involves  one  issue  in  particular:    high-­‐end  statistical  methods.    Use  of  
sophisticated  methods  is  not  required  in  every  instance  but  in  some  cases  the  value  of  the  work  done  is  
diminished  if  proper  consideration  as  to  the  means  of  analysis  is  not  given.    The  virtue  of  the  more  
sophisticated  methods  has  to  do  with  what  one  learns  (potentially)  about  cause  and  effect  relationships;  
the  challenge  has  to  do  with  expertise  that  needs  to  be  assembled  in  order  to  do  the  work.    Expertise  is  
in  some  ĐŽŶƚĞdžƚƐŵŽƌĞĚŝĨĨŝĐƵůƚƚŽ͚ĂĐƋƵŝƌĞ͛͘  
 
A  methods  paper  would  be  one  way  to  articulate  the  issues  involved,  including  the  challenges  one  faces  
when  using  one  set  of  methods  as  opposed  to  another.    For  example,  how  one  describes  the  status  of  
children  descriptively  and  by  what  means  (quantitative  versus  qualitative  methods,  for  example)  affects  
how  one  generalizes  to  larger  populations,  cost,  and  detail  (how  much  can  one  say  about  children).    
Interviews  with  children  provide  rich  detail,  but  it  is  much  harder  to  replicate  over  time  and  much  harder  
to  generalize  without  large  samples.    With  social  indicator  data  it  is,  relatively  speaking,  easier  and  less  
costly  to  track  well-­‐being  over  time  at  the  child  level,  the  community  level,  and  the  national  level,  but  the  
level  of  detail  is  often  lacking.    The  choices  implied  involve  trade-­‐offs,  costs,  and  opportunity  costs.    A  
methods  paper  would  highlight  what  is  involved.  

15