Social work effectiveness evaluation
                      FinSoc-team 2012


20/09/2011     Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   1
Social work evaluation and EBSW-
                 researchc / national level
  • R&D-work is done mainly by Universities, Social
    Welfare Centres of Expertise and National Institute
    for Health and Welfare
  • In communes, few developing projects have been
    running, mainly small and local projects
  • Social work’s national targets are usually not very
    clearly stated, and this makes social work
    evaluation far more difficult as the aims and
    objectives have to be developed in the process of
    evaluation



20/09/2011     Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   2
FinSoc
• Enhances knowledge of evaluation methodology
  and promotes evidence based social work
• Several R&D-projects on evaluation since 1997
• Experts in the group:
        –    Anu Muuri (Head of the Unit)
        –    Minna Kivipelto (Researh manager)
        –    Pekka Karjalainen (Senior researcher)
        –    Sanna Blomgren (Researcher)
        –    Paula Saikkonen (Researcher)
        –    Aune Flinck (Development manager)
        –    Hilkka Nyström (Project assistant)




20/09/2011        Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   3
FinSoc
• Provides evaluation methods for the social sector
• New method developments have been launched,
  where also clients and service users are involved
             •   Realist Evaluation (Karjalainen 2012)
             •   Mirror Method (Yliruka 2006)
             •   Critical Evaluation (Kivipelto & Yliruka 2012)
             •   Participatory Evaluation models (Kivipelto 2008)
             •   AVAIN-measure (Kivipelto & Karjalainen 2012)
• Supports the development of social services by
  conducting evaluation research
• The group co-operates actively with various actors
  in the social sector nationally and internationally


20/09/2011        Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   4
The Finnish Evaluation of Social Services group (FinSoc)

• Currently
        – Gathers information about evaluation measures and
          models of effective adult social work; supports
          evidence based social work.
        – The potential for using single-case and realistic
          evaluation in adult social work is under examination
        – Involvement in National programmes (evaluation
          support): KASTE, Innokylä
        – International networks: INTSOCEVAL (International
          network), Practice Research and Evidence Informed
          Practice (European networks), Nord Plus (Nordic
          network) etc.
        – Dissertations of social work evaluation


20/09/2011    Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   5
In Finland, further development is
                           needed
• National instruments and data-collection methods
• Higher education on social work evaluation (no
  master-level education in evaluation)
• Motivation and commitment
• Structures that integrate research, higher education,
  and social work practice
• International networks and joint-projects around
  evaluation research (measure and method
  development, case-studies, evidence based social
  work-projects etc.)



20/09/2011      Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   6
Challenges in the social work evaluation
                            in Finland

• Standardized scales and measures:
        – Useful in evaluating certain function
        – Useful when tracking changes in the given factor
        – There are often limitations in relation to the
          generalizability of the findings, and their replication or
          transferability in different contexts
• Systematic reviews
        – Synthesizing the results of several studies
        – It possible to generalize the results to a broader
          context
        – Applying the information needs always professional
          reflection and decision making
        – Review process is slow
20/09/2011     Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   7
Evaluation should take into account the
                 complex nature of social work
• Social work evaluation needs a comprehensive
  model, where client’s needs, problems and goals
  are considered against his/her situation and
  environment
• Social work also needs to include questions about
  ”whose perspective”, ”what context” and ”in relation to
  what?”
• Any comprehensive evaluation model must take into
  account the organizational and larger social factors,
  as these contexts also influence workers and how
  the organization is functioning as a whole



20/09/2011       Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   8
It has noticed that the
                                                                 collected information is not
                                                                 evidence until it has been
                                                                 interpreted




20/09/2011   Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc   9
References
•    Karjalainen, P. (2012) Realistinen arviointi kuntouttavan sosiaalityön vaikuttavuudesta. Teoksessa: Pohjola,
     Anneli; Kemppainen, Tarja & Väyrynen, Sanna (toim.) Sosiaalityön vaikuttavuus. Rovaniemi: Lapin
     yliopistokustannus, 166-202.
•    Kivipelto, M. (2012) Developing Effectiveness Evaluation in Adult Social Work (EEA). Paper presentation at 10th
     EES Biennale Conference: evaluation in the Networked Society: New Concepts, New Challenges, New Solutions.
     3rd/5th October. Helsinki, Finland.
•    Kivipelto, Minna & Blomgren, Sanna (2012) Social work with adults as a tool for tackling exclusion. In: Karvonen,
     Sakari; Keskimäki, Ilmo, Kuronen, Maria & Wilskman, Kaarina (ed.) Annual Review 2012. Helsinki: National
     Institute for Health and Welfare. Available at: [http://www.thl.fi/thl-client/pdfs/734b5ed8-decc-451f-9dd3-
     4a19dc4bfa3c]
•    Kivipelto, M. & Karjalainen, P. (2012) Developing Effectiveness Evaluation in Adult Social Work (EEA). Paper
     presentation at Evidence Informed Practice. April 22nd/25th, 2012. Cavan, Ireland.
•    Kivipelto, M. Karjalainen, P. & Blomgren, S. (2012) Developing Effectiveness Evaluation in Adult Social Work
     (EEA). Poster presentation at Evaluation for Practice. June 18th/20th 6, 2012. Pori, Finland.
•    Kivipelto, Minna, Kotiranta, Tuija, Kazi, Mansoor A.F., Korteniemi, Pertti, Jauhiainen, Tuula & Borg, Pekka.
     Finnish experiences of developing effectiveness evaluation in social work: case Helsinki Social Services
     Department, the Unit of Social Rehabilitation. European Journal of Social Work. (Manuscript in referee-process)
•    Kivipelto, M., Kotiranta, T. & Korteniemi, P. (2011) Finnish experiences of developing effectiveness evaluation in
     social work. Paper presentation at Inter-Centre Network for the Evaluation of Social Work Practice
     (INTSOCEVAL). September 26th / 27th, 2011, Olten.
•    Kivipelto, Minna & Yliruka, Laura (2012) Mirror method as an approach for critical evaluation in social work.
     Critical Social Work 13(2). Available at: [http://www.uwindsor.ca/criticalsocialwork/]
•    Kivipelto, M. & Yliruka, L. (2011) Mirror method as an approach for critical evaluation in social work. Paper
     presentation at International and Inter-disciplinary Workshop on Practice Research. June 8, 2011. Helsinki.
     Available online at: [http://www.vits.org/?pageId=380]




20/09/2011          Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc       10

Finsocinenglish2012

  • 1.
    Social work effectivenessevaluation FinSoc-team 2012 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 1
  • 2.
    Social work evaluationand EBSW- researchc / national level • R&D-work is done mainly by Universities, Social Welfare Centres of Expertise and National Institute for Health and Welfare • In communes, few developing projects have been running, mainly small and local projects • Social work’s national targets are usually not very clearly stated, and this makes social work evaluation far more difficult as the aims and objectives have to be developed in the process of evaluation 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 2
  • 3.
    FinSoc • Enhances knowledgeof evaluation methodology and promotes evidence based social work • Several R&D-projects on evaluation since 1997 • Experts in the group: – Anu Muuri (Head of the Unit) – Minna Kivipelto (Researh manager) – Pekka Karjalainen (Senior researcher) – Sanna Blomgren (Researcher) – Paula Saikkonen (Researcher) – Aune Flinck (Development manager) – Hilkka Nyström (Project assistant) 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 3
  • 4.
    FinSoc • Provides evaluationmethods for the social sector • New method developments have been launched, where also clients and service users are involved • Realist Evaluation (Karjalainen 2012) • Mirror Method (Yliruka 2006) • Critical Evaluation (Kivipelto & Yliruka 2012) • Participatory Evaluation models (Kivipelto 2008) • AVAIN-measure (Kivipelto & Karjalainen 2012) • Supports the development of social services by conducting evaluation research • The group co-operates actively with various actors in the social sector nationally and internationally 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 4
  • 5.
    The Finnish Evaluationof Social Services group (FinSoc) • Currently – Gathers information about evaluation measures and models of effective adult social work; supports evidence based social work. – The potential for using single-case and realistic evaluation in adult social work is under examination – Involvement in National programmes (evaluation support): KASTE, Innokylä – International networks: INTSOCEVAL (International network), Practice Research and Evidence Informed Practice (European networks), Nord Plus (Nordic network) etc. – Dissertations of social work evaluation 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 5
  • 6.
    In Finland, furtherdevelopment is needed • National instruments and data-collection methods • Higher education on social work evaluation (no master-level education in evaluation) • Motivation and commitment • Structures that integrate research, higher education, and social work practice • International networks and joint-projects around evaluation research (measure and method development, case-studies, evidence based social work-projects etc.) 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 6
  • 7.
    Challenges in thesocial work evaluation in Finland • Standardized scales and measures: – Useful in evaluating certain function – Useful when tracking changes in the given factor – There are often limitations in relation to the generalizability of the findings, and their replication or transferability in different contexts • Systematic reviews – Synthesizing the results of several studies – It possible to generalize the results to a broader context – Applying the information needs always professional reflection and decision making – Review process is slow 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 7
  • 8.
    Evaluation should takeinto account the complex nature of social work • Social work evaluation needs a comprehensive model, where client’s needs, problems and goals are considered against his/her situation and environment • Social work also needs to include questions about ”whose perspective”, ”what context” and ”in relation to what?” • Any comprehensive evaluation model must take into account the organizational and larger social factors, as these contexts also influence workers and how the organization is functioning as a whole 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 8
  • 9.
    It has noticedthat the collected information is not evidence until it has been interpreted 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 9
  • 10.
    References • Karjalainen, P. (2012) Realistinen arviointi kuntouttavan sosiaalityön vaikuttavuudesta. Teoksessa: Pohjola, Anneli; Kemppainen, Tarja & Väyrynen, Sanna (toim.) Sosiaalityön vaikuttavuus. Rovaniemi: Lapin yliopistokustannus, 166-202. • Kivipelto, M. (2012) Developing Effectiveness Evaluation in Adult Social Work (EEA). Paper presentation at 10th EES Biennale Conference: evaluation in the Networked Society: New Concepts, New Challenges, New Solutions. 3rd/5th October. Helsinki, Finland. • Kivipelto, Minna & Blomgren, Sanna (2012) Social work with adults as a tool for tackling exclusion. In: Karvonen, Sakari; Keskimäki, Ilmo, Kuronen, Maria & Wilskman, Kaarina (ed.) Annual Review 2012. Helsinki: National Institute for Health and Welfare. Available at: [http://www.thl.fi/thl-client/pdfs/734b5ed8-decc-451f-9dd3- 4a19dc4bfa3c] • Kivipelto, M. & Karjalainen, P. (2012) Developing Effectiveness Evaluation in Adult Social Work (EEA). Paper presentation at Evidence Informed Practice. April 22nd/25th, 2012. Cavan, Ireland. • Kivipelto, M. Karjalainen, P. & Blomgren, S. (2012) Developing Effectiveness Evaluation in Adult Social Work (EEA). Poster presentation at Evaluation for Practice. June 18th/20th 6, 2012. Pori, Finland. • Kivipelto, Minna, Kotiranta, Tuija, Kazi, Mansoor A.F., Korteniemi, Pertti, Jauhiainen, Tuula & Borg, Pekka. Finnish experiences of developing effectiveness evaluation in social work: case Helsinki Social Services Department, the Unit of Social Rehabilitation. European Journal of Social Work. (Manuscript in referee-process) • Kivipelto, M., Kotiranta, T. & Korteniemi, P. (2011) Finnish experiences of developing effectiveness evaluation in social work. Paper presentation at Inter-Centre Network for the Evaluation of Social Work Practice (INTSOCEVAL). September 26th / 27th, 2011, Olten. • Kivipelto, Minna & Yliruka, Laura (2012) Mirror method as an approach for critical evaluation in social work. Critical Social Work 13(2). Available at: [http://www.uwindsor.ca/criticalsocialwork/] • Kivipelto, M. & Yliruka, L. (2011) Mirror method as an approach for critical evaluation in social work. Paper presentation at International and Inter-disciplinary Workshop on Practice Research. June 8, 2011. Helsinki. Available online at: [http://www.vits.org/?pageId=380] 20/09/2011 Minna Kivipelto, DSc, Research Manager, Technologies and Practices Evaluation Unit, Finsoc 10