Program Evaluation and Performance Measurement

Program Evaluation and Performance Measurement

• 1.
  Program Evaluation and PerformanceMeasurement www.humanikaconsulting.com
• 2.
  Seta A. Wicaksana 081119 53 43 [email protected] • Managing Director of Humanika Amanah Indonesia – Humanika Consulting • Managing Director of Humanika Bisnis Digital – hipotest.com • Wakil Ketua Asosiasi Psikologi Forensik Indonesia wilayah DKI • Business Psychologist • Certified of Human Resources as a Business Partner • Certified of Risk Professional • Certified of HR Audit • Certified of I/O Psychologist • Dosen Tetap Fakultas Psikologi Universitas Pancasila • Pembina Yayasan Humanika Edukasi Indonesia • Penulis Buku : “SOBAT WAY: Mengubah Potensi menjadi kompetensi” Elexmedia Gramedia 2016, Industri dan Organisasi: Pendekatan Integratif menghadapi perubahan, DD Publishing, 2020 • Organizational Development Expertise • Sedang mengikuti tugas belajar Doktoral (S3) di Fakultas Ilmu Ekonomi dan Bisnis Universitas Pancasila Bidang MSDM • Fakultas Psikologi S1 dan S2 Universitas Indonesia • sekolah ikatan dinas Akademi Sandi Negara
• 3.
  Agenda Program Evaluation •What is program evaluation? • What types of program evaluations are there? • Program evaluation • Design evaluation • Process evaluation • Outcome evaluation • Impact evaluation • Cost-effectiveness evaluation Performance Measurement • What is performance measurement? • How do we determine good measures? • How is performance measurement different from program evaluation?
• 4.
  Introduction • Performance measurementand program evaluation can both help identify areas of programs that need improvement and determine whether the program is achieving its goals or objectives. They serve different but complimentary functions: • Performance measurement is an ongoing process that monitors and reports on a program's progress and accomplishments by using pre-selected performance measures. • Program evaluation, however, uses measurement and analysis to answer specific questions about how well a program is achieving its outcomes and why. • So, performance measurement data describes program achievement, and program evaluation explains why we see those results.
• 5.
  Program Evaluation • What isprogram evaluation? • What types of program evaluations are there? • Program evaluation • Design evaluation • Process evaluation • Outcome evaluation • Impact evaluation • Cost-effectiveness evaluation
• 6.
  What Is Program Evaluation? • Programevaluations are individual systematic studies conducted to assess how well a program is working and why. • Objectives of Program evaluations are: • Support new and innovative approaches and emerging practices • Identify opportunities to improve efficiency and effectiness • Continuously improve existing programs • Subsequently, improve human health and the environment
• 7.
  What Types Of Program Evaluations AreThere? • Program evaluation • Design evaluation • Process evaluation • Outcome evaluation • Impact evaluation • Cost-effectiveness evaluation
• 8.
  Program Evaluation • Program evaluationscan assess the performance of a program at all stages of a program's development. • The type of program evaluation conducted aligns with the program's maturity (e.g., developmental, implementation, or completion) and is driven by the purpose for conducting the evaluation and the questions that it seeks to answer. • The purpose of the program evaluation determines which type of evaluation is needed.
• 9.
  Design Evaluation • Adesign evaluation is conducted early in the planning stages or implementation of a program. • It helps to define the scope of a program or project and to identify appropriate goals and objectives. • Design evaluations can also be used to pre-test ideas and strategies.
• 10.
  Process Evaluation • A processevaluation assesses whether a program or process is implemented as designed or operating as intended and identifies opportunities for improvement. • Process evaluations often begin with an analysis of how a program currently operates. • Process evaluations may also assess program activities and outputs, policies, program design or customer expectations.
• 11.
  Outcome Evaluation • Outcomeevaluations examine the results of a program (intended or unintended) to determine the reasons why there are differences between the outcomes and the program's stated goals and objectives (e.g., why the number and quality of permits issued exceeded or fell short of the established goal?). • Outcome evaluations sometimes examine program processes and activities to better understand how outcomes are achieved and how quality and productivity could be improved.
• 12.
  Impact Evaluation • An impactevaluation is a subset of an outcome evaluation. • It assesses the causal links between program activities and outcomes. • This is achieved by comparing the observed outcomes with an estimate of what would have happened if the program had not existed.
• 13.
  Cost- effectiveness Evaluation Cost-effectiveness evaluations identify programbenefits, outputs or outcomes and compare them with the internal and external costs of the program.
• 14.
  Performance Measurement • What isperformance measurement? • How do we determine good measures? • How is performance measurement different from program evaluation?
• 15.
  What Is PerformanceMeasurement? • Performance measurement is a way to continuously monitor and report a program's progress and accomplishments, using pre-selected performance measures. • By establishing program measures, offices can gauge whether their program is meeting their goals and objectives. • Performance measures help programs understand "what" level of performance is achieved.
• 16.
  How do we determinegood measures? Measurement is essential to making cost-effective decisions. We strive to meet three key criteria in our measurement work: • Is it meaningful? → Measurement should be consistent and comparable to help sustain learning. • Is it credible? → ​Effective measurement should withstand reasonable scrutiny. • Is it practical? → Measurement should be scaled to an agency's needs and budgetary constraints.
• 17.
  How is performance measurement differentfrom program evaluation? • A program sets performance measures as a series of goals to meet over time. • Measurement data can be used to identify/flag areas of increasing or decreasing performance that may warrant further investigation or evaluation. • Program evaluations assess whether the program is meeting those performance measures but also look at why they are or are not meeting them.
• 18.
  Simple Case • Forexample, imagine you bought a new car that is supposed to get 30 miles per gallon. But say, you notice that you are only getting 20 miles per gallon. That's a performance measurement. You looked at whether your car was performing where it should be. So what do you do next? You would take it to a mechanic. The mechanic's analysis and recommendations would be the program evaluation because the mechanic would diagnose why the car is not performing as well as it should. • You need performance measures to know whether your program (or car) is performing where it should be, and you do a program evaluation (or go to the mechanic) to find out the reason why it is not meeting those expectations.
• 19.
  I don't wantto give myself grades. I will leave evaluation of my achievements to history. - Helmut Kohl
• 20.
  Learning and Givingfor Better Indonesia www.humanikaconsulting.com