Downloaded 11 times
More Related Content
Similar to Dylan Willam- Leadership for Teacher Learning- Times Festival of Education
Dylan Willam- Leadership for Teacher Learning- Times Festival of Education
- 1. Leadership for teacher learning DylanWiliam www.dylanwiliam.net1
- 2. Outline In theUK, teacher quality is highly consequential, and highly variable Teacher quality can be improved by replacing existing teachers with better ones by investing in the teachers we already have Only investing in existing teachers produces enough improvement to produce the changes that we need This is not happening systematically in most schools, but could be 2
- 3. Educational Endowment Fund(EEF) toolkit 3 Intervention Cost Quality of evidence Extra months of learning Feedback ££ +8 Metacognition and self-regulation ££ +8 Peer tutoring ££ +6 Early years intervention £££££ +6 One to one tuition ££££ +5 Homework (secondary) £ +5 Collaborative learning £ +5 Phonics £ +4 Small group tuition £££ +4 Behaviour interventions £££ +4 Digital technology ££££ +4 Social and emotional learning £ +4
- 4. Educational Endowment Fund(EEF) toolkit 4 Intervention Cost Quality of evidence Extra months of learning Parental involvement £££ +3 Reducing class size £££££ +3 Summer schools £££ +3 Sports participation £££ +2 Arts participation ££ +2 Extended school time £££ +2 Individualized instruction £ +2 After school programmes ££££ +2 Learning styles £ +2 Mentoring £££ +1 Homework (primary) £ +1
- 5. Educational Endowment Fund(EEF) toolkit 5 Intervention Cost Quality of evidence Extra months of learning Teaching assistants ££££ 0 Performance pay ££ 0 Aspiration interventions £££ 0 Block scheduling £ 0 School uniform £ 0 Physical environment ££ 0 Ability grouping £ -1
- 6. Important caveats aboutresearch findings 6 Educational research can only tell us what was, not what might be. Moreover, in education, “What works?” is rarely the right question, because everything works somewhere, and nothing works everywhere, which is why in education, the right question is, “Under what conditions does this work?”
- 7. An illustrative example:feedback 7 Kluger and DeNisi (1996) review of 3000 research reports Excluding those: without adequate controls with poor design with fewer than 10 participants where performance was not measured without details of effect sizes left 131 reports, 607 effect sizes, involving 12652 individuals On average, feedback increases achievement Effect sizes highly variable 38% (50 out of 131) of effect sizes were negative
- 8. Getting feedback rightis hard Response type Feedback indicates performance… falls short of goal exceeds goal Change behavior Increase effort Exert less effort Change goal Reduce aspiration Increase aspiration Abandon goal Decide goal is too hard Decide goal is too easy Reject feedback Feedback is ignored Feedback is ignored
- 9.
- 10. Teacher quality andstudent learning Subject Correlation Woodhead All 0* Rivkin, Hanushek and Kain (2005) Reading 0.10 Rivkin, Hanushek and Kain (2005) Mathematics 0.11 Rockoff (2004) Reading 0.20 Rockoff (2004) Mathematics 0.25 Aaronson, Barrow and Sander (2007) Mathematics 0.13
- 11. Annual growth inachievement, by age 11 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 5 6 7 8 9 10 11 12 13 14 15 16 annualgrowth(SDs) Age Bloom, Hill, Black, and Lipsey (2008)
- 12. 12 Assuming that one year’s student growth is 0.3 standard deviations the correlation between teacher quality and student achievement is 0.15 Then with a good teacher (1sd above the mean) students learn 50% more And with an outstanding teacher (2sd above the mean) they learn 100% more Note also that students make some progress through maturation so these are probably underestimates of the true effect (Fitzpatrick, Grissmer, Hastedt, 2011)
- 13. Teacher quality 13 Theimpact of teacher quality (Hanushek Rivkin, 2006) In the classroom of the best teacher in a group of 50 teachers, students learn twice as fast as average. In the classroom of the least effective teacher in a group of 50, students learn half as fast as average And in the classrooms of the best teachers, students from disadvantaged backgrounds learn as much as others (Hamre Pianta, 2005) Teachers make a difference But what makes the difference in teachers? In particular, can we predict student progress from: Teacher qualifications? Value-added? Teacher observation?
- 14.
- 15. Teacher qualifications andstudent progress 15 Mathematics Reading Primary Middle High Primary Middle High General theory of education courses Teaching practice courses Pedagogical content courses Advanced university courses Aptitude test scores Harris and Sass (2007) Mathematics Reading Primary Middle High Primary Middle High General theory of education courses — Teaching practice courses — + Pedagogical content courses + + Advanced university courses — + Aptitude test scores —
- 16.
- 17. Framework for teaching(Danielson 1996) Four domains of professional practice 1. Planning and preparation 2. Classroom environment 3. Instruction 4. Professional responsibilities Links with student achievement (Sartain, et al. 2011) Domains 1 and 4: no impact on student achievement Domains 2 and 3: some impact on student achievement 17
- 18. A framework forteaching (Danielson, 1996) Domain 2: The classroom environment 2a: Creating an environment of respect and rapport 2b: Establishing a culture for learning 2c: Managing classroom procedures 2d: Managing student behavior 2e: Organizing physical space Domain 3: Instruction 3a: Communicating with students 3b: Using questioning and discussion techniques 3c: Engaging students in learning 3d: Using assessment in instruction 3e: Demonstrating flexibility and responsiveness
- 19. Reading -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 Unsatisfactory Basic ProficientDistinguished Teachervalue-added 2a 2b 2c 2d 2e 3a 3b 3c 3d 3e
- 20. Mathematics -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 Unsatisfactory Basic ProficientDistinguished Teachervalue-added 2a 2b 2c 2d 2e 3a 3b 3c 3d 3e
- 21. Observations and teacherquality 21 -15 -10 -5 0 5 10 15 20 Unsatisfactory Basic Proficient Distinguished Percentagechangeinrateoflearning Reading Mathematics Sartain, Stoelinga, Brown, Luppescu, Matsko, Miller, Durwood, Jiang, and Glazer (2011) So, the highest rated teachers are 30% more productive than the lowest rated But the best teachers are 400% more productive than the least effective
- 22. Imprecision of lessonobservations 22 Achieving a reliability of 0.9 in judging teacher quality through lesson observation is likely to require observing a teacher teaching 6 different classes, and for each lesson to be judged by 5 independent observers. Hill, Charalambous and Kraft (2012)
- 23. 23 To recap The highest rated teachers generate learning 30% faster than the lowest rated teachers But the most effective teachers generate learning 400% faster than the least effective teachers So the best observation systems we have capture less than 10% of teacher quality (in reality, less than this because most teachers are in the middle two categories) There is therefore a real danger that teachers will “game the system” by aping features of observation protocols while becoming less effective.
- 24.
- 25. Teacher value-added Student fixed-effectsmodel Traditionalmodel 1 2 3 4 5 1 38 22 24 16 0 2 26 28 15 20 11 3 20 20 20 24 16 4 13 24 26 13 24 5 9 5 12 28 47 Goldhaber, Goldschmidt, and Tseng (2013)
- 26. Mathematics -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0 1 23 4 Value-added(studentSDs) Years of teaching experience Quintile of teacher value-added 5th 4th 3rd 2nd 1st Atteberry, Loeb and Wyckoff (2013)
- 27. Correlation of initialand later performance 27 0 0.1 0.2 0.3 0.4 0.5 Base+1 Base+2 Base+3 Base+4 Base+5 Correlationofvalue-addedwithbaseyear Year of prediction All teachers New teachers Mathematics
- 28. Correlation of initialand later performance 28 0 0.1 0.2 0.3 0.4 0.5 Base+1 Base+2 Base+3 Base+4 Base+5 Correlationofvalue-addedwithbaseyear Year of prediction All teachers New teachers Reading
- 29. Issues with value-addedmodels for teachers 29 Different (reasonable) models of value-added give very different estimates of teacher quality (Goldhaber, Goldschmidt and Tseng, 2013) Teacher value-added in their first year accounts for less than 5% of the variation in teacher quality in their fifth year of teaching (Atteberry, Loeb and Wyckoff, 2013) Teachers benefit students for at least three years after they stop teaching them (Rothstein, 2010)
- 30. What makes effectiveteacher learning? 30
- 31. Expertise According toBerliner (1994), experts: Excel mainly in their own domain Often develop automaticity for the repetitive operations that are needed to accomplish their goals Are more sensitive to the task demands and social situation when solving problems Are more opportunistic and flexible in their teaching than novices Represent problems in qualitatively different ways than novices Have faster and more accurate pattern recognition capabilities Perceive meaningful patterns in the domain in which they are experienced Begin to solve problems slower but bring richer and more personal sources of information to bear 31
- 32. The role ofdeliberate practice Music professors at the Hochschule der Kuenst (Academy of Music) Berlin identified 10 violin students who had the potential for careers as international soloists (“best” students) The professors also identified a sample of 10 good, but not outstanding students (“good” students) Researchers recruited another 10 students training to be music teachers who specialised in the violin (“Music Ed” students) An additional 10 middle-aged professional violinists from two local orchestras were recruited to the study Groups were matched in sex (7f, 3m) and for the first three groups, age 32
- 33. How much doviolinists practice? 0 5 10 15 20 25 30 35 4 9 14 19 Hoursofpractieperweek Age Music Ed Good Best Professionals Ericsson, Krampe, Tesch-Römer (1993) 33
- 34. Violinists’ hours ofpractice (cumulative) 0 2000 4000 6000 8000 10000 4 6 8 10 12 14 16 18 20 Cumulativehoursofpractice Age Music Ed Good Best Professionals 34
- 35. These differences aresubstantial… Hours of practice by age 18 Music Education students 3420 Good violin students 5301 Best violin students 7410 Professional musicians 7336 By the age of 18, the best violinists have accumulated 40% more practice than good violinists Since the amount of deliberate practice being undertaken by the best students once they are adults is close to the maximum possible, it is, essentially, impossible for the good students to catch up to the best. 35
- 36. Psychomotor skills andsurgical skill… Ten junior surgical residents were tested on their ability to perform five anastomoses on fresh porcine jenunum Performance evaluated on: a. Time taken in minutes b. A “clumsiness” score (0 to 100), taking into account “unproductive movements, fumbles and gross errors” c. An anastomosis score found by rating on a 1 to 3 scale: suture spacing knots overall tidiness integrity of the anastomosis (tested with 10 cm head of water) accuracy of suture in terms of extramucosal placement Cumulative error score: a + b + 2 x (15 – c)
- 37. …are only weaklyrelated 37 Residents also took three psychomotor tests: Small parts manual dexterity test (Crawford Crawford, 1981) Spiral maze test (Gibson, 1961) Embedded figures test (Witkin, Oltman, Karp, 1971) Results: Surgical performance correlated negatively with tests of manual dexterity, but Improvement in surgical skill correlated positively with the hidden figures score The ability to create, and hold in mind, mental representations appears to be more important than manual dexterity in predicting surgical performance Steele, Walder, and Herbert (1992)
- 38. Visual-spatial ability andsurgical skill 38 37 junior surgical residents at the University of Toronto given six tests of visual/spatial ability Snowy pictures test (Ekstrom et al., 1976) Gestalt completion test (Street, 1931) Shape memory test (Ekstrom et al., 1976) Cube comparison test (Ekstrom et al., 1976) Form board test (Sylvester, 1881) Mental rotations test (Vandenberg Kuse, 1978)
- 39. Visual-spatial ability andsurgery skills Residents were then tested on their ability to learn and perform complex surgical procedures (two-flap and four-flap Z–plasty on pig thighs)
- 40. Correlation of surgicalskill with spatial ability 40 Procedure Try SPT GCT SMT CCT FBT MRT Total global rating Two-flap 1st — — — — — 0.40 Two-flap 2nd — — — — — — Four-flap — — — — 0.34 0.47 Quality of result Two-flap 1st — — — — — 0.40 Two-flap 2nd — — — — — — Four-flap — — — — 0.40 0.49 Wanzel, Hamstra, Anastakis, Matsumoto Cusimano (2002)
- 41. General conclusions aboutexpertise Elite performance is the result of at least a decade of maximal efforts to improve performance through an optimal distribution of deliberate practice What distinguishes experts from others is the commitment to deliberate practice Deliberate practice is an effortful activity that can be sustained only for a limited time each day neither motivating nor enjoyable—it is instrumental in achieving further improvement in performance 41
- 42.
- 43. Effects of experiencein teaching -5 -4 -3 -2 -1 0 1 0 1 2 3 to 5 Extramonthsperyearoflearning Years of teaching experience 43 -5 -4 -3 -2 -1 0 1 0 1 2 3 to 5 Years of teaching experience Mathematics Reading Rivkin, Hanushek and Kain (2005)
- 44. Implications for educationsystems Pursuing a strategy of getting the “best and brightest” into teaching is unlikely to succeed Currently all teachers slow, and most actually stop, improving after two or three years in the classroom Expertise research therefore suggests that they are only beginning to scratch the surface of what they are capable of What we need is to persuade those with a real passion for working with young people to become teachers, and to continue to improve as long as they stay in the job. There is no limit to what we can achieve if we support our teachers in the right way
- 45. Supportive accountability Whatis needed from teachers: A commitment to: The continual improvement of practice Focus on those things that make a difference to students What is needed from leaders: A commitment to engineer effective learning environments for teachers by: Creating expectations for continually improving practice Keeping the focus on the things that make a difference to students Providing the time, space, dispensation, and support for innovation Supporting risk-taking 45
- 46. References Aaronson, D., Barrow,L., Sander, W. (2007). Teachers and student achievement in the Chicago Public High Schools. Journal of Labor Economics, 25(1), 95-135. Atteberry, A., Loeb, S., Wyckoff, J. (2013). Do first impressions matter? Improvement in early career teacher effectiveness. Washington, DC: Center for Analysis of Longitudinal Data in Educational Research. Berliner, D. C. (1994). Expertise: the wonder of exemplary performances. In J. N. Mangieri C. C. Block (Eds.), Creating powerful thinking in teachers and students: diverse perspectives (pp. 161-186). Fort Worth, TX: Harcourt Brace College. Danielson, C. (1996). Enhancing professional practice: a framework for teaching. Alexandria, VA: ASCD. Ericsson, K. A., Krampe, R. T., Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363-406. Fitzpatrick, M. D., Grissmer, D., Hastedt, S. (2011). What a difference a day makes: Estimating daily learning gains during kindergarten and first grade using a natural experiment. Economics of Education Review, 30(2), 269-279. Goldhaber, D. D., Goldschmidt, P., Tseng, F. (2013). Teacher value- added at the high-school level: Different models, different answers? Educational Evaluation and Policy Analysis. doi: 10.3102/0162373712466938 Hamre, B. K., Pianta, R. C. (2005). Academic and social advantages for at-risk students placed in high quality first grade classrooms. Child Development, 76(5), 949-967. Hanushek, E. A., Rivkin, S. G. (2006). Teacher quality. In E. A. Hanushek F. Welsh (Eds.), Handbook of the Economics of Education (Vol. 2, pp. 1051-1078). Amsterdam, Netherlands: Elsevier. Harris, D. N., Sass, T. R. (2007). Teacher training, teacher quality and student achievement (Vol. 3). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. Hill, H. C., Charalambous, C. Y., Kraft, M. A. (2012). When rater reliability Is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56-84. Kluger, A. N., DeNisi, A. (1996). The effects of feedback interventions on performance: a historical review, a meta- analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254-284. Rivkin, S. G., Hanushek, E. A., Kain, J. F. (2005). Teachers, schools and academic achievement. Econometrica, 73(2), 417-458. Rockoff, J. E. (2004). The impact of individual teachers on student achievement: evidence from panel data. American Economic Review, 94(2), 247-252. Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. Quarterly Journal of Economics, 125(1), 175–214. Sartain, L., Stoelinga, S. R., Brown, E. R., Luppescu, S., Matsko, K. K., Miller, F. K., Durwood, C.E., Jiang, J.Y., Glazer, D. (2011). Rethinking teacher evaluation in Chicago: Lessons learned from classroom observations, principal-teacher conferences, and district implementation. Chicago, IL: Consortium on Chicago School Research. Steele, R. J. C., Walder, C., Herbert, M. (1992). Psychomotor testing and the ability to perform an anastomosis in junior surgical trainees. British Journal of Surgery, 79(10), 1065–1067. Wanzel, K. R., Hamstra, S. J., Anastakis, D., Matsumoto, E. D., Cusimano, M. D. (2002). Effects of visual-spatial ability on learning of spatially-complex surgical skills. Lancet, 359(9302), 230–231. 46