The effects of naplan a preliminary report for wappa website

The effects of naplan a preliminary report for wappa website

• 1.
  The Effects ofNAPLAN: A preliminary report on teacher perceptions WAPPA Conference 2012 Dr Greg Thompson [email protected]
• 2.
  Introduction • In thinking about how to begin speaking to a group of principals, I am always mindful of the words of Brookfield: The more I know about their work, the more I feel humbled by their abilities … If I know too much about who they are and what they have done as educators, I start to say to myself ‘what on earth can I say to these people that has any chance of being taken seriously, or considered important, by them? (1994, p.207). • I am particularly conscious of the erroneous assumption often made about expertise • I am no expert on NAPLAN, rather I see myself as collecting, organising, making the voices of experts heard
• 3.
  Why NAPLAN? • Began out of my PhD work on how the concept of a ‘good student’ operates in high schools • I thought about how Education Policy is ‘done’ to people, how it impacts on those in schools, and how we seem to have lost those mechanisms that ask how policy is working • I also wondered how policy ‘actors’ (teachers, principals, school communities) engaged with NAPLAN, what strategies they use and why • No longer any mechanism for shared conversations between schools, universities, bureaucrats, politicians etc
• 4.
  NAPLAN – MyPosition • Broader Federal policy platform: accountability/transparency • Laudable aim: Improving the equity of education outcomes • Heavily influenced by USA and UK approaches to education. • Controversial and very difficult to quantify and/or qualify • Its impacts require scrutiny, coercive approaches to equity may have the opposite effects
• 5.
  An Overview ofthe Research • In 2011 the Australian Research Council awarded me a DECRA to look at the effects of NAPLAN on school communities • $375,000 over three years • Murdoch co-investing around $220,000 • Only 9% success rate with only 3 of the DECRA’s in Education • Three phases to the research: 1. A survey of teachers in SA and WA in 2012 2. Embedded fieldwork in WA schools in 2013 3. Embedded fieldwork in SA schools in 2014
• 6.
  What makes astandardised test „high-stakes‟? • Standardised tests are not new (WALNA, TAE, TEE ...) appropriate summative measure of large populations never ‘perfect’, always design issues not necessarily high-stakes What makes a standardised test „high-stakes‟ is how the data collected is used If some (parents) vote with their feet, that's exactly what the system is designed to do
• 7.
  Interesting stories… Opponentsof NAPLAN would deny parents and students information that sits in a bigger picture. They would deny schools that chance to identify others from which they might usefully learn. Worse, they claim that NAPLAN is creating "test-driven schooling". That is nonsense. If teachers burden students with numerous practice tests they are wasting time for students and themselves NAPLAN will never work as a diagnostic test. A diagnostic test should be administered by the teacher so that they can take action the next day. No national survey can return all of that the next day very easily. There are ways in which we can provide resources—for example, in computer adaptive tests that can provide feedback instantly. Perhaps in the future those sorts of resources can be made available. There is a lot of interest in doing that, but that is not a part of NAPLAN at this stage. It needs to be made clear to schools and teachers that excessive test practising ahead of NAPLAN is unnecessary. While it helps to be familiar with the structure of the tests, carrying out endless practices should not be encouraged. NAPLAN matters, but it is not the be all and end all.
• 8.
  US and UK:International impacts? Underperforming schools can be closed Students can be ‘held back’ Teachers have their value-added measures published/ranked Underperforming schools can be closed Funding tied to league tables Teachers can have their employment terminated based on test results (Polesel, Dulfell, Turnbull, 2012)
• 9.
  NAPLAN? • What makes NAPLAN high-stakes in Australia? publication of the results of the NAPLAN program on the MySchool website (Lobascher, 2011, pp. 9-10) associated media coverage realities of enrolments, encouraging parents “to vote with their feet”, implementation of the recommendations relating to Queensland’s 2008 poor NAPLAN performance (Lingard, 2010) Some more dire consequences in USA/UK. May be interesting to see what happens in 10 years here.…
• 10.
  Aus/US/UK Research www.effectsofnaplan.edu.au
• 11.
  Is there anotherside to the research? Research is rarely one-sided. A number of studies have found high-stakes testing has an educational benefit. They suggest it improves: learning, because what is expected is clearly understood efficiency because teachers/schools work harder increased comparability and cohesion across school systems and contexts. Published (generally) in management/ administration/economic journals
• 12.
  Survey Tool • The survey consists of 5 ‘sections’ that link to explore teacher perceptions about the impact of NAPLAN. These are: Demographics Stress Self-Efficacy Impact on their curriculum and pedagogy Achievement Goal Orientation • It is open to all teachers in WA and will close at the end of June… (www.effectsofnaplan.edu.au) • The more participants the more balanced/representative the information
• 13.
  Teacher Perceptions “You areasking teachers to make value judgements!” Absolutely. Teachers are actually quite good at making professional judgements. “Steps forward?” Valuing professional judgements of educators is a hallmark of Finnish approached to schooling But need to triangulate ...
• 14.
  Impacts on Curriculumand Pedagogy • 14 questions in this section • Derived from Lingard et al (1990) idea of ‘productive pedagogies’ • 20 observable characteristics of classroom pedagogy that represented ‘best practice’ • Many of these ideas we are now familiar with, but in the early 1990s were considered ‘revolutionary’
• 15.
  But… • Two main areas of criticism of the survey method of research: Questions lead the responses Only certain people respond, generally those that already have strongly held opinions at a certain point in time • Possible criticism: teachers responding are probably those with the most extreme/negative views about NAPLAN • If respondents are representative of certain pre-existing attitudes, interesting patterns emerge • The sub-set of questions examined in the following pages refer to impact on curriculum and were identified as statistically related
• 16.
  To summarise… 1. AcrossWA and SA teachers in low-SES schools perceive that NAPLAN has had a greater impact on curriculum in their classroom 2. Teachers in Government schools perceive greater impacts than those in Catholic schools and significantly more than Independent schools 3. Teachers in schools in WA perceive a greater impact on curriculum than their SA counterparts The next challenge is to explain why. Given the research addressed, it may be that NAPLAN is having an opposite effect than intended – widening the achievement/equity gap.
• 17.
  What other information willthe survey provide? • Next 6-12 months, research that examines: Teacher stress across demographics A comparison of positive and negative impacts of NAPLAN across demographics Impact on pedagogy across demographics Impact on teacher self-efficacy across demographics • Applied to ACARA Board to access all of the NAPLAN data for all schools in WA and SA from 2009-2011. This will enable a comparison point to the teacher perceptions. • Beginning Phase 2 of the research: case studies undertaken in WA school communities
• 18.
  Where to next? • Three future possibilities for NAPLAN (as I see it ...): Nothing changes, or we get more testing (canvassed at the Federal level, history from US/UK) Testing side of NAPLAN is kept, but the elements that make it so ‘high-stakes’ are removed or have their impact lessened NAPLAN is removed • “In what ways will schools respond to NAPLAN beyond that of excessive preparation and a return to teacher-centred pedagogies?” • Interested in comparing teacher perceptions with parents, principals and students.
• 19.
  And finally… • I am no expert on NAPLAN, I collect the stories of those people who understand it the best. • With that in mind, what could you suggest as interesting possibilities to look at in schools in WA?