MODULE 4

ASSESSMENT RATIONALE AND PRACTICES

NATALIA LYTKINA

Module 4 1

Table of Contents
THE NCEA AS A SYSTEM OF QUALIFICATION ASSESSMENT

3-5

INTRODUCTION OF A NEW SYSTEM

INTENTIONS AND PURPOSES:
- FLEXIBILITY
- CONSISTENCY
- TRANSPARENCY
- LINKS WITH ACADEMIC, TECHNICAL AND VOCATIONAL TRAINING
- DEVELOPING LIFELONG LEARNING DISPOSITIONS

OUTLINE OF THE PROGRAMME DESIGN PROCESS

6-9

DESING DECISION-MAKING FRAMEWORK

BIG GOALS IN MATHEMATICS AND STATISTICS
FLOW CHART OF THE PROCESS

UNDERSTANDING MODERATION PROCESSES

10-19

WHAT IS MODERATION? WHY IS MODERATION IMPORTANT?

- CONSISTENCY
- COMPARABILITY
- THE PROCESS INVOLVES
- IN SUMMARY: WE MODERATE TO
MODERATION IN SCHOOLS
- ASSESSMENT AS A PART OF MODERATION
- PROFESSIONAL DIALOGUE
- MODERATION OF OTJ
- EVALUATION
MODERATION PROCESSES FOR NCEA INTERNAL ASSESSMENT
- INTERNAL MODERATION
- EXTERNAL MODERATION
- IN SUMMARY

Module 4 2

NCEA as a system of qualification assessment

Part 1: introduction of a new system.
The National Certificates of Educational Achievement (NCEA) are the official
secondary school qualifications in New Zealand. They were phased in between
2002 and 2004, replacing three older secondary school qualifications. The New
Zealand Qualification Authority administers the NCEA. There are three levels of
NCEA: Level 1 replaced School Certificate in 2002, Level 2 replaced Sixth Form
Certificate in 2003 and Level 3 replaced University Bursary qualifications in
2003 and 2004. Since its initial implementation in 2002, the NCEA has been
reviewed and improved: all standards curriculum related were scrutinized and
refined so that they are aligned to the NZC. The reviewed standards are being
implemented over the last three years.

Part 2: intensions and purposes.

Although there was a lot of debate as to whether evolvement of the NCEA was
evolutionary or revolutionary, it is now recognised that a new system of
qualifications was needed by any means and it is a sensible and inevitable
product of the previous 30 years (Bill Lennox, QANews #38, June 2001). The
underlying idea was to bring about an assessment and national qualifications
system based on developed standards that would recognise a wider range of
knowledge, skills, and abilities of school leavers and reflect the more flexible
environments of New Zealand schools.

My understanding of the main intentions behind as well as the main purposes
of the NCEA is that the effort was to develop and introduce a system that:
Provides more flexibility in teaching and learning;
Provides a rich and accurate picture of students skills and knowledge;
Is transparent;

Module 4 3

Enables students to prepare for the full range of academic or vocational

pathways;
Equips students for life-long learning.

Flexibility in teaching and learning.
The flexibility of the NCEA is reflected both in course design and time frame, by
giving schools the greater flexibility to develop a range of programmes to suit the
specific needs of their students (teachers can decide which standards to include,
and how many be unit or achievement, and how many will be internally or
externally assessed), and for students and teachers deciding when students
should be assessed for a particular standard (completion of NCEA is not
constrained by a one-year timeframe).

Consistency.
While offering schools and students much greater flexibility in teaching and
learning, the NCEA framework maintains consistency in assessment. The NCEA
system provides the opportunity for students to be assessed and for teachers to
assess students performance in a wider range of competencies and skills
throughout the school year by using internal assessment along with external. In
addition, being a standards-based system, the NCEA more accurately measures
students abilities and reflects students strengths and weaknesses because
students who have gained credits for a particular standard have demonstrated
the required skills and knowledge for that standard. NZQA has a formal quality
assurance process to ensure that the assessment of each standard is fair across
all students, regardless of the school they attend.

Transparency of the NCEA is quite obvious:
Students, teachers, parents, employers and tertiary providers have
access to all NCEA assessment information;
Candidates can review their marked work and, if they wish, apply for a
review or reconsideration of their results;
National statistics from external and internal assessments, reports are
published;
The standards and examination markers used for assessments are
publicly available;
Examples of students work that meet the standards along with
commentary from moderators are available.

Providing links with academic, technical and vocational training
The NCEA is an assessment system, which is aligned to practice at tertiary level.
The balance of internal and external examinations in NCEA courses is in line with
that used at universities and polytechnics. The unit standards provide a link with
technical and vocational training. Students can start to specialise while they are
at school, or can keep their options open. The NCEA is a national qualification on
the NZQF. Standards that students achieve as a part of NCEA can be used as
building blocks for other qualifications. One study has suggested that low
achieving students who have been disengaged in earlier years might be even
encouraged to re-engage in learning if they experience success in gaining unit
standards credits in a context for which they can see personal relevance and
Module 4 4

practical value, early in the school year (Boyd, with McDowall & Ferral, 2006).
The NCEA also rewards students who achieve at high levels through certificate
and course endorsements. Moreover, the fact that NCEAs are recognized by
employers, and used as the benchmark for selection by universities and
polytechnics, both in New Zealand and overseas, is very encouraging for school
leavers.

Developing lifelong learning dispositions.
The intention of the NCEA assessment regime that allows students be assessed
when ready and so come to see themselves as successful learners is one of the
conditions necessary to encourage the development of lifelong learning
dispositions (Hipkins, 2005). Other research suggests that by focusing more on
internally assessed standards where teachers can support students to
demonstrate their learning, and by limiting less confident students exposure to
external examinations (Hipkins, R., Vaughan, K., Beals, F., Ferral, H., & Gardiner,
B., 2005), the NCEA is providing students with more opportunities to see
themselves as successful learners, and thus encouraging among all students, not
just those who taking the traditional academic path, the development of lifelong
learning dispositions.






























Module 4 5

 Outline of the programme design process

Our plans miscarry because they have no aim. When a man does not know what
harbour he is making for, no wind is the right wind.
Seneca

The NZC gives schools and teachers the flexibility to make their own decisions
about planning learning programmes to meet the diverse needs of their learners.
This decisions need to be aligned with The New Zealand Curriculum document.
The NCEA supports this flexible approach to learning programme.

Design decision-making framework: important factors to consider.

Context

Conditions
for learning

Content

How does the

programme Yit into the
3 year progression?

How Ylexible are the

learning opportunitis?

How does the content

relate to your students'
world?

How does the

programme connect to
the school's priorities?

How culturally
responsive is the
programme?

How Ylexible is the

content selection?

How does the

programme contribute
to raising
studentachievement?

What are the

assessment
opportunities?

How does the content

connectto students'
learning in other
subjects?

Where does this

programme lead to for
students?

What avidence do you use

to monitor effectiveness of
the programme?

What student data do

you use to inform
planning?

Big goals in mathematics and statistics (senior secondary on tki):
Students understanding mathematical and statistical concepts;
Students thinking mathematically and statistically, and seeing the
connections between them;
Keeping doors open for all students to do mathematics and statistics in
future years;
Equipping students with the mathematical and statistical knowledge and
skills to be global citizens in the 21 century;
Making mathematics and statistics real;
Recognising that mathematics and statistics underpin many other areas
and making connections with these areas;
The place of mathematics and statistics within the qualification
framework.

The flow chart below illustrates my understanding of key considerations
that are involved in this process.

Module 4 6

Use the NZC as a starting point:

Make links to the values, principles, and
key competences; consider the big goals
in mathematics and statistics; the role it
can play in terms of the NZC vision.

Gathering information to inform

decisions: a fact-finding stage in which
the needs of community and available
resources are assessed.

Get to know your learners,

parents, and community,
Iwi/hapu: to address and
meet the diverse needs of
students.

Get to know assessment

policy and procedure in your
school, guidelines for
reporting to parents,
cumulative records contents,
ICT available and etc.

Making decision about approaches
to a programme design: achievement
objectives, NS, the Literacy LP, subject
specifics, vocational pathways, NCEA,
effective pedagogy.

Content and context:
Build on prior knowledge,
broad contextual themes
familiar and relevant to
students, local and global
contexts, connect with
other subjects and learning
areas, academic or
vocational pathways.

Assessment and
evaluation: feedback, set
up formative assessment to
inform teaching and
learning prior to
summative assessment
(school exams), NCEA
assessment opportunities
align with school policy and
NZQA.

Curriculum alignment:
motivating, challenging,
and encouraging learning
environment; key concepts,
high expectations, learn to
learn, clear progression,
meet literacy requirements
opportunities to satisfy AO,
NS, NCEA, university
entrance requirements.

Timing and management:

workload and allotted time;
consider exams,
assessments, calendar
events, term dates, and etc.,
plan various learning
activities (excursions, local
sites), plan review and
evaluation throughout the
year.

Implementation and review:

whether the desired outcomes
have been met; monitor and
evaluate, implement changes for
curriculum improvement.

Module 4 7

Module 4 8

In summary,

Effective programmes in senior Mathematics and Statistics:
Are designed to address student needs;
Are coherent and have meaning for students;
Support a broad vision and goals;
Include content and contexts that students will connect with their wider
lives;
Facilitate collaborative learning;
Offer students an element of choice;
Are tied up into appropriate curriculum objectives;
Generate authentic opportunities for assessment;
Set up assessment so that it will inform further leaning.




































Module 4 9

 Understanding Moderation processes

Part 1: What is moderation? Why is moderation important?

Moderation is the term used to describe approaches for arriving at a shared
understanding of standards and expectations for the broad general education.
On TKI Assessment Online 2010 moderation is explained as a process of teachers
sharing their expectations and understanding of standards with each other in
order to improve the consistency of their decisions about student learning and
achievement. During moderation process the teachers work towards making
judgements that are consistent and comparable.

Moderation is concerned with the consistency, comparability and fairness of
professional judgments about the levels demonstrated by students(Maxwell 2002).

What do we mean by consistency?
One of the explanations given by NSW Department of Education and Training is
quite straightforward:

Consistency of teacher professional judgment refers to the degree to which
judgements about students performance are independent of which teacher is
assessing the student.

Comparable means that similar interpretations of the underlying concepts or
skills can be made, using different evidence. Comparable assessment judgments
result from teachers comparing their assessments with an agreed matrix,
progression or specific assessment characteristics and agreeing on a level or
standard.

Students can be set different tasks or tests but demonstrate a common standard of
performance. (Maxwell, 2002, p.16)

There is a need for consistency (of teacher judgments and overall teacher
judgments):
Over time same evidence viewed at different times leading to same
judgment of same teacher;
Against benchmarks or standards equivalent application across
different types of evidence;
By a teacher;
Between teachers within same school and different schools.
(Moderation: professional learning modules for teachers on tki.org.nz).

Moderation leads to interpreting and applying levels or standards in equivalent
ways, and confirming teachers judgments about their students work.
(Moderation: professional learning modules for teachers on tki.org.nz).




Module 4 10

The process involves teachers sharing evidence of learning and collaborating

closely to establish a shared understanding of what quality of evidence and the
achievement of outcome look like and whether or not the student has
demonstrated achievement of the outcome or the standards and at what level.
Moderation helps teachers to make reliable, valid, evidence-based decisions.

Moderation involves:
ConversaHons
about planning for
moderaHon,
sharing
expectaHons;
collecHng and
analysing
evidence of
student learning

Adjustment of
judgments to
align with
common
expectaHons,
benchmarks or
standards.

High quality
teacher
judgments:
appropriate,
comparable
and equitable

Comparison of
that evidence
against
expectaHons,
benchmarks or
standards.

(Moderation: professional learning modules for teachers on tki.org.nz)

In summary,

We moderate to:
Develop shared or common interpretation of standards and expectation
of what constitutes achievement of curriculum (National Standards);
Develop shared understanding of what students achievement look like;
Develop accuracy and reliability in making judgements;
Ensure judgements are equitable in terms of implementations for student
learning;
Strengthen the value of teachers judgements;
Inform well-targeted programmes.

Part 2: Moderation in schools.

Schools can design their assessment policies and moderation processes taking in
consideration many different factors to ensure it suits their situation and needs.
Schools use moderation to increase dependability of teacher judgments.

Module 4 11

National Standards Factsheet states schools need to establish a moderation

process within their assessment programme. The process need to consider how
teachers interpret National Standards as well as how they make their judgements
from the assessment information they have gathered.

The moderation process in school begins with the planning of teaching, learning
and assessment.

Planning for assessment is a part of moderation process.

There are some important questions that needs to be considered:
Do activities allow all learners to develop and demonstrate their
knowledge and understanding, skills, attributes and capabilities?
Do the approaches to assessment enable you to focus on the progress of
each learner?
Is there sufficient variety in the evidence gathered to allow judgements
about progress and next steps?
How are samples of learners work checked against the success criteria or
marking scheme?
Do you ensure that judgements are shared with colleagues to reach a
shared understanding?
How confident are you that your judgements can be justified to the
learner or parent?

This chart illustrates planning for assessment as a part of moderation process
(Education Scotland: adopted and slightly modified):

Design the
learning
intenHons (the
NZC , Learning
Progressions,
Standards,
Achievement
ObjecHves,Course
SpecicaHons
e.t.c.)

Agree the types

of task that
would allow
learners to
demonstrate the
success criteria

Discuss and
reach consensus
on the related
success criteria
for learners

Module 4 12

Professional dialogue is fundamental to moderation and should take place at all

stages of assessment, including the planning stage. In collaboration with others,
agree approaches to learning, teaching and assessment.

Key questions to be considered include:
Do the learning intentions (LI) and success criteria (SC) match the
standards and expectations outlined in the National Standards, Learning
Progressions, etc?
Are learners involved in working out the success criteria?
Do the learners understand the purpose of the assessment activities?

To ensure that learning and teaching and assessment are planned in a coherent
way, it is important that moderation and verification activities include not only
making judgments about learners work but also include moderating approaches
to planning learning, teaching and assessment.

Professional dialogue is an integral part of moderation in schools:

Is the
assessment
t for
purpose?

Does the
evidence
allow
judgements
about
progress and
next steps?

How is
professional
dialogue
encouraged
and
developed?

What
informaHon
do you
collect and
use?

How do you
know what
quality
achievement
looks like?

(Education Scotland: adopted and slightly modified)

Professional dialogue is an integral part of moderation in schools. Therefore, to
make a professional judgement it is essential to understand how professional
dialogue is encouraged and developed among teachers within a group, within a
school or between different schools. Teachers in NZ schools have regular
opportunities to collaborate in examining evidence and agreeing standards. The
moderation and verification process is about arriving at a shared understanding
of the quality of learners work. To help develop a shared understanding of

Module 4 13

standards, schools have arrangements in place to ensure effective dialogue is

taking place within and across learning communities.

Teachers may agree by:
Working together across different departments within a school;
Working with colleagues in associated schools/colleges or with schools
within the local authority;
Becoming involved in professional learning activities.

Professional dialogue as an integral part of moderation in schools:

What input
does each
member of
staff have on
what is
collected and
how?
How is
consistency
promoted, and
how are
judgements and
procedures
rened for
future use?

How is
professional
dialogue
encouraged
and
developed?

What
opportuniHes
are given to
explain the
context of the
evidence?

On what basis
do sta
compare and
discuss
judgements?

(Education Scotland: adopted and slightly modified).

Regular scheduled meetings for teachers working at the same level and in the
same subject/curriculum area provide opportunities for teachers to collaborate
and participate in professional dialogue and collegiate working. This include
teachers discussing and agreeing the best approaches for quality assurance and
moderation, building on existing good practice, including monitoring, self-
evaluation and planning for improvement.

Moderating OTJ.

When teachers draw together evidence to form an overall teacher judgment
there is a need to ensure consistency of those judgements between teachers.
The National Standards Factsheet states,
An overall teacher judgment involves drawing on and applying the evidence
gathered up to a particular point in time in order to make an overall judgment
about a students progress and achievement. Using a range of approaches allows
Module 4 14

the student to participate throughout the assessment process, building their

assessment capability.... No single source of information can accurately summarise
a students achievement or progress. A range of approaches is necessary in order to
compile a comprehensive picture of the areas of progress, areas requiring
attention, and what a students progress looks like.
Overall Teacher Judgment: triangulation of evidence. Triangulated data:
observation, diagnostic, standards. Triangulation of information increases the
dependability of the OTJ.

This chart from professional learning modules for teachers gives a transparent
representation of stages which Moderation of overall teacher judgments
comprise of.

Plan assessment
activities
Reolection and
discussion with peers
to increase validity
and consistency

Clear indications of
progress from oirst
attempts to current
performance

Teacher
collaboraHon

Student
progress

Adequacy of
evidence

Evidence

Did task assess what

Demonstration of
intended to assess?
knowledge, process
Sufoicient evidence of
and skills in different
achievement to assign
contexts or
level or standard?
curriculum areas
Relative performance
with other peers?

(Moderation: professional learning modules for teachers on tki.org.nz).

By moderating OTJ schools check that teachers are making consistent judgement
within and across classes, and at some stage, across schools.
The National Standards and curriculum recourses such as LLP/Numeracy
progressions support consistency of judgment and moderation by providing
examples and illustrations of the standard required.
National Standards Factsheet (MOE):
Assessments of evidence are made using specific shared criteria. The criteria may
be exemplified through annotated examples and other national resources (e.g. the
Running Record DVD/booklet, the Diagnostic Interview and Getting Started
Numeracy Development Project Books and New Zealand Curriculum Exemplars).
Module 4 15

Part 3: Moderation Process for NCEA internal assessment.

Internal moderation: quality assurance system operating in every institution.

A major component of quality assurance of NCEA is an internal moderation
procedure. As it was aforementioned, schools are required to establish a
rigorous internal moderation system. Assessment and Examination Rules and
Procedures, NZQA to have assessment policies and procedures in place to unsure
that reported results for internal assessment are accurate, consistent, and
appropriate (that is, the tasks are in line with the standards and allow students to
demonstrate their ability). It is also expected that schools have effective and
documented processes to ensure that they report reliable results from internal
assessment to NZQA. On an annual basis, the Principal of a school with Consent
to Assess must nominate a staff member to be the Principal's Nominee (who may
be the Principal). One of the main duties of PN as described by NZQA is
maintaining and monitoring quality assurance practices within the school to
ensure the validity and credibility of assessment for qualifications. NCEA resource
kit (Managing Internal Moderation) indicates purposes for Internal Moderation
as:
Moderation is about checking that assessment materials and marking is at
the national standard;
This is necessary to ensure that internal assessment in schools is both
credible and robust; and that only results subject to internal moderation are
sent to NZQA.

Internal moderation procedures at schools must serve to guarantee:

All assessment material is checked or critiqued prior to use. Checking
means thoroughly reviewing assessment materials before first use,
regardless of source, to ensure (NCEA resource kit: Managing Internal
Moderation):
o Context is suitable for our students or modified;
o Authenticity can be assured or make modifications;
o Appropriateness of language for our students;
o Consistency with the registered standard
The critiquing process is to make certain the assessment activity focuses
on the requirements specified in the standard and provides the
opportunity for students to present evidence at all grades.

There is consistency of assessment and judgments across multiple classes
(inter-class consistency of grading). According to NCEA resource kit:
Managing Internal Moderation inter-class consistency means, ensuring
work of all students for a standard/activity is marked in the same manner,
irrespective of class or teacher. They suggest that it may be achieved by:
o Strip marking (e.g. Teacher A marks all of Q1, Teacher B marks all
of Q2...);
o Panel marking;
o Sharing reviewed assessment schedule while marking;
o Reference to guinea pig papers/annotated benchmarks;
Module 4 16

o Check-marking a proportion of each others work.

Samples of assessor grade judgements are verified for all standards.
Verify means (NCEA resource kit, Managing Internal Moderation):
o Another subject specialist familiar with the standard confirms
your marking is at the national standard;
o The samples should be at grade boundaries;
o There is no set number to check.
How to have a sample verified? The NCEA kit suggests:
o In larger departments, some inter-class consistency methods may
double as verification;
o Make reference to benchmark samples in recent moderation
reports for the standard;
o Email/send samples to a colleague (reciprocal arrangement for
mutual benefit);
o Share samples at an association or cluster meeting.
Staff is using external collegial links to maintain a current understanding
of the national standards; maintain students understanding of national
standards.
Feedback from external moderation is followed up as required; all
recommendations are acted before the materials are used again.
All assessment material is reviewed prior to further (or future) use.

Ways of gaining and maintaining an understanding of national standards
(ideas from various NCEA resources):
- Use critiqued tasks and annotated exemplars on TKI subject pages;
- Previously moderated tasks;
- Hold a workshop to review tasks;
- Share reviewed tasks with other schools;
- Participate in professional development for teachers provided by clusters.
Verify that marking is at the national standard:
- Check grade boundary samples with a colleague;
- Use meetings/email/Skype/Google Docs/video- conference/etc. to
discuss student work and grade boundaries with colleague(s);
- Use clarification documents for standards provided by moderators;
- Moderation newsletters, examiners reports, NZQA circulars etc.;
- Best Practice Workshops run by moderators (NZQA).

External Moderation: the aim is to equip teachers to make accurate and
consistent judgement.

An important part of the quality assurance system for NCEA is called external
moderation. External moderation is one of the ways that NZQA monitors the
consistency and quality of internal assessment across the country. This involves
NZQA moderators (teachers with subject expertise) checking the quality of
teachers' assessments of students' work.
NZQA has a good framework to help schools maintain or improve their internal
assessment capability. There are opportunities for NZQA and teachers to work
with each other in ways that support ongoing improvements in internal
Module 4 17

assessment: assessment workshops, provision of clarification statements by

moderators, SRMs helping schools to administer NCEA. These include feedback
to teachers on their assessments of students' work and regular reviews of
schools' quality systems. Schools use these reviews to request help from NZQA.

Through external moderation, NZQA is able to:
Give feedback to teachers to help them assess students' work;
Identify aspects of assessment practice within schools, or particular
subjects within schools, that could improve;
Provide assurance to schools and teachers that assessment practices are
robust; and calculate moderator/teacher agreement rates (a measure of
the extent to which moderators and teachers agree on whether samples
of student work meet the applicable standards.

Evaluation is imperative to moderation:
How might our
moderaHon
processes be
extended to
other curricular
areas?
What did the
moderaHon
process reveal
about the
knowledge of the
curriculum and
progression in
learning, or about
assessment?

What further
professional
development
might be
needed?

How does this

impact on our
improvement
planning
process?

How can the

informaHon
gained from
moderaHon be
shared across
other areas of
the school?

Quality assurance procedures such as moderation, both at a local and national
level (internal and external), provide opportunities for staff to reflect on the
performance of learners.

The assessment information gathered through these processes also informs both
individual and establishments self-evaluation and improvement planning.
Through learning together and sharing good practice next steps can be
identified.






Module 4 18

In Summary,

Taking all aforementioned important aspects of moderation into consideration
and using basic facts from MNA leaflet, it could be summarised that:
The schools assessment information, policies and procedures are:
Helping teachers to carry out the schools assessment policy and procedures;
Communicating to students and their families effectively;
Regularly reviewed for accuracy and are comprehensive and up to date.
The school complies with moderation requirements by:
Reporting only results that have been subject to internal moderation and are
complete, up to date and checked for accuracy;
Submitting materials for external moderation and using moderation reports
to validate and/or improve assessment quality.

Module 4 19