Assessing Grammar and

Vocabulary

Grammatical Knowledge
Communicative

Competence consist of four components:

grammatical, sociolinguistic, discourse and strategic
competencies. In this model, Grammatical Competence
was defined as knowledge of rules of phonology, lexis,
syntax and semantics.

Larsen-Freeman(1991,1997)

influenced by the
communicative model of language pedagogy,
characterised grammatical knowledge to show the
interrelatedness of phonology, lexis, syntax and semantics.

Grammatical

knowledge consisted of three interconnected

elements :
1. grammatical forms or the structures of a language
2. the grammatical meanings of those forms
3. their pragmatic meaning or use in a given context

The

three elements refer to

the following :
Form is both morphology and syntax
are concerned with the linguistic
accuracy of language.
Grammatical meaning consists of both
the literal and intended message that
is conveyed by the form.
The pragmatic or implied meaning
results from the appropriate language
choices a learner makes in a given
communicative event.

Grammatical

knowledge of form,
meaning and use occurs at the
sentence level and beyond the
sentence. ( Discourse level)
Purpuras (2004) framework offers
taxonomy of components of
grammatical knowledge along with a
list of possible grammar points that
could be used to measure each of the
points.

Designing Assessment
Tasks : Selected Response

SR

tasks can be language ( or

nonlanguage as in a gesture or
picture) of any length from one
word to several sentences of
discourse.
Test-taker expected to select the
correct response, which is meant
to measure the knowledge of
grammatical form and/or
meaning.

Multiple Choice Tasks

Most

SR tasks presents a blank or underlined

words in a sentence and the test-taker must
choose the correct answer from the options.
Advantage : Easy to administer or score
Disadvantage : Difficult to create, promote
guessing and sometimes viewed as not being
authentic language use.

Examples of Multiple Choice Response

Grammatical Form
Carson : Did you see the movie Titanic last week?
Ethan
: Yes, Mary loved it and ____________________ .
A) I loved too
B) I do
C) do did I
D) so do I
The

first part of this example provides the

context for the response, but because all
four responses convey the intended
meaning, an understanding of the context is
not essential to get the item correct.

Grammatical Meaning
Yuko
: Do you have plans for tonight?
Christina : Not really. ______________________
Yuko
: Thanks, but I have a final paper to write.
A)
B)
C)
D)

How about you?

Need any help?
How about a movie?
Need to work?

First

part asks a question about a future event. In

the second part, correct response could be any of
the four options because they are all grammatically
correct, but the third part of the dialog provides the
context that indicates the meaning of the correct
response, which is a refusal to a suggestion.

Grammatical Form and Meaning

Yuko
: Are you visiting your family this year?
Christina : I dont know. _________________ ; it depends on the
airfares.
A)
B)
C)
D)

I
I
I
I

didnt
may be
might
had to

Here

both the grammatical form and meaning of

the options need to be considered for the correct
response. As the second part of the dialogue
indicates, there is a sense of uncertainty and
added to that is the indication that the visit is a
future event. Thus the correct response is
limited to one that shows both these aspects.

Grammatical Form and Meaning ( Error

identification )
In America, most children begin to work at home, where they
are having daily
A
B
And/or weekly responsibilities such as washing the dishes and
feeding the dog.
C
D

In

this example the item is designed to

assess grammatical form, specifically the
present tense form (have) that is needed to
show the habitual actions of most American
children. These types of tasks are scored
either right or wrong because the test-taker
needs to identify only one error.

Discrimination Tasks
Another

type of SR task that ask the testtaker to attend to input that can be either
language or nonlanguage and to respond in
the form of a choice between contrast or
opposites, such as true/false.
Discrimination item are used to measure the
difference between two similar areas of
grammatical knowledge such as pronouns in
subject or object position.

To

test recognition of gender pronouns, the testtaker must be able to discriminate between two
pictures, correctly choosing the picture that
corresponds to the stimulus sentence.

Noticing Tasks or Consciousness-Raising

Tasks
Contain

wide range of input in the form of

language or nonlanguage and are considered
particularly helpful for learners.
Test-takers are asked to indicate (underline or
circle) that they have identified a specific
feature in the language sample.

Example:
Directions

: Circle would when it refers

to the habitual past. Underline it when
it refers to the present or future.

You know? You think youve got it

bad. When I was kid we would
have to walk up hill to and from
school everyday. We would even
do it when it snowed-winter or
summer..

DESIGNING
ASSESSMENT TASK
LIMITED PRODUCTION
BY MEA

Input in the item language or non language information

- could be simple sentence or a longer stretch of discourse

Response limited amount of language production

- can vary from a single word to a sentence depending on
the grammatical ability or construct that is defined.
- range of possible correct answer is large.

Scoring dichotomous : one criterion for correctness

: form or meaning
- Partial credit : multiple criteria for correctness
: form and meaning
: allows for adding up the scores
: Example : full (2), partial (1), no credit (0)

Gap-Filling Tasks

Language is presented in the form of a sentence, dialogue or passage.

Some words are deleted.

Deletions- to test one or more areas of grammatical knowledge (vocabulary

knowledge, grammatical ability)

Response- must be chosen based on the context given. (page 300)

Grammatical form
I _____________ a book right before I go to sleep. Recently, I have been
reading biographies. I ____________ the biographies of Charlie Chaplin and
Mahatma Ghandi, and for the past few nights, I _______________ about Oprah
Winfrey.

Grammatical form and meaning

The Mississipi River, also called the Great River, is the longest river in the
United States. The river ___________________ at Lake Itasca in the state of
Minnesota. The river ends at the Gulf of Mexico where it ___________________
over half a million cubic feet of water into the gulf.

Short-Answer Tasks

Input - presented in form of a question or questions following a reading

passage
or oral/visual stimulus.

Response vary from a single word to a sentence or two.

- must be grammatically correct and meaningful based on
the question.

Scoring can be dichotomously (right or wrong) for a single criterion for

correctness.
- partial credit for multiple criteria for correctness.
(page 301)

Dialogue-Completion Tasks

Input presented in the form of a short conversation or dialogue.

- some parts are left blank.

Response should be grammatically correct.

Criterion for correctness can be the form or form and meaning

- (page 302)

ASSESING GRAMMAR AND

VOCABULARY
Designing Assessment Tasks : Extended Production

 Presented in form of a prompt

Vary in length
Can be either language or
nonlanguage (gesture or
picture) information

Obtain larger amounts of

language from test-taker
Allow more creative
construction
Elicit instances of authentic
language use

Input

Purpose

Scoring
rubric

Respons
e

Construct rating scale define

grammatical ability that will
be assessed and levels of
ability
Explicitly describe in scoring
rubric

Usually open-ended with a

number of possible correct
options
Scored using rating scales

Scoring rubric

Scoring rubric on a five-point scale for

assessing the knowledge of syntax adapted

from Bachman and Palmer (1996)

Information gap tasks

Present input in
terms of incomplete
information

One test-taker is
given half or some
of the information
and another test
taker is given
complementary
information

Both have to
question each other
to get all the
information

To measure
Suitable to test
grammatical knowledge to
communicate functional
meaning

pragmatic knowledge
(reciprocal nature of
performance requires language
of test taker to display
politeness, formality,
appropriateness)
knowledge of question
information, other interactional
form and meaning and request
for clarification

Example of an info-gap task

adapted from Purpura (2004)

Role play or simulation tasks

using rating
scale since
response
are
extended
and do not
have one
correct
answer

scored

contain
large
amount of
language to
measure
test-takers
grammatica
l form,
meaning
and
pragmatic

response

Input

take on a
role or
simulate a
situation to
solve a
problem,
make a
decision or
perform
some
transaction
collaborativ

Example of problem-solving role play

ASSESSING
VOCABULARY

INTRODUCTION
OWords are building blocks of a

language (create sentences, larger

paragraphs, whole texts)
ONative speaker acquire vocabulary in
childhood
O2nd language learner jotting down,
memorizing lists of words etc.

TYPES :
do not
count
words that
TOKENS :
are
all the
repeated,
words in
only words
the
that are of
paragraph
different
(53)
There are big lies. And little lies.
forms

THE NATURE OF VOCABULARY

And somewhere in between there

are the lies we tell our doctors.
Even the most famous doctor of
all, Hippocrates, knew that those
pesky Greek patients might tell a
fib or two. To find out if they were
stretching the truth, Hippocrates

FUNCTION WORDS
E.g. : Preposition,
articles,
conjunctions and
other little words
Belonging more to
grammar than
vocabulary

CONTENT WORDS
E.g. : nouns, verbs,
adjectives and
adverbs
More focus on in
vocabulary tests

* Some vocabulary tests might focus

on phrasal verbs (put up with, run
into), compound nouns (personal
computer, fish tanks) or idioms
(action speaks louder than words).

PREFABRICATED LANGUAGE
OGroups of words that seems to

have a grammatical structure but

operate as a single unit and have
a particular function in
communication

4 types:
O Poly words (short fixed

phrases that perform a variety

function such as marking
fluency etc)
E.g. : so to speak, wait a
minute
O Institutionalized

expressions ( longer
utterances that are fixed in

O Phrasal constraints (medium-

length phrases that have basic

structure with one or two slots
that can be filled by various
words/ phrases)
E.g. : Yours truly/sincerely, as
far as I know
O Sentence builders (phrases

that provide the framework for

a complete sentence with one/
two slots where whole ideas
can be expressed)

DEFINING LEXICAL KNOWLEDGE

3 components make up Chapelles definition
of vocabulary ability:
1. The context of vocabulary use
2. Vocabulary knowledge and fundamental
processes
3. Metacognitive strategies for vocabulary
use

VOCABULARY IN CONTEXT
O The context of vocabulary may vary

across generations, formal and informal

language, varieties and dialects of
language as well as between nonspecialized or technical vocabulary.
O When teenagers talk about a babe or

describe an event as da bomb, the

context of the conversation means that the
first case is not a small baby but a rather
nice-looking girl and da bomb refers to
awesome
O In British English, the term to table

means to discuss now but for American

FUNDAMENTAL PROCESSES OF
VOCABULARY ACQUISITION
O Learners knowledge of:

~ word characteristic
~ perceiving different forms of
words
~ recognizing linguistic roots
to make
out meaning
~ using context for guessing
meaning
~ knowing the parts of speech

METACOGNITIVE STRATEGIES
O Strategies in trying to read illegible

handwriting
O Strategies to convey sad message
O Strategies when talking to non
native speaker of our language
O 2nd language learner
to overcome the lack of
vocabulary knowledge
when communicating (practice
avoidance use
lexical item)
- Paraphrase a word, fall back on
their first language.

CONSIDERATIONS IN DESIGNING
ASSESSMENT TASKS
PREPARED BY LIYANA AZIZ

Steps In Designing A Vocabulary

Test

Select your target words

Nation (1990) suggested teaching and testing of vocabulary should be
based on high-frequency words (basis for all language users)
Low-frequency words (less valuable)
Specialized vocabulary (often found in subject matter test than in
language test)
Subtechnical words (use to assess different meanings and
definition)

Determine mood of performance

Important features (receptive and productive)
Can receive and produce vocabulary in two ways:
i. Vocabulary recognition
ii. Vocabulary recall

Designing Assessment Tasks :

Receptive Vocabulary

It is crucial to consider what role the context plays in the test item.

Role of context :

i. to indicate specific meaning of a high-frequency word

Ii. help the learner to be able to recognize the word based on given context.

Example : Vocabulary in one sentence context : high-frequency word

My grandfather is a very independent
person.
A. never willing to give help
B. hard-working
C. not relying on other people
D. good at repairing things
a test taker must be able to show understanding of the underline adjectives.

Designing Assessment Tasks :

Receptive Vocabulary

Example : Vocabulary in one sentence context : Low-frequency word

The hazardous road conditions were

the cause of many fatal accidents.
A. difficult
B. problematic
C. dangerous
D. complicated
even a limited context can provide information that can help test-taker to
recognize and infer the meaning of lexical items

Example : Vocabulary matching exercise

Test taker match the target word with its meaning

pg 313

Designing Assessment Tasks :

Receptive Vocabulary

Example : Word Association

assess a leaners vocabulary size by present the target word as a stimulus

and ask learner to select a word that is closely associated with the target
word :
Team
Alternative
Orbit
together

chalk
sports

group

Designing Assessment Tasks :

Productive Vocabulary

Involve recall and use and are also better performed within a context or
situation.

Example : Fill-in-the-blank

Target vocabulary item is deleted from a sentence and test-taker must

understand the context in which the word occurs in order to produce the
missing word.

Example : Selective deletion cloze

Longer passages is used.

Assess other aspect of vocabulary (form and use of lexical item)

Test-taker must be able to identify not only the meaning but also the form of
the lexical needed to fill in the blank.

Designing Assessment Tasks :

Productive Vocabulary

Example : Selective deletion cloze

Pg 314 : the deletions working with children who had been ___________ away
in mental ____________.......

The first deletion indicates that the vocabulary item needed is a verb in a
form of past participle.

The next deletion needs to be noun and it must be in plural form.

Vocabulary choice can be dependent on the style and register of the

language used.

Pg 314 : and her experience with these __________...

The vocabulary item is a plural form noun. kids and children are suitable
words.

The formality of the language found (written text) would prefer children