Eighth lecture: Testing Reading PART 2

Asst. Prof. Narmeen Mahmood

Approaches to Reading:
What exactly is reading? In daily life, we certainly encounter a wide range of
reading material and tasks, yet it is difficult to define exactly what we mean by
reading. Is reading mainly taking in new information, or is it a process of
synthesizing new information with what we already know? Is reading a matter of
decoding symbols to form words and sentences, or is it understanding how
arguments are presented and recognizing typical texts used for certain purposes,
such as a narrative story or assembly instructions for a bookcase? In fact, then we
think of texts, do we think of prose passages, or do we consider advertisements,
maps, graphs, and cartoons as texts, too?

Today, most English language teachers would agree that reading includes
both:
 bottom-up skills ; recognizing and making sense of letters, words, and
s e nt ences.
 top-down processing that deals with whole texts.
Note: Read more about these two terms and provide
one definition for each.
They would also agree that text applies to both:
 linear passages of prose.
 non-linear sources of information such as maps and pie charts.
Reading is widely regarded as an interactive skill in which the
background knowledge or schemata that t h e reader brings to the task
is constantly interwoven with the new material. There are many processes
involved in reading, but also important are products or results o f
read in g .

1
Assessing reading comprehension skills (cognitive skills)
There are wide range of techniques are available for assessing reading
comprehension, i n c l u d i n g multiple- choice items which are no longer on the
top of the list of techniques; nevertheless, they are still used in TOEFL and ILETS
( Al-Juboury, 2014).
1. True /false items 5. Matching
2. Multiple -choice items 6. Labelling
3. Short answer 7. Cloze test
4. Rearrangement
(ibid.)
Reading comprehension techniques that tests cognitive skills
1. True/False /Not Given technique

Initially, T/F/NGs seem easier to write than MCQs with all their pitfalls.
However, there are points to keep in mind. Phrase all statements affirmatively
and avoid extremes such as always or never because they tip off students that
the answer is false. ( Note: For more information, refer to notes on how to design
T/F items in week 1).

2
 2. Multiple Choice Technique:
According to Coombe et al. (2010), a wide variety of reading subskills are
tested using MCQs, but the format is well-suited for testing the ability to
distinguish between:
1. Main ideas
2. Supporting details
Note: MCQs can be used to directly check on the main idea for:

a . a particular section of the text (such as a paragraph), o r

b. the text as a whole, often done through choosing the best title.

Note: Response options of main idea could be:

• JR (just right). This option should be the correct or best answer.

• TG (too general). This distractor relates to an option that- is too broad.
• TS (too specific). This distractor focuses on one detail within the text or
paragraph.
• OT (off topic). Depending on the level of the students, this distractor
is written so that it reflects an idea that is nQ1 developed in the
paragraph or text.
Note: Finding the main idea of a paragraph or longer passage of text is one
of the most important reading skills to master, along with concepts like:
making an inference, finding the author's purpose, or understanding
vocabulary words in context.
The following is example paragraph followed by multiple -choice items to
test different reading comprehension skills. The source of this example is
Brown(2004).

3
 Questions 1-10
The Hollywood sign in the hills that line the northern border of Los Angeles is a famous landmark
recognized the world over. The white-painted, SO-foot-high, sheet metal letters can be seen from great
distances across the Los Angeles basin.
(5)The sign was not constructed, as one might suppose, by the movie business as a means of celebrating the
importance of Hollywood to this industry; instead, it was first constructed in 1923 as a means of advertising
homes for sale in a 500-acre housing subdivision in a part of Los Angeles called "Hollywood land." The sign
that was constructed at the time, of course, said "Hollywood land." Over the years, people began referring
to the area by the shortened version "Hollywood," and after the sign and its site were donated to the city
in 1945, the last four letters were removed.
(10) The sign suffered from years of disrepair, and in 1973 it needed to be completely replaced, at a cost
of $27,700 per letter. Various celebrities were instrumental in helping to raise needed funds. Rock star
Alice Cooper, for example, bought an 0 in memory of Groucho Marx, and Hugh Hefner of Playboy fame
held a benefit party to raise the money for the Y. The construction of the new sign was finally completed
in 1978.

1. What is the topic of this passage?

a. A famous sign
b. A famous city
c. World landmarks
d. Hollywood versus Hollywoodland

2. The expression "the world over" in line 2 could best be replaced by-----------.
a. in the northern parts of the world
b. on top of the world
c. in the entire world
d. in the skies
3. It can be inferred from the passage that most people think that the Hollywood sign
was first constructed by:

a. an advertising company
b. the movie industry
c. a construction company
d. the city of Los Angeles

4. The pronoun "it" in line 5 refers to-------------.

a. the sign
b. the movie business
c. the importance of Hollywood
d. this industry
4
5. According to the passage, the Hollywood sign was first built in---------.
a. 1923
b. 1949
c. 1973
d. 1978
6. Which of the following is NOT mentioned about Hollywood land?
a. It used to be the name of an area of Los Angeles.
b. It was formerly the name on the sign in the hills.
c. There were houses for sale there.
d. It was the most expensive area of Los Angeles
7. The passage indicates that the sign suffered because--------------.
a. people damaged it
b. it was not fixed
c. the weather was bad
d. it was poorly constructed
8. It can be inferred from the passage that the Hollywood sign was how old when
it was necessary to replace it completely?
a. Ten years old
b. Twenty-six years old
c. Fifty years old
d. Fifty-five years old
9. The word "replaced" in line 10 is closest in meaning to which of the
following?
a. Moved to a new location
b. Destroyed
c. Found again
d. Exchanged for a newer one.
10. According to the passage, how did celebrities help with the new sign?
a. They played instruments
b. They raised the sign
c. They helped get the money
d. They took part in work parties to build the sign
Notice that this set of questions, based on a 250-word passage, covers the
comprehension of these features:
• main idea (topic) (done through skimming strategy )
• expressions/idioms/phrases in context.
• inference (implied detail).
5
 • grammatical features.
• detail (scanning for a specifically stated detail).
• excluding facts not written (unstated details).
• supporting idea(s).
• vocabulary in context.
Examples on open-ended reading comprehension questions (Note: these questions are
based on another reading passage).

1. What do you think the main idea of this passage is?

2. What would you infer from the passage about the future of air travel?
3. In line 6 the word sensation is used. From the context, what do you think this
word means?
4. What two ideas did the writer suggest for increasing airline business?
5. Why do you think the airlines have recently experienced a decline?
(ibid.)
According to Al- Juboury (2014), inferences and drawing a conclusion from facts or
reasoning is another reading skill is tested using this technique:
Direction: Check the following table then choose either “T” if the statement is True or
“F” if the statement is False.

A. Increased production can lower the cost of a product. T/F

B. Goods were more expensive early in last century than later on. T/F
C. There is an increasing interest in mechanical inventions. T/F
D. People are interested in items that cost more money. T/F

Data on the module "T" Ford

YEAR PRICE NUMBER PRODUCED

1909 $ 950 10.500
1910 $ 780 18.500
1911 $ 690 34,500
1912 $ 600 78,500
1913 $ 550 170,000
3.Short Answer Forma t
According to Coombe et al. ( 2010), all too often, short answer items focus on
details, but they can be used to assess other reading skills.

For example, students may supply an appropriate title for a chart or diagram
to check their grasp of main ideas. If chart, map, or graph reading is important
in your program, frame short answer items so students have to demonstrate
they understand how these graphics work. Check the example below:

Example 1 : Non-linear text

6
Scan the following chart on the World's Longest Rivers and answer each question with
three words or less.
1. Which river is longer than 4,000 miles? ----------------------.
2. How many of the longest rivers are in Asia? -----------------.
3. How many rivers start in mountainous areas? ---------------------------.
4. Which river is really two rivers together-------------------------------.
5. Which rivers are not important for shipping? -----------------------------.
6. Did the information come from the Internet or a book? ------------------.
7. Which rivers flow north? --------------.
8. Which river has the most water in it?---------------.

The World's Longest Rivers

Direction Length in Miles

Name Continent Starts in Ends in
of Flow

Nile Africa Lake Mediterra- North 4,180

Victoria nean Sea
Amazon* South. Andes Atlantic East 3,912
America Mountains Ocean

Mississippi North Rocky Atlantic South 3,710

(including America Mountains Ocean
Missouri
River)
Yangtze Asia Tibet Pacific East 3,602
Plateau Ocean
Ob Asia Altai Arctic North 3,459
Mountains Ocean

Example 2: Short answer questions based on a linear text

Test technique: Short answer questions that test understanding implicitly and explicitly
stated information( Al-Juboury,2014) (Note: the underlined information is a reading
comprehension skill).

7
Direction: Read the following interview and answer the questions
Interviewer: This is Paul, Ashley, Michael, and Carlos. They are basketball players. Now,
you're American, aren't you?
Ashley: Well, Paul and I are American. But, Carlos and Michael aren't.
Michael: No, I'm not American. I'm from Canada, and Carlos is from Cuba.
Interviewer: An international team! And you're on tour in England.
Ashley: That's right.
Interviewer: How old are you?
Paul: I'm 21. Ashley and Michael are 22.
Carlos: And I'm 24.
Interviewer: OK. Thanks! And welcome to England.
1. Where are Paul and Ashley from?
2. Are Michael and Carlos American?
3. Where are they from?
4. How old are Paul and Michael?
5. Is Carlos 21?
Another Example :

Note: this example is taken from first -intermediate textbook (English for Iraq).
8
 3. Matching: in this example, the addressed reading comprehension skill is:
Understanding the communicative value or text function (AL- Juboury, 2014).
Direction: Match the following texts and their functions

1. Persuasion a)
2. Warning A: Is there a sport centre near here?
B: Yes there is one near Wendover
3. Giving information
Street. It is next to the art gallery.
4. Giving directions
5. Invitation
6. Asking for permission
c)
b) The Turkish government
We request the Pleasure of Your intends to build a major
Company at a Party Which will dam in the south east to
be given At 21 Park Street, generate electricity.
London On June 19th (7 p.m.)

d)
Ahmed : A chicken sandwich, please.
Waitress: I'm sorry. There is nothing left.
But we've got delicious cheese
sandwich.
Ahmed: OK a cheese sandwich,
please. And a coffee.

f)
You mustn't feed the animals in the zoo.
e)
You mustn't go too close to the lion cage.
Is it OK if I come late
on Thursday night? A
friend of mine and I like
to play video games in
his house.

9
 4. Sequencing Tasks (or Rearrangement)
Coombe et al(2010) say that there are many ways to check students'
understanding of the organization of a text, but sequencing is especially useful
in:

 reading a narrative
 following steps in instructions or a recipe,
 or understanding biographical or historical events.
Note: understanding of the organization of a text is one of the reading
subskills.
Cognitively, these tasks are closely related to text insertion tasks
because they focus on seeing how the parts of a text fit together to form a
coherent whole.
Note: When you design these tasks, it is important to create logical
linkages between each component so that it is clear how they fit together.
You can do this by repeating certain key words or names, by chronological
sequencing, or by logical connections. Examine this example for some of
these techniques.
 Example on understanding biographical or historical events.
Direction: Number the paragraphs in the correct order to make a reading
passage about Amelia Earhart's life. Paragraphs 1, 5, and 7 are done for you.
7 . After these successes, Earhart decided to try to fly around the world. She started
out from Florida in June, 1927. As she flew eastward, she stopped to rest and refuel.
At each stop, she wrote about her experiences in her diary and sent her writings to
her husband. On July 2, her plane disappeared over the Pacific Ocean. Searchers
never found her or her plane.
Earhart left university to serve as a nurse in World War 1. When the war ended,
Earhart took her first airplane ride in 1920. She loved flying and started taking
lessons immediately. Six months later, she bought her own small airplane, named
The Canary because it was bright yellow.
During the 1920s, Earhart practiced flying The Canary. Women pilots were
unusual then, but Charles Lindbergh became very famous by becoming the first
pilot to fly across the Atlantic in 1927.
After his wife disappeared, Putnam published her diary entries, but no one has
ever solved the mystery of what happened to Amelia Earhar.

10
 1 The aviator Amelia Earhart's story is one of success,
achievement, and mystery because no one knows how it ended.
The year after her marriage, Earhart became the first
woman to fly alone across the Atlantic. She won awards from President
Hoover and Congress for her achievement. Later she set other records such
as the first woman to fly across the United States alone and the first pilot to
fly from Hawaii to California.
Earhart was born in Atchison, Kansas in 1897. A
good student, she was educated.at Columbia University and also attended
HaNard.
5 In 1928, Amelia Earhart became the first woman passenger to
make the trans-Atlantic flight by air. Her trip was organized by George
Putnam, a New York publisher. He and Amelia fell in love and married in
1931.
 Example on following steps in instructions or a recipe.
Here, according to Al- Juboury (2014) the reading skill
being tested is giving close attention to and follow step by
step detail.
Direction: Arrange the following sentences to make a meaningful text.
Making Tea
1. Leave for five minutes on fire.
2. Boil some water.
3. Serve with sugar.
4. Add some tea to the boiling water

Using language cues to help in comprehension is a reading skill that can be tested
using arrangement technique(ibid). Look at the example below:
Note: The example is taken from the same reference
Direction: The following is a newspaper article. Put the letters in the right order, and
make use of the words and phrases in bold type.

11
 Ship Accident in Channel

No Casualties
a) In spite of the damage the two ships managed to reach
the nearby port of Dunkirk.
b) However, because of the weather conditions, the captains
did not realize the danger until only seconds before the
collision took place.
c) One was a cargo boat carrying fruit, and apparently bound
for London.
d) The official added that the British experts would be asked
to join the committee.
e) Fortunately, there were no causalities among the crews or
the passengers, but both ships were holed close to the
water-line.
f) According to eyewitnesses neither of the ships was going
very fast at the time of the accident.
g) Yesterday evening two ships collided in thick fog in the
English Channel.
h) Consequently, there was insufficient time for them to take
avoiding action.
i) The other was a British Rail cross-channel ferry on its
5. Labelling:
Labelling regular
is usedrun
to from Dover to Dunkirk.
test student's ability to transfer information from a text to a
diagramj) A spokesman for the port authorities said that a
committee
Direction: Read of enquiry
the text and labelwould be set upwith
the diagram to ascertain
the namestheof the planets taken
cause
from the text of the collision.
below.

12
 The Sun is a star at the center of our solar system. Nine planets move round
the sun. Mercury is the planet nearest to the sun. Venus is the second planet. The
Earth is the third planet from the sun. It is the planet that we live on. Mars is the
fourth planet. It is called the red planet. Jupiter, the fifth planet from the sun, is
the largest planet in the solar system. Saturn is the sixth planet from the sun.
Uranus is the seventh planet and Neptune is the eighth planet from the sun. Pluto
is the furthest from the sun. It is the smallest planet(ibid).

6. Cloze test

One of the most popular types of reading assessment task is the cloze
procedure. The word cloze was coined by educational psychologists to capture
the Gestalt psychological concept of "closure," that is, the ability to fill in gaps
in an incomplete image (visual, auditory, or cognitive) and supply (from
background schemata) omitted details( Brown, 2004). According to Al-Juboury
(2014), this type of multiple-choice cloze test is after testing students' ability to
understand relations within the sentence (reading comprehension skill).
Direction: Encircle the right word from the three words given in brackets.
14 th October
Dear Carlos,
Hi! My name is Jason and I'm your new (brother /penfriend
/sister). I'm from Cambridge in (England I Australia I Egypt). I'm thirteen
years old (but I and I or) I'm a student. I'm tall and I've got short (brown I
blue I green) hair and blue eyes. I have got three brothers and (one/
two/three) sister. (I I Me I My) brothers are students but my sister is an (
doctor/ engineer /teacher ).I love football. My favourite (colour I poet I
team) is Manchester United. It's (fantastic I bad I boring). My friends and (I I
my I they) like computer games too. We use the computers in a(cafe I
mosque I hospital) near our school. What sports do you like? What's your
favourite football team? Please write and (say I talk I tell) me.(Hello I Bye I
Good morning)!
Jason

DESIGNING READING COMPREHENSION TESTS

According to Al-Juboury(2014)When designing a reading test, it is important that
the teacher should decide which information is worth asking about. Second he has to
assess a number of the aforementioned reading abilities not just one of them for example
scanning only. Then, he has to choose the technique that best suits the text.
Consider the following text being tested by different techniques

13
 MCQ
Note This reading passage is taken from Al-Juboury (2014)
Read the following passage then write the number of the item and the letter of the correct
answer:

A mystery object landed on Gibbet Hill near Croydon yesterday. The object is saucer-
shaped, about forty feet in diameter and silver-grey in colour. The local police acting
on a report from a local resident were quickly on the scene. They sealed off the area from
the public who gathered to look at the object. Scientists from the Ministry of Defense
arrived few hours later to examine the flying saucer. Rumours, that the object is a
Russian spaceship which went off course have not been confirmed or denied. A
Ministry spokesman said: There is no cause for alarm. The situation is under control.

A mystery object landed on Gibbet Hill near Croydon yesterday. The object is saucer-
shaped, about forty feet in diameter and silver-grey in colour. The local police acting
on a report from a local resident were quickly on the scene. They sealed off the area from
the public who gathered to look at the object. Scientists from the Ministry of Defense
arrived few hours later to examine the flying saucer. Rumours, that the object is a
Russian spaceship which went off course have not been confirmed or denied. A
Ministry spokesman said: There is no cause for alarm. The situation is under control.

1. Which of the following titles is the most adequate one?

A. A flying Saucer on Land B. Unknown Objects Land
C. Russian Spaceship in Croydon D. Silver -grey objects
2. The text is taken from a ........
A book B. film C. newspaper D. student copybook
3. People went to Gibbet Hill to ........
A. examine the object B. look at the object
C. tell the police about it D. know if it was Russian
4. The area was sealed off to ......
A. confirm the rumour B. study the situation
C. fly the object D. keep people away from it
5. Who saw the object first? .....
A. The police 'B. A local resident
C. The scientists D. The Ministry of Defense

6. Rumors that the spaceship is Russian have ......

A. been confirmed
B. been denied
C. neither been confirmed nor denied
D. not been denied by the Ministry of Defense

14
7. One of the following statements is true.
A. The flying saucer landed two days ago.
B. The Ministry of Defense examined the object.
C. The object weighed 40 kg.
D. It was announced that they had controlled the situation.
2) MATCHING ITEMS
Read the previous passage again. Then match the numbers in the first list with the
letters in the second.
First list Second list

1. A local resident a) is under control.

2. Some scientists b) sealed off the area from the public

3. Rumours c) should be told about it

4. The, object d) is alarming

5. The police e) were examining the object.

6. The situation f) told the police about it

g) is forty feet long

h) said that it is Russian

3) SHORT ANSWER QUESTIONS

Answer the following questions according to the passage.
1. What is the best title for this text?
2. Where did the flying saucer land?
3. What did the person who saw it first do?
4. Was the spaceship American or Russian?
5. Why did the people gather around it?
6. Who examined the spaceship?
7. Why did the spokesman talk to people?
8. What does the word 'mystery' means?
9. What is meant by 'the situation is under control'?
10. What does the word 'object' refer to?

15
 Assessing basic reading skills
At the most basic levels of second language literacy, it may be important to
focus on decoding skills and the ability to recognize letters and words.
Assessment of early literacy is best accomplished with the simplest formats that
involve minimal instructions to read, ideally using picture cues whenever feasible.
The focus should be on the content, not on the complexities of the task.
Here are some of the techniques that test reading only
1. Word matching:
The student is required to draw a line under the word which the same as the word on
the left
1. Now bow / not/ how/ now/ mow
2. Sheep shop /shape/ sleep/ heap / sheep
Etc.
Here is another example from Brown(2004)

piece peace
book boot

*In the case of very low level learners, the teacher/administrator reads
directions.

2.Picture-cued

a. Picture-cued word identification (Brown & Sahni, 1994, p . 124)

cat clock chair

16
With the same picture, the test-taker might read sentences and then point to the
correct part of the picture:

B.Picture-cued sentence identification

Test-takers hear:

Test-takers see the picture and read each sentence written on a separate card.

Or a true/false procedure might be presented with the same picture cue:

C.Picture-cued true/false sentence identification

1 . The pencils are under the table. F

2. The cat is on the table. T F
3. The picture is over the couch.

Finally, test-takers might see a word or phrase and then be directed to

choose one of four pictures that is being described, thus requiring the test-
taker to transfer from a verbal to a nonverbal mode. In the following item,
test-takers choose the correct letter:
D.Multiple-choice picture-cued word identification

Test-takers read: Rectangle

Test-takers see, and choose the correct item:

A B C D

17
Tsk of this week: Define he following terms that are related to reading comprehension
skill:
1. Skimming
2. Scanning
3. Main idea
4. Supporting details
5. Reading for gist
6. Inference
7. Bottom –up processing and top down processing

The End of Part Two (Testing Reading)

The content of this file is based on different references including

A Language Teacher's Guide to Assessment" by Prof. Najat Al-Juboury, 2014, A
Practical Guide to Assessing English Language Learners by Christine Coombe,
Keith Folse, and Nancy Hubley, 2010, and Language Assessment, Principles and
Classroom Practices by H. Douglas Brown, (2004).

18