Multiple Sclerosis

http://msj.sagepub.com/

The Brief Repeatable Battery of Neuropsychological Tests: normative values allow application in
multiple sclerosis clinical practice
J B Boringa, R HC Lazeron, I EW Reuling, H J Adèr, L EMA Pfennings, J Lindeboom, L MJ de Sonneville, N F Kalkers
and C H Polman
Mult Scler 2001 7: 263
DOI: 10.1177/135245850100700409

The online version of this article can be found at:

http://msj.sagepub.com/content/7/4/263

Published by:

http://www.sagepublications.com

Additional services and information for Multiple Sclerosis can be found at:

Email Alerts: http://msj.sagepub.com/cgi/alerts

Subscriptions: http://msj.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

Citations: http://msj.sagepub.com/content/7/4/263.refs.html

>> Version of Record - Aug 1, 2001

What is This?

Downloaded from msj.sagepub.com at PACE UNIV LIBRARY on December 1, 2013

 Multiple Sclerosis (2001) 7, 263±267
ã 2001 Arnold All rights reserved 1352 ± 4585/01
www.arnoldpublishers.com/journals

The Brief Repeatable Battery of Neuropsychological Tests: normative

values allow application in multiple sclerosis clinical practice
JB Boringa1,5, RHC Lazeron2, IEW Reuling3, HJ AdeÁr4, LEMA Pfennings3, J Lindeboom3,
LMJ de Sonneville1, NF Kalkers1 and CH Polman1
1
Department of Neurology, Academisch Ziekenhuis Vrije Universiteit 1007 MB Amsterdam, The Netherlands;
2
Department of Diagnostic Radiology, Academisch Ziekenhuis Vrije Universiteit 1007 MB Amsterdam, The Netherlands;
3
Department of Medical Psychology, Academisch Ziekenhuis Vrije Universiteit 1007 MB Amsterdam, The Netherlands;
4
Department of Clinical Epidemiology and Biostatistics, Academisch Ziekenhuis Vrije Universiteit 1007 MB Amsterdam,
The Netherlands; 5Department of Neurology, Ziekenhuis Eemland 3800 BM Amersfoort, The Netherlands

The Brief Repeatable Battery of Neuropsychological Tests (BRB-N) is a sensitive measure of cognitive impairment in multiple sclerosis (MS)
patients. It consists of the Selective Reminding Test, the 10/36 Spatial Recall Test, the Symbol Digit Modalities Test, the Paced Auditory Serial
Addition Test and the Word List Generation Test. We administered one of two parallel versions of the test battery to 140 healthy subjects to
produce normative values for both versions. As expected, test scores were in¯uenced by certain variables like age, gender and education. Although
constructed as two equivalent versions, for some tests the two versions showed signi®cant differences in test scores, which could not be explained
by differences in these variables.
Multiple Sclerosis (2001) 7, 263±267
Keywords: neuropsychological testing; Brief Repeatable Battery; normative values; multiple sclerosis

versions (version A and version B) were constructed

Introduction
for all the individual subtests (European Study Group
The Brief Repeatable Battery of Neuropsychological on Interferon beta-1b in secondary progressive MS,
Tests (BRB-N) was developed by the Cognitive unpublished results). Normative values for the test
Function Study Group of the National Multiple battery are not available, which currently limits their
Sclerosis Society as a tool for evaluating cognitive application in MS clinical practice.
functions in patients with multiple sclerosis (MS).1 It The aim of this study was threefold: to determine
has its origin in a comprehensive neuropsychological normative values for the ®ve subtests, to present the
test battery which consists of 23 tests (resulting in 31 effects of age, gender and education, as well as to
test indices), chosen according to the guidelines for evaluate whether the parallel versions indeed are
Neuropsychological Research in MS.2,3 Rao et al. found exchangeable.
that the four most sensitive tests for detecting
cognitive impairment in MS were the Selective
Reminding Test, the 7/24 Spatial Recall Test, the
Materials and methods
Controlled Oral Word Association Test and the Paced Subjects
Auditory Serial Addition Test.2 With these tests a A total of 140 healthy persons (78 women, 62 men)
sensitivity of 71% and a speci®city of 94% was found recruited from the community volunteered to partici-
in discriminating cognitively intact from impaired MS pate in the study and gave informed consent.
patients as de®ned by the comprehensive battery. Exclusion criteria were: disease of the central nervous
Modi®cation of some of the tests and extension of system, major psychiatric illness, history of alcohol or
the test battery resulted in the BRB-N which consists drug abuse, serious head injury, learning disability and
of ®ve tests: the Selective Reminding Test (SRT), the recent heart attack or other major medical illness. All
10/36 Spatial Recall Test (SPART), the Symbol Digit persons had adequate vision to complete testing.
Modalities Test (SDMT), the Paced Auditory Serial Education level was categorised in three groups based
Addition Test (PASAT) and the Word List Generation on years of education: low is less than 9 years, middle
test (WLG).1,4 In an attempt to reduce training effects is 9 or 10 years and high is more than 10 years of
with longitudinal usage of the battery, two parallel education.

Neuropsychological test procedures

*Correspondence: CH Polman, Department of Neurology, Two test versions were administered in a standardized
Academisch Ziekenhuis Vrije Universiteit, PO Box 7057, way in a ®xed order under the same circumstances,
1007 MB Amsterdam, The Netherlands e.g. administration during the day in a quiet room.
Received 20 January 2001; revised 21 March 2001; accepted Three persons who administered the tests were trained
30 April 2001 to perform the tests according to a ®xed procedure.

Downloaded from msj.sagepub.com at PACE UNIV LIBRARY on December 1, 2013

 Brief Repeatable Battery: normative values in MS
JB Boringa et al
264
The test battery was administered in the following The Paced Auditory Serial Addition Test (PASAT) is
order: SRT, SPART, SDMT, PASAT, Delayed Recall of a measure of sustained attention and information
the SRT, Delayed Recall of the SPART and the WLG. processing speed by adding pairs of digits presented
Administration of the total test battery took 25 ± at two rates of speed.8 The subject is instructed to add
30 min. 60 pairs of digits such that each number is added to
the one that immediately precedes it and report the
The Selective Reminding Test (SRT) is a test to outcome verbally. The digits are presented by tape,
measure verbal learning and memory during a list ®rst with a speed of every 3 s one digit, the second
learning task of six trials.5 The list consists of 12 words trial with every 2 s one digit. The subject is required to
which the examiner reads at a rate of one word per respond verbally quickly, inhibit encoding of his own
two seconds. The subject is instructed to recall all 12 response while attending to the next stimulus in a
words. Every consecutive trial only the words that are series, and perform at an externally determined pace.
missed on the preceding trial are given. After 15 min The score is the number of correct responses per trial
(following the administration of the PASAT) the (PASAT3, PASAT2). The two versions differ only in
subject is asked to recall the word list. sequence of presentation of the digits.
The SRT distinguishes between short-term and long-
term components of memory and examines also the The Word List Generation (WLG) is a semantic verbal
consistency of retrieval from long-term memory. The ¯uency test evaluating the spontaneous production of
scoring is according to published rules.5 Three SRT names of a given category within a limited amount of
indices were used in our study. A word recalled on time.1 The subject is asked to give as many names of
two consecutive trials is considered to have entered `vegetables and fruits' (version A) or `animals' (version
long-term storage (LTS) on the ®rst of these trials and B) as possible during 90 s. The score is the number of
scored as LTS on all following trials regardless of correct names (WLGT).
subsequent recall. The total sum of the words in LTS
of all six trials is taken (SRTL). If a word in LTS is Statistical analyses
consistently recalled on all subsequent trials then it is For all indices, univariate Analysis of Variance
scored as in Consistent Long Term Retrieval (CLTR). (ANOVA) was used to analyse the in¯uence of age,
The total sum of the words in CLTR of all six trials is gender and education, as well as to analyse version
taken (SRTC). The Delayed Recall (SRTD) is the total differences. Associations between indices were as-
number of words recalled after the delayed period. sessed using Pearson's correlation coef®cients. Calcu-
The two versions differ in the list of 12 words, and lations were performed using the microcomputer
were made equivalent on the basis of norms for word version of the Statistical Package for the Social
frequency. Sciences (SPSS).9 In this exploratory study alpha=0.05
was used as a signi®cance level.
The 10/36 Spatial Recall Test (SPART) was devel-
oped to assess visuospatial learning and delayed
recall. The original test, the 7/24 Spatial Recall Test,
Results
consists of a 466 checkerboard with seven checkers The 140 subjects who performed the test battery had a
randomly placed.6 In the adapted version the checker- mean age of 45.8 (range 22 ± 73 years); 31 persons had
board is wider (666) and more checkers (10) are used. a low education (16 women, 15 men), 55 persons were
The board is put in front of the subject for 10 s. After in the middle group (28 women, 27 men) and 53
presentation the subjects attempt to reproduce the persons had a high education (34 women, 19 men).
original design on an empty board. This process is One person refused to state his education. The test
repeated twice and after 15 min (following the SRTD) battery was completed by all subjects, except that ®ve
the subject is asked to recall the design again. The refused to perform the PASAT after the practice trial
score is the total number of correct responses for the and one refused the 2 s PASAT after performing the
three trials (SPARTT) and the delayed recall trial 3 s PASAT. Version A was administered to 67 persons
(SPARTD). The versions differ in the con®guration of (34 women, 33 men), version B to 73 persons (44
the 10 checkers. women, 29 men). The neuropsychological test results
for the whole group, and for each version, are shown
The Symbol Digit Modalities Test (SDMT) examines in Table 1. There were no signi®cant differences
sustained attention and concentration by primarily between those who completed version A versus those
assessing complex visual scanning and tracking.7 The who completed version B with regard to age, gender or
subject examines a series of nine meaningless geo- education. Table 2 shows that the correlation of
metric symbols which are labelled 1 to 9. During 90 s indices of the same subtests were high (0.66 to 0.83),
the subject substitutes symbols in a row by the whereas correlations between indices of different tests
corresponding number and responds verbally. The were low to moderate (0.04 to 0.47). We analysed the
score is the number of correct substitutions (SDMTT). in¯uence of age, gender, education and version for the
For each version the geometric symbols are the same 9 indices as shown in Table 3.
but placed in a different sequence, so labelled with a Age had a signi®cant in¯uence on all indices except
different number. on the PASAT3 and the WLGT. With an increase of age
Multiple Sclerosis

Downloaded from msj.sagepub.com at PACE UNIV LIBRARY on December 1, 2013

 Brief Repeatable Battery: normative values in MS
JB Boringa et al
265
Table 1 BRB-N results of all subjects and by version applied Table 3 P values for indices of BRB-N indicating which
variables have signi®cant impact
Total Version A Version B
Test Min. Max. Mean SD Mean SD Mean SD Test Age Gender Education Version

SRT L 16 70 48.9 11.8 47.6 11.2 50.1 12.3 SRT L 0.037 0.001 0.131 0.527
C 6 70 37.1 14.2 34.7 13.1 39.3 14.8 C 0.024 0.001 0.283 0.148
D 2 12 9.2 2.2 8.9 2.3 9.5 2.1 D 0.001 <0.001 0.051 0.321
SPART T 10 30 22.4 4.5 20.9 4.4 23.9 4.1 SPART T <0.001 0.189 0.256 <0.001
D 3 10 8.0 2.0 7.4 2.1 8.5 1.7 D <0.001 0.373 0.993 0.003
SDMT T 27 106 56.1 12.4 52.3 11.0 59.5 12.8 SDMT T <0.001 0.007 0.004 0.001
PASAT 3 13 60 48.7 10.7 47.3 11.9 50.0 9.4 PASAT 3 0.060 0.592 0.043 0.191
2 13 59 38.2 11.0 36.8 11.6 39.6 10.2 2 0.014 0.357 0.014 0.083
WLG T 8 52 30.7 7.9 26.3 6.1 34.7 7.2 WLG T 0.769 0.002 0.020 <0.001

Table 2 Pearson's correlation coef®cients between indices of BRB-N

STRL SRTC SRTD SPARTT SPARTD SDMTT PASAT3 PASAT2

SRTC 0.83 ± ± ± ± ± ± ±
SRTD 0.74 0.75 ± ± ± ± ± ±
SPARTT 0.21 0.18 0.14 ± ± ± ± ±
SPARTD 0.21 0.23 0.22 0.66 ± ± ± ±
SDMTT 0.41 0.44 0.36 0.43 0.34 ± ± ±
PASAT3 0.11 0.15 0.10 0.35 0.29 0.46 ± ±
PASAT2 0.10 0.18 0.04 0.33 0.28 0.45 0.68 ±
WLGT 0.30 0.32 0.28 0.30 0.25 0.47 0.24 0.29

there was a decrease in mean score for all indices. high correlations between indices of the same subtests
Gender in¯uenced the three indices of the SRT, the suggest that these measures are assessing similar
SDMTT and the WLGT. Women performed better on all constructs. In order to construct a brief instrument,
of these indices than men. Education in¯uenced the which would be more useful for MS patients, some of
SDMTT, the two indices of the PASAT and the WLGT. the original tests were modi®ed. The original version
High education had signi®cantly higher mean scores for of the SRT employs twelve learning trials, but in the
all of these indices than middle and low education. The BRB-N this is shortened to six in order to reduce
two indices of the SPART, the SDMTT and the WLGT administration time and patient fatigue. It has been
were in¯uenced by the version used. Version B had shown that the results are highly consistent with the
signi®cantly higher mean scores for all of these indices. 12-trial version.15 The SPART originally used seven
checkers with a 466 checkerboard and contained ®ve
trials. Major disadvantages of this test were that its
Discussion dif®culty level was too low and the patterns could be
The BRB-N was developed as a brief screening battery easily verbalized, detracting from the purpose of
for cognitive impairment in MS patients. It is measuring visuospatial memory.16 The 10/36 version
increasingly used as a screening instrument to detect obviates these problems and is shorter to administer
patients who are in need of more comprehensive because three instead of ®ve trials are used.
neuropsychological testing and also as a research tool, For a screening test battery it is important to be very
for example in actual history studies, to assess the sensitive. The previous version of the BRB-N with four
relation between cognitive dysfunction and disease of the ®ve tests had a sensitivity of 71% for detecting
burden on MRI, and in double blind clinical trials of cognitive impairment in MS patients.2 Probably this
potential treatments for MS.10 ± 14 It is constructed to will be even higher with the use of the SDMT; several
test different domains of cognition which are fre- studies consistently showed that MS patients perform
quently impaired in MS, like verbal memory and signi®cantly worse than controls on this test.17,18 Some
delayed recall (SRT), visuospatial memory and de- found that the SDMT was the most sensitive measure
layed recall (SPART), sustained attention and concen- of cerebral integrity and that the oral version was,
tration (SDMT and PASAT) and semantic retrieval compared with the PASAT and the Stroop, the single
(WLG). The low to moderate correlations between the best measure of reduced speed of processing in brain-
®ve subtests as found in this study indicate that injured patients.7,19 For some of the subtests norms are
indeed different aspects are being investigated. The not available (as in the case of the SPART) or not

Multiple Sclerosis

Downloaded from msj.sagepub.com at PACE UNIV LIBRARY on December 1, 2013

 Brief Repeatable Battery: normative values in MS
JB Boringa et al
266
applicable because the administration was modi®ed. the subjects to whom the two versions were adminis-
For instance Smith's norms of the SDMT are based on tered. To our knowledge no studies of the equivalence
the oral version given shortly after the written form, of different categories for semantic verbal ¯uency are
which later is omitted in use of this test battery.7 In available. The difference of the versions of the WLG is
addition, the administration of a test within the setting probably quite easy to explain, because it is obvious
of a test battery can in¯uence the performance that there are more animals (version B) than fruit and
especially for the delayed recall indices. vegetables (version A). The two versions of the SDMT
Normative data of the BRB-N have recently been consist of the same symbols but in a different order.
published, though based on a smaller sample.12 Others also noted discrepancies between two versions
Demographic characteristics are not speci®cally given when they were examined initially, but they are of
because this group was matched with a subgroup of clinical and research utility when used with the
MS patients from a larger cohort, and only this cohort appropriate normative values.20 A good explanation
was described in detail. Therefore these normative for the difference between the two versions of the
data cannot be easily applied to other MS populations. SPART is hard to give. Because of major modi®cations
Besides, these authors did not formally investigate the comparison with data from the literature is not
relation between test scores and version, age, gender possible.
and education, factors that in our study turned out to Our ®nding of signi®cant differences between the
be of paramount importance. versions has major consequences. When used in
For all subtests except the PASAT3 and the WLG, longitudinal testing, for example in clinical trials to
age was an important factor that in¯uenced the test test medication effects on cognitive function, the
score, even for ages under 50 years. This is in appropriate normative values must be used to interpret
accordance with the literature for the SRT and the the test scores in the right way. For instance when
SDMT.7,21 Some found that age did in¯uence perfor- version A of the WLG is used ®rst and later the test
mance on the PASAT, whereas others did not.22,23 We score of the B version is better, this change can be
found that age did not modify the results on the explained by differences in normative values. Also in
PASAT3, but did on the PASAT2. This probably can natural history studies of cognitive function the results
be explained by the PASAT2 being more dif®cult than must be interpreted with the knowledge that the two
the PASAT3 and therefore more prone to the in¯uence versions are not equivalent. In conclusion, we have
of age. For the category naming of the WLG the data shown, based on test results in 140 healthy subjects,
are also con¯icting because some groups did ®nd age that normative values for the BRB-N are signi®cantly
effects (Tombaugh, Kozak and Rees, pers. comm.) affected by a number of variables, including age,
whereas other studies, like the present, did not.24 gender and education. The two versions, although
Because of the major modi®cations of the SPART constructed with the aim to develop equivalent
normative values are not available in the literature so subtests also, differed in normative values for some
comparison is not possible. of the tests. This should be kept in mind when used in
Gender is another factor which can in¯uence test longitudinal testing in clinical trials or in natural
performance. As also found in other studies gender history studies.
affected performance, with women performing better
than men on the SRT and the WLG.21,24 For the SDMT
it is not clear, while some found no gender differences, Acknowledgements
others, as in our study, reported that women outper- The authors would like to thank Schering AG for
form men.7,25 In accordance with others, our study providing the test batteries.
found no correlation between gender and performance
on the PASAT.23 Education in¯uenced the test scores
on the SDMT, as well as both indices of the PASAT
and WLG, which is in accordance with the litera-
References
ture.7,23,24 As far as we know, no data are available for 1 Rao SM and the Cognitive Function Study Group of the
the in¯uence of language on the test battery. Although National Multiple Sclerosis Society. (1990) A manual for
our normative values are based on test scores in a the Brief Repeatable Battery of Neuropsychological Tests
Dutch population, the limited impact of language on in multiple sclerosis Medical College of Wisconsin:
the BRB-N, except for the SRT, most likely indicates Milwaukee, WI.
that they can also be applied to other versions of the 2 Rao SM, Leo GJ, Bernardin L, Unverzagt F. (1991)
BRB-N which are available in several languages Cognitive dysfunction in multiple sclerosis 1. Fre-
(European Study Group on Interferon beta-1b in quency, patterns and prediction. Neurology 41: 685 ±
secondary progressive MS, unpublished data). 691.
3 Peyser JM, Rao GJ, LaRocca NG, Kaplan E. (1990)
Although the versions constructed should be
Guidelines for neuropsychological research in multiple
equivalent, three (SPART both indices, SDMT and sclerosis. Arch Neurol 47: 685 ± 691.
WLG) of the ®ve tests differed signi®cantly across the 4 Bever CT, Grattan L, Panitch HS, Johnson KP. (1995) The
two studied versions in test score. For all three tests brief repeatable battery of neuropsychological tests for
the B version had higher scores. This could not be multiple sclerosis: a preliminary serial study. Multiple
explained by differences in age, gender or education of Sclerosis 1: 165 ± 169.

Multiple Sclerosis

Downloaded from msj.sagepub.com at PACE UNIV LIBRARY on December 1, 2013

 Brief Repeatable Battery: normative values in MS
JB Boringa et al
267
5 Buschke H, Fuld PA. (1974) Evaluating storage, retention, 16 Lezak M. (1995) Neuropsychological Assessment, third
and retrieval in disordered memory and learning. edition Oxford University Press: New York.
Neurology 24: 1019 ± 1025. 17 Beatty WW, Goodkin DE, Monson N, Beatty PA. (1989)
6 Barbizet J, Cany E. (1968) Clinical and psychometrical Cognitive disturbances in patients with relapsing remit-
study of a patient with memory disturbances. Interna- ting multiple sclerosis. Arch Neurol 46: 1113 ± 1119.
tional Journal of Neurology 7: 44 ± 54. 18 Krupp LB et al. (1994) Cognitive functioning and
7 Smith A. (1991) Symbol digit modalities test. Los depression in patients with chronic fatigue syndrome
Angeles: Western Psychological Services. and multiple sclerosis. Arch Neurol 51: 705 ± 710.
8 Gronwall DM. (1977) Paced auditory serial-addition task: 19 Pondsford J and Kinsella G. (1992) Attentional de®cits
a measure of recovery from concussion. Percept Mot following closed head injury. Journal of Clinical and
Skills 44: 367 ± 373. Experimental Neuropsychology 14: 822 ± 838.
9 Norusis MJ. (1996) SPSS/PC+ statistic 8.5. Chicago: 20 Uchiyama CL et al. (1994) Longitudinal comparison of
SPSSInc. alternate versions of the Symbol Digit Modalities Test:
10 Sperling RA et al. (2001) Regional Magnetic Resonance Issues of form comparability and moderating demo-
Imaging Lesion Burden and Cognitive Function in graphic variables. Clin Neuropsych 8: 209 ± 218.
Multiple Sclerosis. Arch Neurol 58: 115 ± 121. 21 Larabee GJ, Trahan DE, Curtiss G, Levin HS. (1988)
11 Hohol MJ et al. (1997) Serial Neuropsychological Normative data for the verbal selective reminding test.
Assessment and Magnetic Resonance Imaging Analysis Neuropsych 2: 173 ± 182.
in Multiple Sclerosis. Arch Neurol 54: 1018 ± 1025. 22 Brittain JL et al. (1991) Effects of age and IQ on Paced
12 Camp SJ et al. (1999) Cognitive function in primary Auditory Serial Addition Task (PASAT) performance.
progressive and transitional progressive multiple sclero- Clin Neuropsychol 5: 163 ± 175.
sis: A controlled study with MRI correlates. Brain 122: 23 Stuss DT, Stethem LL and Poirier CA. (1987) Comparison
1341 ± 1348. of three tests of attention and rapid information
13 Smits RCF et al. (1994) The effects of 4-aminopyridine on processing across six age groups. Clin Neuropsych 1:
cognitive function in patients with multiple sclerosis: A 139 ± 152.
pilot study. Neurology 44: 1701 ± 1705. 24 Ruff RM, Light RH, Parker SB. (1996) Benton controlled
14 Weinstein A et al. (1999) Neuropsychologic status in word association test: Reliability and updated norms.
multiple sclerosis after treatment with glatiramer. Arch Arch Clin Neuropsych 11: 329 ± 338.
Neurol 56: 319 ± 324. 25 Polubinski JP and Melamed LE. (1986) Examination of
15 Smith RL, Goode KT, la Marche JA, Boll TA. (1995) the sex difference on a symbol digit substitution task.
Selective reminding test short form administration: A Perceptual motor skills 62: 975 ± 982.
comparison of two through twelve trials. Psychol
Assessment 7: 177 ± 182.

Multiple Sclerosis

Downloaded from msj.sagepub.com at PACE UNIV LIBRARY on December 1, 2013