Journal of the International Neuropsychological Society (2006), 12, 549–558.

Copyright © 2006 INS. Published by Cambridge University Press. Printed in the USA.
DOI: 10.10170S1355617706060723

Validity of the minimal assessment of cognitive function

in multiple sclerosis (MACFIMS)

RALPH H.B. BENEDICT,1 DIANE COOKFAIR,2 REBECCA GAVETT,1 MARK GUNTHER,2

FREDERICK MUNSCHAUER,2 NEETA GARG,2 and BIANCA WEINSTOCK-GUTTMAN 1
1 StateUniversity of New York (SUNY) at Buffalo School of Medicine, Department of Neurology,
Division of Departmental and Behavioral Neurosciences, Buffalo, New York, USA
2 Jacobs Neurological Institute, Buffalo General Hospital, Buffalo, New York, USA

(Received January 23, 2006; Final Revision March 23, 2006; Accepted March 23, 2006)

Abstract
Cognitive impairment occurs in roughly 50% of patients with multiple sclerosis (MS). It is well known that
processing speed and episodic memory deficits are the most common neuropsychological (NP) sequelae in this
illness. Consensus has emerged about the specific tests that prove most helpful for routine monitoring of MS
associated cognitive impairment. The purpose of this study was to examine the validity of the Minimal Assessment
of Cognitive Function in MS (MACFIMS), a recommended battery based on the findings of an international
conference held in 2001. We tested 291 MS patients and 56 healthy controls. Frequencies of impairment paralleled
those reported in previous work for both individual cognitive domains and general impairment. All tests were
impaired in the MS group, and distinguished relapsing-remitting (RR) from secondary progressive (SP) course.
Principle components analysis showed a distinct episodic memory component. Most of the MACFIMS tests
discriminated disabled from employed patients. However, in regression models accounting for all NP tests, those
emphasizing verbal memory and executive function were most predictive of vocational status. We conclude that the
MACFIMS is a valid approach to routine NP assessment of MS patients. Future work is planned to determine its
psychometric properties in a longitudinal study. (JINS, 2006, 12, 549–558.)
Keywords: Multiple sclerosis, Processing speed, Memory, Neuropsychological testing, Validity, Employment

INTRODUCTION ing is often costly and time consuming. Combined, these

factors impede the implementation of routine NP testing for
The impact of multiple sclerosis (MS) on neuropsycholog- MS patients.
ical (NP) function is well recognized, in that nearly half of In 2001, we reasoned that routine NP testing of MS
all MS patients show deficiency on NP testing (Heaton, patients might be more feasible if it were conducted in a
1985; Rao et al., 1991a). Research also indicates that cog- more standardized and cost-effective manner. In collabora-
nitive impairment gives rise to vocational disability (Bene-
tion with the Consortium of MS Centers, we convened a
dict et al., 2005a; Rao et al., 1991b), social impairment
panel of neuropsychologists and psychologists with exper-
(Benedict et al., 2001b; Rao et al., 1991b), and distress in
tise in MS from the United States, Canada, United King-
persons caring for MS patients (Knight et al., 1997). Yet dom, and Australia. The objectives of the meeting were to
NP testing is not routinely employed in MS clinics for a (a) propose a minimal NP examination for clinical monitor-
variety of reasons. NP impairment is not a diagnostic crite- ing of MS patients and research and (b) propose strategies
rion for MS diagnosis (McDonald et al., 2001). Board cer-
for improving NP assessment of MS patients. Literature on
tified or well-trained clinicians with MS expertise are not
MS-related cognitive dysfunction was reviewed with an eye
available to many MS Centers. Most importantly, NP test- toward both psychometric factors (reliability, discrimina-
tive validity) and practical considerations (e.g., alternate
forms, brevity of administration). The meeting culmi-
Correspondence and reprint requests to: Ralph Benedict, PhD, Buffalo
General Hospital, D-6, 100 High Street, Buffalo, NY 14203. E-mail: nated in a publication (Benedict et al., 2002) summarizing
[email protected] the panel discussion and a 90-minute NP battery called
549
550 R.H.B. Benedict et al.

the Minimal Assessment of Cognitive Function in MS ature. We then explored the construct validity of these tests
(MACFIMS). Included were seven tests covering the five using principle components analysis. Because episodic
cognitive domains most commonly impaired in MS patients memory was distinguished from processing speed in previ-
(Table 1). Tests emphasizing information processing speed, ous literature reviews (Benedict et al., 2002), we predicted
working memory, and unaided recall of recently presented that such an analysis would show a clearly discernible epi-
information were judged to be most sensitive to the cere- sodic memory component. Finally, we predicted that the
bral pathology of MS (Archibald & Fisk, 2000; Beatty, 1996; MACFIMS tests would show good ecological validity
DeLuca et al., 1998; Demaree et al., 1999; Fischer, 1988; regarding their associations with vocational disability status.
Rao et al., 1989a; Rao et al., 1989b). Reviewed research
also showed that tests in other domains are also, if less
METHODS
frequently, affected in MS patients, especially those involv-
ing generative fluency (Beatty & Monson, 1994), spatial
Participants
processing (Vleugels et al., 2000), and conceptual reason-
ing (Arnett et al., 1997; Beatty & Monson, 1996). While an Studied were 291 patients diagnosed with clinically defi-
encouraging first step, the authors acknowledged that the nite MS (McDonald et al., 2001) by treating or research
MACFIMS battery, merely a collection of recommended neurologists. Patients entered the study for one of three
tests, had yet to be validated as a single battery in a large, reasons: participation in brain imaging research (n 5 77,
diverse sample of MS patients. 27%), routine monitoring of cognitive function (n 5 106,
In the course of preparing this manuscript, we searched 36%), or referral for evaluation of a specified management
the literature for studies that had included any of the indi- problem related to suspected cognitive impairment (n 5
vidual NP tests comprising the MACFIMS battery to eval- 108, 37%). All patients met the following inclusion and
uate MS patients. Included in Table 2 are those studies exclusion criteria: (a) age 18 or older; (b) fluent in English;
comparing MS patients and healthy controls and reporting (c) able to provide informed consent to all procedures; (d)
data that permit calculation of effect size using the well no neurological disorder other than MS; (e) no psychiatric
known Cohen d statistic (Cohen, 1988). Table 2 shows that disorder (American Psychiatric Association, 1994) other than
of 28 between-group comparisons only 2 have effect sizes mood, personality, or behavior change following the onset
less than d 5 .5. Most show medium or large effects. Ten of MS; (f ) no other medical condition that might influence
comparisons show a between-group difference of 1.0 SD or cognition; (g) no history of developmental disorder (e.g.,
greater. Such strong effects are not surprising considering ADHD, learning disability); (h) no history of substance or
that the MACFIMS tests were chosen based largely on evi- alcohol dependence, or current abuse; (i) no motor or sen-
dence for reliability and discriminative validity. sory defect (e.g., corrected near vision of at least 20070)
Here, for the first time, we examined the concurrent and that might interfere with cognitive test performance; and
construct validity of the entire MACFIMS battery in a large ( j) no relapse and0or corticosteroid pulse within four weeks
sample of 291 prospectively accrued MS patients. We pre- of assessment. All research participants signed informed
dicted a priori that each of the individual tests that com- consent forms approved by institutional review panels prior
prise the MACFIMS battery would discriminate MS patients to participating in the study.
from healthy controls, and that effect sizes and frequencies Mean patient age (6SD) was 45.4 6 8.9 years. On aver-
of impairment would parallel previous findings in the liter- age, patients completed 14.4 6 2.4 years of education. The

Table 1. Tests included in the Minimal Assessment of Cognitive Function in MS (MACFIMS) battery

Test Domain Authors Most common measures

Controlled Oral Word Association Test Language (Benton et al., 1994) Total # Correct
Judgment of Line Orientation Test Spatial processing (Benton et al., 1994) Total # Correct
California Verbal Learning Test, Second Edition New learning and (Delis et al., 2000) Total Learning Trials 1–5
memory Delayed Recall
Brief Visuospatial Memory Test—Revised New learning and (Benedict et al., 1996; Total Learning Trials 1–3
memory Benedict, 1997) Delayed Recall
Symbol Digit Modalities Test Processing speed and (Smith, 1982; Total # Correct
working memory Rao et al., 1991b;
Rao, 1991)
Paced Auditory Serial Addition Test Processing speed and (Gronwall, 1977; 3.0 ISI and 2.0 ISI,
working memory Rao et al., 1991b; # Correct
Rao, 1991)
Delis-Kaplan Executive Function System Sorting Test Executive function (Delis et al., 2001) # Correct Sorts
Description Score
Minimal assessment of cognitive function in MS 551

Table 2. Effect sizes derived from previously published studies using the MACFIMS battery tests to discriminate MS patients
from healthy controls

Test NC NC MS MS
Study MS Sample variable mean SD Mean SD d P
(Archibald & Fisk, 2000); (Fisk & Archibald, 2001) 35 mixed course CVLT-DR 13.3 2.1 11.9 2.6 0.6 nr
(Beatty & Monson, 1996) 30 mixed course CCST-CS 17.8 2.4 14.6 3.8 1.0 .001
CCST-DS 31.9 4.8 26.6 7.8 0.8 .01
COWAT 41.5 9.9 32.3 14.0 0.8 0.01
(Beatty & Monson, 1994) 30 mixed course COWAT 41.5 9.9 32.3 14.0 0.8 0.01
(Beatty et al., 1995c; Beatty et al., 1995b) 100 mixed course COWAT 47.7 14.2 32.1 13.3 1.1 0.001
JLO 25.3 3.7 23.7 5.2 0.4 ns
SDMT 59.0 10.2 43.7 15.0 1.2 0.001
DKEFS-CS 6.2 1.0 5.0 1.6 0.9 0.001
DKEFS-RS 0.2 0.5 0.6 0.8 0.6 0.01
(Beatty et al., 1989) 42 mixed course SDMT 59.0 9.1 48.3 11.4 1.0 0.001
COWAT 42.8 12.7 34.5 11.9 0.7 0.05
(Benedict et al., 2001a) 34 mostly SP JLO 24.1 4.7 17.7 8.6 1.0 0.01
CVLT-TL 57.2 17.2 41.3 15.1 1.0 0.01
BVMT-R-TL 21.1 5.5 14.1 7.9 1.1 0.01
(D’Esposito et al., 1996) 36 mostly RR COWAT 48.7 15.7 42.2 10.9 0.5 ns
SDMT 63.1 7.6 53.1 12.4 1.0 .01
(Krupp et al., 1994) 20 mixed course COWAT 48.0 8.3 39.8 13.6 0.8 0.04
(Rao et al., 1991b) 100 mixed course COWAT 43.8 11.8 34.7 12.1 0.8 ,.001
JLO 27.2 4.1 25.0 4.8 0.5 ,0.01
PASAT3 48.5 9.6 41.6 11.5 0.7 ,.001
PASAT2 37.1 10.1 30.8 10.5 0.6 ,.001
(Ryan et al., 1996) 177 RR COWAT 41.7 10.3 37.0 11.7 0.4 ,.01
(Scarrabelotti & Carroll, 1998) 50 mixed course CVLT-TL 54.0 14.6 46.2 15.7 0.5 0.02
(Sperling et al., 2001) 28 mixed course COWAT 34.8 9.8 26.6 6.6 1.0 ,0.01
PASAT3 PC 87.8 13.8 67.9 22.8 1.1 ,0.01
(Thornton et al., 2002) 49 mixed course CVLT-TL 58.7 9.5 50.9 11.9 0.7 ,.001
CVLT-DR 12.4 2.8 10.6 3.7 0.6 ,.01

Note. Studies included report mean and SD for MS and control group, allowing calculation of effect size d. Only test forms included in the MACFIMS
listed. For example, studies using the written form of the Symbol Digit Modalities Test or the Gronwall version of the PASAT are not included.
Abbreviations: nr 5 not reported. ns 5 not significant. MS 5 multiple sclerosis, NC 5 normal control, RR 5 relapsing-remitting course, SP 5 secondary
progressive course after relapsng-remitting course, COWAT 5 Controlled Oral Word Association Test, JLO 5 Judgment of Line Orientation Test, CVLT 5
California Verbal Learning Test, CVLT-TL 5 CVLT Total Learning, CVLT-DR 5 CVLT Delayed Recall, BVMTR 5 Brief Visuospatial Memory
Test–Revised, BVMTR-TL 5 BVMTR Total Learning, PASAT 5 Paced Auditory Serial Addition Test, PASAT3 5 3.0 inter-stimulus interval of PASAT,
PASAT2 5 2.0 inter-stimulus interval of PASAT, PC 5 Percent Correct, SDMT 5 Symbol Digit Modalities Test, CCST 5 California Card Sorting Test, and
an early version of the DKEFS 5 Delis Kaplan Executive Function System Sorting Test, DKEFS-CS 5 DKEFS Correct Sorts, DKEFS-DS 5 DKEFS
Description Score.

majority were Caucasian (n 5 277 or 95%) and women MATERIALS AND METHODS
(n 5 227 or 78%), consistent with the MS population which
is primarily female (Jacobs et al., 1999). Scores derived Each patient underwent NP testing in accordance with recent
from the Expanded Disability Status Scale (Kurtzke, 1983), consensus panel recommendations (Benedict et al., 2002).
which assess neurologic and physical disability, were avail- The MACFIMS includes the following tests: Controlled Oral
able for 186 patients; the mean EDSS for this subset of Word Association Test (COWAT; Benton et al., 1994), Judg-
patients was 3.0 6 1.8, representing mild and moderate ment of Line Orientation Test (JLO; Benton et al., 1994),
impairment. Most patients (n 5 200 or 69%) had relapsing- California Verbal Learning Test, second edition (CVLT-II;
remitting (RR) rather than secondary-progressive (SP) (n 5 Delis et al., 2000), Brief Visuospatial Memory Test-Revised
78 or 27%), progressive-relapsing (n 5 6 or 2%), or primary- (BVMT-R; Benedict, 1997), Paced Auditory Serial Addi-
progressive (n 5 7 or 2%) course. tion Test (PASAT; Gronwall, 1977), Symbol Digit Modal-
Patients were compared with 56 healthy controls with a ities Test (SDMT; Smith, 1982), and the Sorting Test from
mean age of 43.8 6 9.5 years and 14.6 6 2.2 years of the Delis-Kaplan Executive Function System (DKEFS; Delis
education. The majority of the controls were Caucasian et al., 2001). In accordance with the position paper (Bene-
(n 5 50, or 89%) and women (n 5 42 or 75%). The differ- dict et al., 2002), we employed Rao’s adaptations (Rao,
ences between the MS and control groups on these demo- 1991) of the PASAT and SDMT.
graphic variables did not reach significance by ANOVA or The COWAT was administered in the standard manner,
x 2 test. following the method of Arthur Benton (Benton et al., 1994).
552 R.H.B. Benedict et al.

In successive one-minute trials, participants generated as negative self-evaluation or guilt) and mood (e.g., dyspho-
many words as possible, beginning with each of three des- ria) and avoids assessment of vegetative signs that can occur
ignated letters. The dependent measure was the total num- in medical illness without depression. This test was recently
ber of correct words over the three trials. validated in a MS sample (Benedict et al., 2003).
The JLO required participants to identify the angle defined Finally, we assessed vocational status by asking patients
by two stimulus lines from among those defined by a visual about their present work activities (Benedict et al., 2005a).
array of lines covering 180 degrees. Both oral and pointing There were two methods used to classify patients as either
responses were allowed. The dependent variable was the employed or disabled. A conservative approach to classifi-
total number of correct responses over 30 items. cation required that working patients be gainfully employed
The CVLT-II and BVMT-R are both learning and mem- with pay, full time, without demotion, reported reprimands,
ory tests with similar formats. Both require discrete expo- or loss of pay due to MS related problems. Disabled patients
sures to new material that are followed by the participant’s were required to be receiving formal disability benefits from
unaided recall immediately after presentation. There is a either public (e.g., social security disability income) or pri-
25-min interval following the final learning trial, after which vate (e.g., commercial benefits) sources. The liberal approach
participants are asked to recall the information again with- to classification required that working patients be gainfully
out further exposure to the to-be-learned material. Delayed employed with pay at least 20 hours per week. Disabled
recall is followed by a yes or no, forced-choice recognition patients for this scheme were required to be receiving either
task. For the CVLT-II there were five learning trials. Exam- formal disability as described above, or to be unemployed
iners read 16 words and asked participants to repeat as for reasons reported by them or informants to be disease
many words as possible. The entire word list was repeated related.
each time. For the BVMT-R, the stimulus material was a
matrix of six visual designs, held before the participant for
10 sec. Participants were asked to render the designs using Procedure
paper and pencil, taking as much time as needed for repro-
The research participants (including all HCs) were con-
duction. Each design received a score of 0, 1, or 2 based on
tacted by mail or were approached during the course of
accuracy and location scoring criteria. There were three
their usual clinical care at an MS center. The clinical par-
free-recall trials followed by 25-min delayed recall and yes
ticipants were patients undergoing evaluation either for rou-
or no recognition trials. In this study, we considered the
tine monitoring of cognitive functioning, or for specific
following measures for each memory test: total recall over
clinical reasons (e.g., concerns about driving or work capac-
all learning trials (Total Learning or TL) and recall after the
ity, disability evaluation, differential diagnosis of cognitive
delay interval (Delayed Recall or DR, i.e., long delayed
impairment vs. depressive disorder, etc). The tests were
recall in the case of the CVLT-II).
administered individually by a psychologist, trained assis-
The PASAT included 60 trials presented at inter-stimulus
tant, or a graduate student under the supervision of a board-
intervals of 3 (PASAT-3) and 2 (PASAT-2) sec. The depen-
certified (ABPP-CN) neuropsychologist. The entire test
dent measure was the number of correct responses from
battery, including tests of personality, required 90–120 min
each of the two trials. For the SDMT, participants were
of face-to-face testing time.
presented with a series of nine symbols, each paired with a
single digit in a key at the top of an 8.5 3 11-inch sheet.
The remainder of the page presented a pseudo-randomized Statistical Analysis
sequence of symbols. Participants responded by voicing
the digit associated with each symbol as quickly as possi- All statistical analyses were performed using SPSS 13.0.
ble. The dependent measure was the number of correct Group differences in continuous variables were assessed
responses in 90 sec. using unpaired Student’s t-test or univariate ANOVA with
The DKEFS Sorting Test was employed to assess alpha set at p , .01. The chi-square test was used to deter-
higher executive function. Participants were presented mine differences in categorical variables. We considered
with six cards each depicting a single word. The cards limiting the study to RR and SP patients, because there
varied in many ways, allowing conceptual sorting in accor- were few patients with progressive-relapsing and primary-
dance with at least eight different principles (e.g., card progressive course. However, one of the goals of the study
shape, card color, semantic association among words). This was to test the validity of the MACFIMS in a sample that
was a timed test with 4 min allowed for each of two card represents all MS patients receiving clinical treatment. There-
sets. We recorded the total number of correct sorts (CS) fore, normal versus MS comparisons included these rarer
and calculated the verbal description score (DS), which disease courses.
was based on the abstractness and accuracy of the sort For descriptive purposes, we examined effect sizes using
descriptions. Cohen’s d statistic, which is the difference between means
Depression was assessed using the Beck Depression divided by the pooled SD. Z scores were calculated for each
Inventory—Fast Screen for Medical Patients (BDI-FS; Beck individual NP test based on the normal control group. Impair-
et al., 2000). The BDI-FS emphasizes thought content (e.g., ment for a single test was defined as a z score , 21.5. NP
Minimal assessment of cognitive function in MS 553

impairment generally was defined as a defect on two or final stepwise model, specifically, the measure that showed
more test measures. the highest frequency of impairment. Vocational status
Principal Components Analysis (PCA) with varimax (disabled0employed) served as the dependent variable for
rotation was used to assess the construct validity of the each model. Two sets of models were run using two differ-
MACFIMS battery. Because mean NP test scores and levels ent definitions of vocational status (conservative and lib-
of impairment were found to differ by course for all NP eral, see Methods). In preliminary analyses we noted that
tests in the MACFIMS battery, separate PCAs were con- only 13% of SP patients were employed, leaving only 6
ducted for RR and SP patients to evaluate whether the under- employed SP patients for analysis. Therefore, we also ran a
lying components (dimensions) in the MACFIMS battery separate set of models for RR patients only, using the same
vary according to disease course. All 11 test variables were criteria and following the same steps outlined earlier.
included in both models. Components were identified using
a criteria of Eigenvalue .1.0. Each component was required
to consist of at least two variables with loadings of .5 or RESULTS
more as well as explain at least 15% of the variance in the
model. The NP variables most representative of a specific Group Differences and Frequency
component were considered to be those with loadings ⭌ of Impairment
.500 for that variable.
Estimates of the risk of being disabled or unemployed Table 3 shows the group data for the MACFIMS tests.
associated with NP impairment were obtained from odds ANOVA showed significant between-group effects for all
ratios calculated using unconditional logistic regression mod- tests with effect sizes ranging from 0.49 for JLO to 1.31 for
els. Like the PCA, this analysis did not include the 13 SDMT. Frequencies of impairment in MS patients (Fig-
progressive-relapsing and primary-progressive patients. One ure 1) were as follows: COWAT 13.1%, JLO 22.3%, CVLT-
of the criteria for inclusion in final stepwise regression II-TL 29.6%, CVLT-II-DR 34.4%, BVMT-R-TL 54.3%,
testing was that NP variables significantly discriminate BVMT-R-DR 56.0%, PASAT-3 27.4%, PASAT-2 27.5%,
employed from disabled patients after adjusting for covari- SDMT 51.9%, DKEFS-CS 15.8%, DKEFS-DS 25.8%.
ates that could also impact on vocational status. Thus, uncon- Based on the standard of impairment on two or more tests,
ditional logistic regression models were first used to assess 59.5% of MS patients were found to be impaired.
the association between individual NP tests and employment0 MS patients also showed greater degrees of depression
disability status after adjusting for age, sex, years of edu- on the BDI-FS (MS mean 5 3.6 6 3.3, Normal mean 5
cation, depression status (depressed: BDF-FS Score .3; 0.9 6 1.5; p , .001). However, there were no significant
not depressed BDI-FS ⱕ 3), and disease course (SP vs. correlations between BDI-FS and NP test performance in
RR). NP tests that were not significant after adjusting for either group (largest MS r 5 2.15 for SDMT).
covariates in the preliminary analyses were not entered into ANOVAs also showed that all 11 NP variables signifi-
the final models. In order to further limit the number of cantly discriminated RR and SP patients at a probability of
predictor variables in the final models, we selected only p , .01. In each case, RR patients performed better than SP
one measure from each test (i.e., COWAT, JLO, CVLT-II- patients. Effect sizes ranged from 0.42 for JLO to 0.92 for
DR, BVMT-R-DR, DKEFS-DS, PASAT-3, SDMT) in the SDMT.

Table 3. Group data comparing MS patients and normal healthy controls

Test NC Mean NC SD MS Mean MS SD d P

COWAT 40.7 12.3 34.6 10.7 0.53 ,.001
JLO 25.0 3.4 22.7 5.9 0.49 .004
CVLT-II Total Learning 55.7 9.4 48.3 11.6 0.70 ,.001
CVLT-II Delayed Recall 12.4 2.5 10.0 3.6 0.79 ,.001
BVMT-R Total Learning 27.7 5.2 21.0 7.7 1.04 ,.001
BVMT-R Delayed Recall 10.6 1.6 8.2 2.9 1.07 ,.001
PASAT 3.0 ISI 47.2 10.6 39.6 14.5 0.61 ,.001
PASAT 2.0 ISI 36.7 12.0 29.7 12.3 0.58 ,.001
SDMT 62.8 8.8 47.8 14.1 1.31 ,.001
DKEFS Sorting CS 11.0 2.1 9.5 2.6 0.64 ,.001
DKEFS Sorting Description 42.2 8.4 35.3 11.6 0.69 ,.001

Note. Groups matched on all demographics.

Abbreviations: COWAT 5 Controlled Oral Word Association Test; JLO 5 Judgment of Line Orientation
Test; CVLT-II 5 California Verbal Learning Test, second edition; BVMT-R 5 Brief Visuospatial Mem-
ory Test—Revised; PASAT 5 Paced Auditory Serial Addition Test; SDMT 5 Symbol Digit Modalities
Test; DKEFS 5 Delis-Kaplan Executive Function System.
554 R.H.B. Benedict et al.

DKEFS-DS) were significant in the logistic models that

included only a single NP test adjusted for covariates (i.e.,
age, sex, education, course, and depression), with odds ratios
ranging from 3.07 for SDMT to 6.02 for COWAT ( p ,
.01). The final model, which included covariates and the
five NP variables found to be significant in preliminary
analyses, retained CVLT-DR and PASAT-3 (Table 5).
When the liberal definition of vocational status was used,
the following measures were found to be significant in pre-
liminary logistic models that included only a single NP test:
COWAT, CVLT-II-DR, BVMT-R DR, PASAT-3, SDMT,
DKEFS-DS. After adjusting for covariates, odds ratios for
these individual NP tests ranged from 2.05 ( p 5 .03) for
SDMT to 3.25 ( p , .01) for COWAT. In the final model
predicting disability0employment, which included all six of
the NP variables found to be significant in the preliminary
Fig. 1. Frequencies of impairment using a cut-off z score of 21.5 logistic regression analyses, after controlling for covari-
for the healthy comparison group. ates, DKEFS-DS and CVLT-II-DR were retained (Table 5).
These analyses were repeated including only the RR
Principle Components Analysis patients. In the final model predicting the conservative
The *PCA yielded similar results for RR and SP patients disability0employment status for RR patients, there were 3
(Table 4). Three components were derived for RR patients: retained NP tests: DKEFS-DS (odds ratio 3.06, p 5 .10),
(1) processing speed0working memory, (2) memory, and CVLT-II-DR (odds ratio 2.85, p 5 .07), and PASAT-3 (odds
(3) executive function. Each of the three components ratio 3.57, p 5 .03). In the liberal model, BVMT-R-DR
explained over 15% of the variance in the 11 variables (odds ratio 2.25, p , .05) and DKEFS-DS (odds ratio 2.65,
included in the analysis and 70.1% of the variance was p , .05) were retained.
explained by the three components combined. COWAT
showed only modest loadings on all three components. Sim- DISCUSSION
ilarly, the low communality for this variable (.243) indi-
cates that the three components explained only a very small The primary purpose of this study was to investigate the
percentage of COWAT variance. The two BVMT-R vari- validity of the MACFIMS battery in a single, prospectively-
ables loaded most heavily onto the memory component, but accrued sample. All seven NP tests discriminated MS patients
they also loaded significantly onto the processing speed from healthy controls and RR from SP patients. Effect sizes
component. Although the loading for SDMT was only over ranged from medium to very large, and tests emphasizing
0.5 for component 1 (processing speed), it also loaded mar- processing speed and memory were most sensitive to ill-
ginally on the memory component. ness. PCA identified a single episodic memory component
The PCA yielded two components for SP patients: a gen- in both RR and SP samples, and components corresponding
eral component encompassing language, processing speed, to either processing speed and working memory, or a more
working memory, executive function, and a more specific general component involving these constructs, as well as
memory component. Each explained over 15% of the total spatial and executive functions. For most of the MACFIMS
variance. Overall, 67.9% of the variance was explained by tests, poor cognitive performance was associated with voca-
the two components combined. The only variable that loaded tional disability; in logistic regression models accounting
onto both components (i.e., loading ..5) was the SDMT. for all NP tests, tests emphasizing processing speed, verbal
memory, and executive function were most predictive of
vocational status, even after controlling for the effects of
Association with Vocational Status age, education, sex, depression, and disease course.
When the more conservative definition of vocational status As predicted, the MACFIMS tests significantly discrim-
was used, 5 NP tests (COWAT, CVLT-DR, PASAT-3, SDMT, inated MS patients and controls and showed a frequency of
impairment consistent with the previous MS literature. The
*Because of likely correlation between NP measures, oblique rotation effect sizes were roughly parallel with those obtained pre-
was also pursued. This approach resulted in the same frequency of vari- viously using the same test measures (Table 2). Like the
able overlap seen with varimax rotation. In addition to significant overlap seminal work of Rao and colleagues (Rao et al., 1991a), we
between test scores, the oblique rotation resulted in components that did
not meet the a priori criteria that a component must explain at least 15% of found that roughly half of MS patients exhibit cognitive
the variance in the model. In at least one instance, a variable that had impairment and that impairment is most common on tests
loaded over 0.5 on a single component and less than 0.5 on the other that emphasize processing speed, working memory, and
components with varimax method now loaded as almost 0.5 on multiple
components. Both rotation methods resulted in overlap among test scores, recent or episodic memory. Differences in the frequency of
but with the varimax rotation we met the a priori criteria described above. impairment can be expected to differ across studies because
Minimal assessment of cognitive function in MS 555

Table 4. Results of Principle Components Analysis

RR (N 5 200) SP (N 5 78)
C1 C2 C3 h2 C1 C2 h2
COWAT .345 .174 .315 .248 COWAT .453 .385 .375
JLO .589 .180 .142 .399 JLO .604 .385 .513
CVLT-II Total Learning .124 .833 .292 .795 CVLT Total Learning .136 .855 .750
CVLT-II Delayed Recall .128 .902 .143 .851 CVLT Delayed Recall .135 .892 .814
BVMT-R Total Learning .594 .627 .103 .757 BVMT Total Learning .368 .746 .693
BVMT-R Delayed Recall .589 .628 .071 .747 BVMT Delayed Recall .378 .757 .716
PASAT 3.0 ISI .836 .159 .262 .793 PASAT 3.0 .832 .269 .765
PASAT 2.0 ISI .775 .008 .285 .682 PASAT 2.0 .815 .191 .701
SDMT .655 .466 .097 .655 SDMT .583 .567 .661
DKEFS Correct Sorts .266 .237 .866 .877 DKEFS Correction Sorts .835 .181 .729
DKEFS Description Score .195 .118 .919 .896 DKEFS Description Score .844 .191 .749
% variance explained 27.6% 24.3% 18.1% 70.1% % variance explained 37.3% 31.6% 67.9%

Note. Table shows loadings using varimax with Kaiser rotation for each component (C). Communalities (h 2 ) are the proportion of variance in each
variable explained by the derived factors. Final row shows percent of variance explained by all 11variables for factor and the total model.

of variability in testing methods and sample composition. dict et al., 2002). Therefore, these results should not be
This sample, which includes patients undergoing routine viewed as a representation of cognitive functioning in MS.
monitoring of cognitive function, patients referred to address Also, the MACFIMS includes only one test of language,
specific clinical problems (e.g., disability, cause of estab- the COWAT, which equally emphasizes lexical search and
lished memory disorder), as well as research volunteers, is response speed. Indeed, this test is also commonly referred
representative of patients attending a hospital-based MS to as a test of executive ability. However, we did expect a
center. Lower frequencies of impairment would be expected priori that a single component representing episodic mem-
in a population-based or community sample, and higher ory would be seen in the data, a hypothesis borne out by the
frequencies might be found in samples of patients seen only PCA models for both RR and SP patients.
for specified clinical purposes. Fewer components emerged for the SP sample, which
In evaluating the PCAs, it is important to bear in mind may be related, in part, to the low number of participants. It
that the MACFIMS tests were chosen to represent a mini- is also possible that as the disease progresses there is less
mal record of cognitive function in MS, and not to assess variation in the NP presentation of MS. Although cognitive
cognitive functioning in a comprehensive manner (Bene- impairment can be found early in the disease, most studies

Table 5. Logistic regression models for MACFIMS tests associated with increased risk of
vocational disability in all MS patients

Model predicting disabled0employed Model predicting disabled0employed

status, conservatively defined (n 5 102) status, liberally defined (n 5 162)
95% 95%
Odds Confidence Odds Confidence
Ratio Interval p Ratio Interval p
Depression† 2.30 0.95–5.54 0.065 2.70 1.45–5.04 .002
Disease Course†† 5.79 1.88–17.8 0.002 5.00 2.2–11.4 ,.001
COWAT ns ns
CVLT-II-DR 3.20 1.19–8.63 0.022 2.82 1.41–5.66 .003
BVMT-R-DR ns ns
PASAT 3.0 4.34 1.63–11.6 0.003 ns
SDMT ns ns
DKEFS DS ns 2.27 1.03–5.01 .042

†BDI-FS Score .3 5 depressed. †† RR versus SP.

For both models, NP variables (impaired vs. not impaired), depression status, course and demographics entered
first. Backward stepwise logistic regression then used to establish the best final model. In order to increase the ratio
of sample size to variables, thereby increasing power, all covariates with p ⱕ .15 retained, Variables with p . 0.15
eliminated from the final model.
556 R.H.B. Benedict et al.

show that patients with progressive disease are more neuro- tive assessment of neurological status using the EDSS was
psychologically affected than RR patients (Comi et al., 1995; missing in about one-third of the sample, which means we
Heaton, 1985; Kraus et al., 2005). In a more demented state, could not control for this factor in the logistic regression
impairment becomes more common across all NP tests, analysis. However, we were able to control for disease course
which makes identification of underlying components (RR vs. SP), which has been shown to be associated with
difficult. EDSS (Jacobs et al., 1999). While the patients were screened
It is interesting that the SDMT had a split loading in for gross deficiency in upper extremity motor control, we
the PCA for SP patients, and nearly so in the RR analysis. did not administer the copy trial of the BVMT-R, and it is
This test is regarded to be very strongly correlated with conceivable that mild motor dysfunction may have contrib-
brain atrophy in recent MS studies (Benedict et al., 2004; uted to performance on this test. The effect size for this
Christodoulou et al., 2003) perhaps because, like the PASAT particular measure was large, and may have been inflated
(Lockwood et al., 2004), performance depends on rapid by drawing problems due purely to manual or motor dys-
communication between widely dispersed cortical regions function. In future work, we plan to study the contribution
via long white matter tracts. However, in our previous work, of motor control to BVMT-R performance using standard-
the SDMT was correlated not only with general measures ized tests of manual speed and dexterity. One of the advan-
of brain atrophy but also regional temporal lobe atrophy tages of these MACFIMS tests lies with their potential for
(Benedict et al., 2005b). The explanation for this may lie in using manifold variables from a single procedure. This is
the fact that SDMT performance is also enhanced by remem- especially true of the CVLT-II and the DKEFS. Because the
bering new associations between symbols and numbers. This number of variables was intentionally limited to control for
multimodal aspect of the SDMT may account for its very type 1 error, future studies might examine these tests in
high sensitivity in the present study. greater detail.
The ecological validity of NP testing is increasingly The above notwithstanding, we find support for the valid-
emphasized in the MS literature (Goverover et al., 2006; ity of the MACFIMS battery in MS patients. These tests
Higginson et al., 2000; Beatty et al., 1995a). Evaluations demonstrate construct validity, discriminate MS patients from
are often requested to determine whether or not a patient healthy controls and RR from SP patients, and are associ-
has a legitimate claim for disability benefits. In this study, ated with greater risk for vocational disability. Future work
logistic regression analysis determined the extent to which will investigate the test-retest reliability of these tests in a
these MACFIMS tests are associated with a dichotomous large sample, and the longitudinal course of NP dysfunc-
distinction of disabled versus employed. This is a difficult tion in MS using these measures.
psychosocial construct to measure. Our method was based
on patient and informant reported status ascertained at the
time of NP testing. Whereas we have no reason to suspect ACKNOWLEDGMENTS
that these reports were biased, our measurements do not The author acknowledges the support of the clinical and adminis-
account for why a person is disabled. To account for this, trative staff from Jacobs Neurological Institute and Buffalo Neuro-
we eliminated patients for whom the decision to stay out of imaging Analysis Center. Some of the research was supported, in
the work force is probably multifactorial (e.g., home- part, by an unrestricted educational grant from Biogen Idec.
maker). Nevertheless, there were strong associations between
NP testing and vocational outcomes, even after controlling
for the influence of demographics, depression, and disease REFERENCES
course. While depression and disease course also were sig-
American Psychiatric Association (1994). Diagnostic and Statis-
nificantly associated with employment status, the associa- tical Manual of Mental Disorders. (4th ed.) Washington, DC:
tions between NP testing and vocational outcomes remained American Psychiatric Association.
even after accounting for the effects of these variables. As Archibald, C.J. & Fisk, J.D. (2000). Information processing effi-
would be expected, verbal memory and reasoning tests dem- ciency in patients with multiple sclerosis. Journal of Clinical
onstrated the strongest associations with vocational status. & Experimental Neuropsychology, 22, 686–701.
While the SDMT was the NP test showing the most fre- Arnett, P.A., Rao, S.M., Bemardill, L., Luchetta, T., Frost, J.,
quent impairment in this sample, the PASAT-3 was the pro- Meidenbauer, K., & Heise, R. (1997). Executive functions in
cessing speed measure showing closest alignment with multiple sclerosis: An analysis of temporal ordering, semantic
vocational disability. One explanation for this may be that encoding, and planning abilities. Neuropsychology, 11,
the PASAT also involves numerical processing, a skill 535–544.
Beatty, W.W. (1996). Memory disturbance in multiple sclerosis:
required in many occupations.
Reconsideration of patterns of performance on the Selective
In addition to our concerns about the measurement of Reminding Test. Journal of Clinical & Experimental Neuro-
vocational disability in this study, there are other method- psychology, 18, 56– 62.
ological limitations that merit comment. The study was cross- Beatty, W.W., Blanco, C.R., Wilbanks, S.L., & Paul, R.H. (1995a).
sectional in design, and whereas we can say something about Demographic, clinical, and cognitive characteristics of multi-
the validity of these NP tests at a single point in time, their ple sclerosis patients who continue to work. Journal of Neuro-
predictive validity, in a strict sense, is unknown. Quantita- logic Rehabilitation, 9, 167–173.
Minimal assessment of cognitive function in MS 557

Beatty, W.W., Goodkin, D.E., Monson, N., & Beatty, P.A. (1989). Benton, A.L., Sivan, A.B., Hamsher, K., Varney, N.R., & Spreen,
Cognitive disturbances in patients with relapsing remitting mul- O. (1994). Contributions to Neuropsychological Assessment.
tiple sclerosis. Archives of Neurology, 46, 1113–1119. (2nd ed.) New York: Oxford University Press.
Beatty, W.W., Hames, K.A., Blanco, C.R., Paul, R.H., & Wil- Christodoulou, C., Krupp, L.B., Liang, Z., Huang, W., Melville,
banks, S.L. (1995b). Verbal abstraction deficit in multiple scle- P., Roque, C., Scherl, W.F., Morgan, T.F., MacAllister, W.S.,
rosis. Neuropsychology, 9, 198–205. Li, L., Tudorica, L.A., Roche P., & Peyster, R. (2003). Cogni-
Beatty, W.W. & Monson, N. (1996). Problem solving by patients tive performance and MR markers of cerebral injury in cogni-
with multiple sclerosis: Comparison of performance on the tively impaired MS patients. Neurology, 60, 1793–1798.
Wisconsin and California Card Sorting Test. Journal of the Cohen, J. (1988). Statistical Power for the Behavioral Sciences.
International Neuropsychological Society, 2, 134–140. (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Beatty, W.W. & Monson, N. (1994). Picture and motor sequencing Comi, G., Filippi, M., Martinelli, V., Campi, A., Rodegher, M.,
in multiple sclerosis. Journal of Clinical and Experimental Alberoni, M., Siranbian, G., & Canal, N. (1995). Brain MRI
Neuropsychology, 16, 165–172. correlates of cognitive impairment in primary and secondary
Beatty, W.W., Paul, R.H., Wilbanks, S.L., Hames, K.A., Blanco, progressive multiple sclerosis. Journal of the Neurological
C.R., & Goodkin, D.E. (1995c). Identifying multiple sclerosis Sciences, 132, 222–227.
patients with mild or global cognitive impairment using the D’Esposito, M., Onishi, K., Thompson, H., Robinson, K., Arm-
screening examination for cognitive impairment (SEFCI). Neu- strong, C., & Grossman, M. (1996). Working memory impair-
rology, 45, 718–723. ments in multiple sclerosis: Evidence from a dual-task paradigm.
Beck, A.T., Steer, R.A., & Brown, G.K. (2000). BDI-Fast Screen Neuropsychology, 10, 51–56.
for Medical Patients: Manual. San Antonio, TX: Psychologi- Delis, D.C., Kaplan, E., & Kramer, J.H. (2001). Delis-Kaplan Exec-
cal Corporation. utive Function System. San Antonio, Texas: Psychological
Benedict, R.H.B. (1997). Brief Visuospatial Memory Test–Revised: Corporation.
Professional Manual. Odessa, Florida: Psychological Assess- Delis, D.C., Kramer, J.H., Kaplan, E., & Ober, B.A. (2000). Cal-
ment Resources, Inc. ifornia Verbal Learning Test Manual, Adult Version (2nd ed.).
Benedict, R.H.B., Fischer, J.S., Archibald, C.J., Arnett, P.A., Beatty, San Antonio, TX: Psychological Corporation.
W.W., Bobholz, J., Chelune, G.J., Fisk, J.D., Langdon, D.W., DeLuca, J., Gaudino, E.A., Diamond, B.J., Christodoulou, C., &
Caruso, L., Foley, F., LaRocca, N.G., Vowels, L., Weinstein, Engel, R.A. (1998). Acquisition and storage deficits in multi-
A., DeLuca, J., Rao, S.M., & Munschauer, F. (2002). Minimal ple sclerosis. Journal of Clinical and Experimental Neuropsy-
Neuropsychological Assessment of MS Patients: A Consensus chology, 20, 376–390.
Approach. Clinical Neuropsychologist, 16, 381–397. Demaree, H.A., DeLuca, J., Gaudino, E.A., & Diamond, B.J. (1999).
Benedict, R.H.B., Fishman, I., McClellan, M.M., Bakshi, R., & Speed of information processing as a key deficit in multiple
Weinstock-Guttman, B. (2003). Validity of the Beck Depres- sclerosis: Implications for rehabilitation. Journal of Neurol-
sion Inventory—Fast Screen in multiple sclerosis. Multiple Scle- ogy, Neurosurgery & Psychiatry, 67, 661– 663.
rosis, 9, 393–396. Fischer, J.S. (1988). Using the Wechsler Memory Scale–Revised
Benedict, R.H.B., Priore, R.L., Miller, C., Munschauer, F., & Jacobs, to detect and characterize memory deficits in multiple sclero-
L. (2001a). Personality disorder in multiple sclerosis correlates sis. Clinical Neuropsychologist, 2, 149–172.
with cognitive impairment. Journal of Neuropsychiatry & Clin- Fisk, J.D. & Archibald, C.J. (2001). Limitations of the Paced Audi-
ical Neurosciences, 13, 70–76. tory Serial Addition Test as a measure of working memory in
Benedict, R.H.B., Schretlen, D., Groninger, L., Dobraski, M., & patients with multiple sclerosis. Journal of the International
Shpritz, B. (1996). Revision of the Brief Visuospatial Memory Neuropsychological Society, 7, 363–372.
Test: Studies of normal performance, reliability, and validity. Goverover, Y., Kalmar, J., Gaudino-Goering, E., Shawaryn, M.,
Psychological Assessment, 8, 145–153. Moore, N.B., Halper, J. & DeLuca, J. (2006). Relation between
Benedict, R.H.B., Shapiro, A., Priore, R.L., Miller, C., Mun- subjective and objective measures of everyday life activities in
schauer, F.E., & Jacobs, L.D. (2001b). Neuropsychological coun- persons with multiple sclerosis. Archives of Physical Medicine
seling improves social behavior in cognitively-impaired multiple & Rehabilitation, in press.
sclerosis patients. Multiple Sclerosis, 6, 391–396. Gronwall, D.M.A. (1977). Paced auditory serial addition task: A
Benedict, R.H.B., Wahlig, E., Bakshi, R., Fishman, I., Mun- measure of recovery from concussion. Perceptual and Motor
schauer, F., Zivadinov, R., & Weinstock-Guttman, B. (2005a). Skills, 44, 367–373.
Predicting quality of life in multiple sclerosis: Accounting for Heaton, R.K. (1985). Neuropsychological findings in relapsing-
physical disability, fatigue, cognition, mood disorder, person- remitting and chronic-progressive multiple sclerosis. Journal
ality, and behavior change. Journal of the Neurological Sci- of Consulting & Clinical Psychology, 53, 103–110.
ences, 231, 29–34. Higginson, C.I., Arnett, P.A., & Voss, W.D. (2000). The ecological
Benedict, R.H.B., Weinstock-Guttman, B., Fishman, I., Sharma, validity of clinical tests of memory and attention in multiple
J., Tjoa, C.W., & Bakshi, R. (2004). Prediction of neuropsy- sclerosis. Archives of Clinical Neuropsychology, 15, 185–204.
chological impairment in multiple sclerosis: Comparison of Jacobs, L.D., Wende, K.E., Brownscheidle, C.M., Apatoff, B.,
conventional magnetic resonance imaging measures of atro- Coyle, P.K., Goodman, A., Gottesman, M.H., Granger, C.V.,
phy and lesion burden. Archives of Neurology, 61, 226–230. Greenberg, S.J., Herbert, J., Krupp, L., Lava, N.S., Mihai, C.,
Benedict, R.H.B., Zivadinov, R., Carone, D., Weinstock-Guttman, Miller, A.E., Perel, A., Smith, C.R., & Snyder, D.H. (1999). A
B., Gaines, J., Maggiore, C., Sharma, J., Tomassi, M., & Bak- profile of multiple sclerosis: The New York State Multiple Scle-
shi, R. (2005b). Regional Lobar Atrophy Predicts Memory rosis Consortium. Multiple Sclerosis, 5, 369–376.
Impairment in Multiple Sclerosis. American Journal of Neuro- Knight, R.G., Devereux, R.C., & Godfrey, H.P.D. (1997). Psycho-
radiology, 26, 1824Y1831. social consequences of caring for a spouse with multiple scle-
558 R.H.B. Benedict et al.

rosis. Journal of Clinical and Experimental Neuropsychology, Rao, S.M., Leo, G.J., & St. Aubin-Farbert, P. (1989a). Information
19, 7–19. processing speed in patients with multiple sclerosis. Journal of
Kraus, J.A., Schutze, C., Brokate, B., Kroger, B., Schwendemann, Clinical & Experimental Neuropsychology, 11, 471– 477.
G., & Hildebrandt, H. (2005). Discriminant analysis of the Rao, S.M., Leo, G.J., Bernardin, L., & Unverzagt, F. (1991b).
cognitive performance profile of MS patients differentiates their Cognitive dysfunction in multiple sclerosis. I. Frequency, pat-
clinical course. Journal of Neurology, 252, 808–813. terns, and prediction. Neurology, 41, 685– 691.
Krupp, L.B., Sliwinski, M., Masur, D.M., Friedberg, F., & Coyle, Rao, S.M., Leo, G.J., Haughton, V.M., Aubin-Faubert, P.S., &
P.K. (1994). Cognitive functioning and depression in patients Bernardin, L. (1989b). Correlation of magnetic resonance imag-
with chronic fatigue syndrome and multiple sclerosis. Archives ing with neuropsychological testing in multiple sclerosis. Neu-
of Neurology, 51, 705–710. rology, 39, 161–166.
Kurtzke, J.F. (1983). Rating neurologic impairment in multiple Ryan, L., Clark, C.M., Klonoff, H., Li, D., & Paty, D. (1996).
sclerosis: An expanded disability status scale (EDSS). Annals Patterns of cognitive impairment in relapsing-remitting multi-
of Neurology, 13, 227–231. ple sclerosis and their relationship to neuropathology on mag-
Lockwood, A.H., Linn, R.T., Szymanski, H., Coad, M.L., & Wack, netic resonance images. Neuropsychology, 10, 176–193.
D.S. (2004). Mapping the neural systems that mediate the Paced Scarrabelotti, M. & Carroll, M. (1998). Awareness of remember-
Auditory Serial Addition Task (PASAT). Journal of the Inter- ing achieved through automatic and conscious processes in
national Neuropsychological Society, 10, 26–34. multiple sclerosis. Brain & Cognition, 38, 183–201.
McDonald, W.I., Compston, A., Edan, G., Goodkin, D.E., Har- Smith, A. (1982). Symbol digit modalities test: Manual. Los Ange-
tung, H., Lublin, F., McFarland, H.F., Paty, D.W., Polman, les: Western Psychological Services.
C.H., Reingold, S.C., Sandberg-Wollheim, M., Sibley, W. A., Sperling, R.A., Guttmann, C.R., Hohol, M.J., Warfield, S.K., Jakab,
Thompson, A., van der Noort, S., Weinshenker, B.Y., & Wolin- M., Parente, M., Diamond, E.L., Daffner, K.R., Olek, M.J.,
sky, J. S. (2001). Recommended diagnostic criteria for multi- Orav, E.J., Kikinis, R., Jolesz, F.A., & Weiner, H.L. (2001).
ple sclerosis: Guidelines from the international panel on the Regional magnetic resonance imaging lesion burden and cog-
diagnosis of multiple sclerosis. Annals of Neurology, 50, nitive function in multiple sclerosis: A longitudinal study.
121–127. Archives of Neurology, 58, 115–121.
Rao, S.M. (1991). A Manual for the Brief, Repeatable Battery of Thornton, AE., Raz, N., & Tucke, K.A. (2002). Memory in mul-
Neuropsychological Tests in Multiple Sclerosis. New York, NY: tiple sclerosis: contextual encoding deficits. Journal of the Inter-
National Multiple Sclerosis Society. national Neuropsychological Society, 8, 395– 409.
Rao, S.M., Leo, G.J., Ellington, L., Nauertz, T., Bernardin, L., & Vleugels, L., Lafosse, C., van Nunen, A., Nachtergaele, S.,
Unveragt, F. (1991a). Cognitive dysfunction in multiple scle- Ketelaer, P., Charlier, M., & Vandenbussche, E. (2000). Visuo-
rosis. II. Impact on employment and social functioning. Neu- perceptual impairment in multiple sclerosis patients diagnosed
rology, 41, 692– 696. with neuropsychological tasks. Multiple Sclerosis, 6, 241–254.