Benedict et al.

BMC Neurology 2012, 12:55

http://www.biomedcentral.com/1471-2377/12/55

ORIGINAL PAPER Open Access

Brief International Cognitive Assessment for MS

(BICAMS): international standards for validation
Ralph HB Benedict*, Maria Pia Amato, Jan Boringa, Bruno Brochet, Fred Foley, Stan Fredrikson, Paivi Hamalainen,
Hans Hartung, Lauren Krupp, Iris Penner, Anthony T Reder and Dawn Langdon

Abstract
An international expert consensus committee recently recommended a brief battery of tests for cognitive
evaluation in multiple sclerosis. The Brief International Cognitive Assessment for MS (BICAMS) battery includes tests
of mental processing speed and memory. Recognizing that resources for validation will vary internationally, the
committee identified validation priorities, to facilitate international acceptance of BICAMS. Practical matters
pertaining to implementation across different languages and countries were discussed. Five steps to achieve
optimal psychometric validation were proposed. In Step 1, test stimuli should be standardized for the target culture
or language under consideration. In Step 2, examiner instructions must be standardized and translated, including all
information from manuals necessary for administration and interpretation. In Step 3, samples of at least 65 healthy
persons should be studied for normalization, matched to patients on demographics such as age, gender and
education. The objective of Step 4 is test-retest reliability, which can be investigated in a small sample of MS and/or
healthy volunteers over 1–3 weeks. Finally, in Step 5, criterion validity should be established by comparing MS and
healthy controls. At this time, preliminary studies are underway in a number of countries as we move forward with
this international assessment tool for cognition in MS.

Background mental function, including psychomotor speed and dex-

Multiple sclerosis (MS) is an inflammatory disease of the terity, personality or psychopathology [via standardized
central nervous system, causing demyelination and neuro- questionnaires or surveys], intelligence, memory and other
degeneration in most patients [1,2]. As would be expected aspects of cognitive processing [eg attention, language, ex-
in such a disease with prominent cerebral pathology, a ecutive function].
substantial number [3-5] of MS patients are compromised The term neuropsychological test conveys the idea that
neuropsychologically. In recently diagnosed or benign the psychometric test result is relevant for conclusions per-
course patients, the incidence of cognitive impairment taining to cerebral function. Neuropsychological tests are
ranges from 20-40% [5,6]. In clinic based samples where used to examine brain-injured patients or to study hypoth-
secondary progressive course is more common, roughly eses in neuroscience. A deficient neuropsychological test
50-60% of patients are affected [4]. value is often judged to be indicative of cerebral dysfunc-
Neuropsychological (NP) testing provides quantification tion. The Wechsler Adult Intelligence Scale (WAIS) [9], is
of cognition, and is used clinically to diagnose impairment a classic example of a psychometric test of intelligence. It
and to inform medical and behavioral treatment decisions has carefully standardized instructions, scoring criteria, ex-
[7]. Two descriptors, psychometric or neuropsychological tensive age-based normative data, and information derived
tests, are often used inter-changeably to describe the cog- from extensive research concerning reliability and validity.
nitive testing procedures used with MS patients [8]. Psy- It can also be construed as a neuropsychological test [espe-
chometric tests are standardized, behavioral measures of cially nonverbal components] because there are extensive
mental phenomena. They measure many domains of data that show its relationship with cognitive aging, de-
mentia, and other changes in cerebral status.
* Correspondence: [email protected] Psychometric data regarding normal performance, test
Buffalo General Medical Center, Department of Neurology, Suite E2, 100 High reliability and the validity of test interpretation are
Street, Buffalo, NY 14203

© 2012 Benedict et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative
Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly cited.
Benedict et al. BMC Neurology 2012, 12:55 Page 2 of 8
http://www.biomedcentral.com/1471-2377/12/55

necessary for accurate application of NP testing [10]. well as psychometric research to insure the reliability and
Most tests in common use are carefully standardized validity of new test forms. With this in mind, a second
such that the same instructions, stimuli and marking cri- conference was held to develop consensus on a BICAMS
teria are used by all examiners. Most often, high test- validation protocol.
retest reliability is emphasized in order to avoid error in
repeat testing circumstances. Unfortunately, alternate Brief International Cognitive Assessment for MS
test versions and normative data are not available for (BICAMS) description
some NP tests in all languages and cultures. In addition, The SDMT [13] presents a series of nine symbols, each
many centers lack expertise in psychometrics and NP paired with a single digit in a key at the top of a stand-
test interpretation. ard sheet of paper. An adapted version of the test is pre-
The Brief International Cognitive Assessment for MS sented in Figure 1. Patients are asked to voice the digit
(BICAMS) initiative was undertaken to recommend a brief, associated with each symbol as rapidly as possible for
cognitive assessment for MS that is optimized for small 90 sec. There is a single outcome measure – the number
centers, with perhaps one or few staff members, who may correct over the 90 sec time span.
not have NP training [11]. BICAMS was particularly fo- The auditory/verbal learning test is the CVLT2 [16].
cused on international use, to facilitate comparison across The test begins with the examiner reading a list of 16
settings. An expert committee of twelve neurologists and words [Figure 2]. Patients listen to the list and report as
neuropsychologists representing the main cultural groups many of the items as possible. There is no instruction as
that have so far contributed extensive data about cognitive to the order in which items are recalled. After recall is
dysfunction in MS was convened. The opinions generated recorded, the entire list is read again followed by a sec-
from the meeting are published elsewhere [11]. In brief, ond attempt at recall. Altogether, there are five learning
the panel recommended one particular test with high reli- trials. The reader will note that the 16-item list [see faux
ability and good sensitivity, the Rao [12] adaptation of the example in Figure 2] has words that conform to four
Symbol Digit Modalities Test (SDMT) [13]. Consensus was semantic categories, in this case sports, vegetables,
also achieved on optimal measures for learning and mem- clothes, and tools. Some subjects will recall items in
ory in MS patients, time permitting: the initial learning a grouped fashion, and others may recall the list in
trials of the second edition of the California Verbal Learn- serial order. There are many variables of recall avail-
ing Test (CVLT2) [14] and the revised Brief Visuospatial able in the CVLT2, as a second list is presented, and
Memory Test (BVMTR) [15]. after 25 min there is a delayed recall trial as well as
In order to facilitate international implementation of the a yes/no recognition memory task. The BICAMS
BICAMS assessment, multiple translations are needed, as panel noted that few studies have shown incremental

Figure 1 Faux stimuli for Symbol Digit Modalities Test.

Benedict et al. BMC Neurology 2012, 12:55 Page 3 of 8
http://www.biomedcentral.com/1471-2377/12/55

Figure 2 Faux stimuli for California Verbal Learning Test Second Edition.

validity with these measures, as the total number of on US samples and discussion shifted to practical matters
recalled items over the five learning trials is most pertaining to implementation across different languages
sensitive [17]. and countries.
Visual/spatial memory is assessed in BICAMS using the
BVMTR [18]. In this test, six abstract designs [Figure 3]
are presented for 10 sec. The display is removed from view
and patients render the stimuli via pencil on paper manual
responses. Each design receives from 0 to 2 points repre-
senting accuracy and location. Thus, scores range from 0
to 12. There are three learning trials, and the primary out-
come measure is the total number of points earned over
the three learning trials. Because there is little evidence
that the delayed recall trial adds to discriminant validity in
MS [4,19], as in the MATRICS consensus battery [20],
only the initial learning trials are recommended for
BICAMS.

Conference process
Having already decided on the core [SDMT] and ancillary
memory tests [CVLT2, BVMTR], the focus of the confer-
ence was on a BICAMS psychometric validation protocol.
The committee reviewed basic psychometric standards
from the literature [10,21-26] that are widely accepted for
validation of behavioral or psychological outcome mea-
sures. Recognizing that economic resources for validation
will vary across country and region, the committee dis-
cussed priorities for validation, that is, those aspects of re-
search design that would enable empirical appraisal of
core psychometrics that may engender confidence and
wide application of BICAMS. Following consensus on
Figure 3 Faux stimuli for Brief Visuospatial Memory Test
these central components, it was noted that most of the
Revised.
psychometric evidence relating to the BICAMS tests relies
Benedict et al. BMC Neurology 2012, 12:55 Page 4 of 8
http://www.biomedcentral.com/1471-2377/12/55

Psychometric standards The SDMT has particularly high test-retest reliability. In

Standardization one US study of MS patients the test-retest r value was
The first step in the development of any test is to ensure 0.97. In a US study repeating the test over six monthly ses-
that test stimuli and administration procedures have good sions, r values approximated 0.80 for healthy controls and
face validity and consistent stimulus presentation. While 0.90 for MS patients. Acceptable test-retest reliability
this can be a painstaking process during the initial phases [CVLT2 = 0.78; BVMTR = 0.91] was found in a well con-
of psychometric test development, the work is already trolled investigation with US MS patients [31].
done for the tests that were selected for BICAMS. The
SDMT, CVLT2 and BVMTR stimuli are well established Validity
and are readily mass produced using digital and print Rather than accuracy, validity refers to the meaning of a
technology. Examiners can be easily trained to use stan- test score. There are multiple aspects of validity in psy-
dardized instructions, more or less verbatim, to enhance chometric science. Does a low score, for example, repre-
reliability across settings. The BICAMS tests are adequate sent the presence of neuropsychological dysfunction, a
in this regard. particular disease state such as MS, a high likelihood of
brain atrophy or some other marker of cerebral involve-
Normalization of raw scores ment, or perhaps an increased risk of failing at work?
Normative data are of course essential for the clinical Correlations between test scores and other measures [ie
application of psychometric, neuropsychological tests. validity coefficient r] and comparing differences in the
Acquiring normative data can be an expensive endeavor. performance from specified samples [eg, MS vs controls;
For example, the recently revised US normative data for employed vs disabled MS patients] are common meth-
BVMTR has a sample size of 588 [27]. These data were ods of investigation.
fairly recently acquired and are judged by the BICAMS The BICAMS committee decided that the most import-
committee to be current. ant aspect of validity for clinical purposes is criterion-
Normalization data for SDMT are more complicated. related validity, most notably differentiating MS patients
The manual based norms date to 1982, raising the spectre from healthy controls. All of the BICAMS tests discrimin-
of cohort effects [ie gradual shift upward over time] or ate well with SDMT most often the most sensitive meas-
poor generalizablity to patients and controls in the present ure in NP batteries [4,19].
day. Benedict et al published normative data on the MAC- A more difficult endeavor is to establish the ecological
FIMS battery, which includes the SDMT, in 2006 [4] and or predictive validity of a psychometric test [32]. Neuro-
2010 [28], using US healthy samples numbering 56 and psychological testing is correlated with a wide range of
120 respectively. These are controlled research studies with activities of daily living in MS [33-40], as well as work
applicability largely restricted to clinicians treating MS disability [41-44]. The BICAMS tests are correlated with
patients. A potential problem is that normative data from vocational outcomes [4,45-47] and recently job loss was
one region [North Eastern USA in this case] may result in associated with specific decline of 4–5 points on SDMT
interpretive error when applied to raw test scores derived [48]. More such research is needed in order to clearly
from a different culture, language, region or country. establish thresholds for clinically meaningful changes for
the BICAMS measures.
Reliability
By reliability we mean the degree to which there is error Alternate test forms
when using the same instrument across settings, exami- When NP tests are repeated in healthy volunteers or
ners, etc. It is perhaps the most critical psychometric cri- stable MS patients, performance often improves for two
terion - if the test is unreliable, there is little confidence reasons: item-specific or task-specific learning [49]. The
in the validity of the outcome. Of the various forms of former refers to the learning of, or memory for, specific
reliability, the panel decided that test-retest reliability test stimuli. For example, on the CVLT2, one might re-
has the highest priority and is most relevant for future member specific words from one testing session to the
BICAMS validation. The coefficient of variation can be next. Indeed, on a similar test, the 1st trial recall was
used in very small samples to determine the extent to similar to the delayed recall trial of the same test admi-
which changes in mean values outweigh the variance in nistered two weeks earlier, but only when the same list
test scores [29]. A more valid measure of test-retest reli- was repeated [49]. Test- or task-specific learning refers
ability is the Pearson correlation coefficient [30]. Most to the benefit of performing the same behavioral proced-
commonly, a brief test-retest interval ranging from ure successively, even if the to-be-remembered stimuli
1–3 weeks is employed. For most purposes, r values for are altered. For example, one could possibly learn to
test-retest correlation are considered adequate if >0.70 label BVMTR figures verbally, a strategy that may carry
and good if >0.80 [23]. over to the next session, despite presenting different
Benedict et al. BMC Neurology 2012, 12:55 Page 5 of 8
http://www.biomedcentral.com/1471-2377/12/55

figures. While both item- and task-specific practice likely One consideration is the pronounciation of numbers
play a role in retest effects, in MS, we [31] have found which may vary from monosyllabic to polysyllabic utter-
that changing stimuli reduces practice effects on mem- ances [eg one in English and nueve in Spanish], or be sim-
ory tests such as the CVLT2 and BVMTR. ply longer in others [eg üheksa in Estonian]. Rarely, the
The SDMT was originally published with one test meaning of the SDMT and BVMTR symbols could be-
form. Alternate forms were generated by Rao and collea- come important. These stimuli have little semantic mean-
gues [3,12], but in the only work examining inter-form ing in English but could conceivably have meaning in
equivalence [50], there was little support for the forms some cultures. Like the SDMT stimuli, the BVMTR test
being equivalent. Recently, two new forms were created stimuli are adequate for international application.
and found to be equivalent to the standard form [51]. The CVLT2 is of course entirely another matter. Here,
There are two forms for the CVLT2. In the CVLT2 test precise translation necessitates as close approximation of
manual, the normative data are very similar for each the English words as possible, while maintaining word fre-
form, and the forms yielded similar data in a test-retest quency in the target language, semantic relationships among
within-subjects design [31]. There is strong support for the target words, orthography, and alike. As noted above, in
inter-form reliability for the BVMTR [20,31,49,52-54]. some languages this could mean a very arduous process.
The BICAMS committee agreed that in some countries, an-
Consensus opinion: the BICAMS validation other, simple, auditory word-list learning test could replace
protocol the CVLT2, provided that the procedure is in the common
The above list of psychometric criteria is not intended to format – that is reading the list on each learning trial and
be comprehensive, but the discussed items are essential in including at least three learning trials. Some English lan-
the test development process. As we move forward with guage examples are the Rey Auditory Verbal Learning Test
implementing BICAMS internationally, each of these cri- [55] and the revised Hopkins Verbal Learning Test [56,57].
teria may be difficult to achieve in other languages and cul- All BICAMS tests must re-standardize the administra-
tures. In this section, we describe the suggested, core tion and scoring instructions in the new language. The
validation process, highlighting special considerations for time required for Step 2 will depend on the specific test
each of the BICAMS measures. and technical support available. The patient instructions
The BICAMS tests were selected, in part, due to extant for SDMT are brief, and there is minimal instruction ne-
validation findings and thus it is not surprising that they cessary for scoring the test in the standardized manner.
hold up well to psychometric scrutiny. Table 1 sum- In contrast, the scoring aspects of the BVMTR manual
marizes what we know about the current English versions are quite detailed. Patient responses could be delivered
of BICAMS. Note that the tests are good on the most pri- to another party and scored blindly in lieu of translating
mary criteria, involving standardization, normalization, the entire scoring sections of the manual.
test-retest reliability and criterion-related validity. More In Step 3, a sample of at least 65 healthy volunteers
variable data are available pertaining to alternate forms must be studied with the new BICAMS to develop norma-
and predicting clinically meaning changes over time. tive data in the native language. This minimum sample
Looking forward, as summarized in Appendix 1 below, size should provide enough power to detect a medium ef-
we envisage five steps in future validation protocols in fect size in a two-group [eg MS vs controls] comparison.
populations for whom English is not the first language. Unless a larger sample is available, the normalization sam-
Preliminary work in Step 1 will be needed to maximize ple should be group matched to population studies of MS
standardization while remaining true to the meaning of patients in terms of demographic characteristics. Linear
the original version, where possible. The extant SDMT regression approaches can be employed to extend the ap-
stimuli are deemed adequate for international use, at least plicability of the data to demographics that are not fully
for cultures where Arabic numerals are in common use. represented in the database.

Table 1 Manner in which BICAMS measures meet psychometric criteria in samples with English as a first language
SDMT CVLT2 BVMTR
Standardization Smith 1982 Teat Manual Delis 2000 Test Manual Benedict 1997 Test Manual
Normalization Parmenter 2010. Peer review journal article Delis 2000 Test Manual Benedict 2005 Test Manual
Reliability I: Test-Retest Benedict 2005 r = 0.91 Benedict 2005 r = 0.80 Benedict 2005 r = 0.91
Reliability II: Alternate Form Rao 1991. Benedict 2012. Good Delis 2000. Fair Benedict 1996. Good
Validity I: Criterion Related Many Studies. Good Many Studies. Good Many Studies. Good
Validity II: Clinically Meaningful Change Morrow 2010. Fair No or Little Data No or Little Data
Benedict et al. BMC Neurology 2012, 12:55 Page 6 of 8
http://www.biomedcentral.com/1471-2377/12/55

Test-retest reliability can be assessed in both patients and translated, back translated,and checked for errors.
controls by a repeat testing session 1–3 weeks after baseline Where possible the translated instructions should be
[Step 4]. While both samples are of interest, the panel validated againstexpected participant performance in
believes that reliability in MS is more important than in terms of accuracy and error profile. Step 4 will alsocon-
healthy volunteers. In order to assess criterion-related valid- tribute to the accuracy of the test instructions.
ity [Step 5], the controls must be compared to MS patients, ▪Step 3, Normalization. Large samples of 150 or more
with control for demographics. If Steps 1 and 2 have been healthy persons are needed for dataapplicable to persons
completed effectively, all BICAMS variables should discrim- of all ages and diverse ethnicity. The minimum sample
inate the groups significantly, with d values >0.5. Studies to size is 65healthy volunteers, provided they are group
determine the validity of BICAMS in distinguishing MS matched on demographics to either a concurrentMS
populations from healthy controls will need to be carefully sample, or matched to samples in other published de-
constructed, because the criteria of diagnosis of MS scriptive MS studies. Wherepossible, the distribution of
adopted may vary among countries [58]. It may be neces- test scores and error profile of the normalization sample
sary for published data to be segmented to allow compari- shouldbe examined and compared to published distribu-
son with MS samples from other language groups and the tions from other language groups.
BICAMS committee will facilitate this wherever possible. ▪Step 4, Test-Retest Reliability. Assessment of this cri-
Finally, other psychometric considerations include terion can be achieved by evaluatingan MS and/or
inter-rater reliability, alternate forms, and various forms healthy volunteer sample on two occasions separated by
of convergent and discriminant validity. These are not 1–3 weeks. This is thegold standard separation where
deemed essential, but potentially valuable. For example, the question is only test reliability, controlling for matur-
does BICAMS predict vocational outcomes? Are low ationeffects. A Pearson’s correlation coefficient >0.70 will
BICAMS scores associated with brain atrophy? Alternate usually be required.
form reliability work has already been commenced by ▪Step 5, Criterion-Related Validity. This step can be
some members of the BICAMS committee. pursued in conjunction with Step 3, inthat an MS sam-
ple can be compared to a healthy control group that also
Conclusion serves fornormalization. To determine if a new Italian
The Brief International Cognitive Assessment for MS BVMTR is sensitive to MS disease state, forexample,
(BICAMS) initiative was undertaken to recommend a compare 50 patients to the healthy controls in Step 3.
brief, cognitive assessment for MS that can be utilized After the study, the investigator adds another 35 healthy
internationally, in small centers, with perhaps one or few volunteers to round out the normalization sample.
staff members, who may or may not have formal neuro-
Competing interests
psychological training. Consensus was earlier achieved RHBB receives royalties from Psychological Assessment Resources that are in
regarding the BICAMS tests, with special consideration part associated with the Brief Visuospatial Memory Test Revised.
for SDMT, and supplementation by CVLT2 and
Authors’ contributions
BVMTR, time permitting. Research is needed to validate All authors participated in discussion and correspondence to develop this
BICAMS where English is not the first language. In this consensus opinion on the topics covered in this article. RHBB is the lead
article, we have summarized a second consensus opinion author because he led the development of the manuscript. All authors read
and approved the final manuscript.
which offers a process by which BICAMS can be vali-
dated in other languages. Research projects pursuing Competing financial interests
some of the aims described herein are underway. RHBB has acted as a consultant or scientific advisory board member for
Bayer, Biogen Idec, Actelion, and Novartis. He receives royalties from
Psychological Assessment Resources, Inc. He has received financial support
Appendix 1 Recommended Step-by-Step Protocol for research activities from Shire Pharmaceuticals, Accorda and Biogen Idec.
for BICAMS Validation MPA has received research grants and honoraria for serving as speaker at scientific
meetings, consultant, and as member of scientific advisory boards from Bayer
▪Step 1, Standardization and Translation of Test Stimuli. Pharma AG, Biogen Idec, Merck Serono, Sanofi Aventis, Teva and Novartis.
For visual stimuli, determine ifthere are any semantic JB has consulted for Bayer Healthcare and served on speaker bureau for
associations to stimuli in the culture or language under Exencia Pharma Academy.
BB or his institution has received honorarias for speaking at scientific meetings
consideration. For CVLT2 must match new words on and serving as member of scientific advisory boards for Bayer Pharma, Biogen
word frequency and appropriate similarity ofmeaning. If Idec, Merck Serono, Genzyme, Novartis and Teva and BB’s instistution received
these parameters cannot be applied scientifically, then ex- research grants from Bayer Pharma, Teva, Merck Serono, Novartis, Biogen-Idec,
Sanofi-Aventis and ARSEP and Roche.
pert review andperformance on test by appropriate partici- FF has received honoraria for ad boards and lectures from Biogen, Teva
pants will be utilized to assess translation. Neuroscience and Novartis. He has received an investigator grant from Bayer
▪Step 2, Standardization and Translation of Test Healthcare.
SF has received honoraria for lectures, consultancy and educational activities
Instructions. All information from the testmanual neces- from Allergan, Bayer, BiogenIdec, MerckSerono, Sanofi, Teva. PH received
sary for administration and interpretation must be personal compensation from Bayer Healthcare and Novartis for serving on
Benedict et al. BMC Neurology 2012, 12:55 Page 7 of 8
http://www.biomedcentral.com/1471-2377/12/55

scientific advisory boards; consulting for Sanofi-Aventis; served on speaker 15. Benedict RHB: Brief Visuospatial Memory Test - Revised: Professional Manual.
bureau for Bayer Healthcare and Sanofi-Aventis. Odessa, Floriday: Psychological Assessment Resources, Inc.; 1997.
PH Dr Hämäläinen received personal compensation from Bayer Healthcare 16. Benedict RHB, Schretlen DS, Groninger L, Dobraski M. Revision of the brief
and Novartis for serving on scientific advisory boards; consulting for Sanofi- Visuospatial Memory Test: Studies of normal performance, reliability and validity.
Aventis; served on speaker bureau for Bayer Healthcare and Sanofi-Aventis. Psychological Assesment 1996, 8:145–153.
HPH received honoraria with approval by the Rector of Heinrich-Heine- 17. Stegen S, Stepanov I, Cookfair D, et al: Validity of the California Verbal Learning
University from Bayer Healthcare GmbH, Biogen Idec GmbH, Novartis Pharma Test-II in Multiple Sclerosis. Clin Neuropsychol 2010, 24:189–202.
GmbH, Teva Sanofi Aventis, Hoffman-La Roche and Genzyme Corporation for 18. Benedict R: Brief Visuospatial Memory Test-Revised: Professional manual. Odessa,
consulting and speaking at scientific symposia. Florida: Psychological Assessment Resources; 1997.
LK has served on speaker bureau, scientific advisory boards and/or been a 19. Strober L, Englert J, Munschauer F, Weinstock-Guttman B, Rao S, Benedict RH:
consultant for Teva Neurosciences, BiogenIdec, EMD Serono, Multiple Sensitivity of conventional memory tests in multiple sclerosis: comparing
Sclerosis Association of America, Betaseron/Bayer Healthcare the Rao Brief Repeatable Neuropsychological Battery and the Minimal
Pharmaceuticals, Pfizer, Sanofi-Aventis, Axon Advisors; she has received Assessment of Cognitive Function in MS. Mult Scler 2009, 15:1077–1084.
royalties from Genzyme, ER Squibb & Sons, NMSS, Novartis, MedImmune, 20. Buchanan RW, Davis M, Goff D, et al: A Summary of the FDA-NIMH-
Abbott Laboratories, Johnson & Johnson, Roche, Health Professions MATRICS Workshop on Clinical Trial Design for Neurocognitive Drugs for
Conferencing Corp. Schizophrenia. Schizophr Bull 2005, 31:5–19.
IKP has received research grants from Bayer AG Switzerland and the Swiss 21. Benton AL, Sivan AB, Hamsher K, Varney NR, Spreen O: Contributions to
Multiple Sclerosis Society; has received honoraria forserving as speaker at Neuropsychological Assessment. 2nd edition. New York: Oxford University Press;
scientific meetings, consultant, and as member of scientific advisory boards for 1994.
Actelion, Bayer Pharma AG, Biogen Idec, Merck Serono, Roche, and Teva Aventis. 22. Spreen O, Strauss E: A compendium of neuropsychological tests: administration,
ATR has no conflicts involving NPsych testing. DWL has received funding for norms, and commentary. 2nd edition. New York: Oxford University Press; 1998.
travel to scientific meetings from Bayer Healthcare, Vertex; her institution has 23. Anastasi A: Psychological Testing. 6th edition.: Macmillan Publishing Company; 1988.
received honoraria, consultancy fees, research contracts and sponsorship 24. Benedict RHB, Fischer JS, Archibald CJ, et al: Minimal neuropsychological
from Bayer Healthcare, Serono Symposia, Merck-Serono. assessment of ms patients: a consensus approach. Clin Neuropsychol 2002,
DWL has received funding for travel to scientific meetings from Bayer Healthcare, 16:381–397.
Vertex; her institution has received honoraria, consultancy fees, research contracts 25. Amato MP, Zipoli V, Portaccio E: Mulitple-sclerosis related cognitive
and sponsorship from Bayer Healthcare, Serono Symposia, Merck-Serono. changes: a review of cross-sectional and longitudinal studies. J Neurol Sci
2006, 245:41–46.
Acknowledgements 26. Camp SJ, Thompson AJ, Langdon DW: A new test of memory for multiple
We acknowledge Bayer AG for financial support of the consensus conference sclerosis I: format development and stimuli design. Mult Scler 2001, 7:255–262.
meeting. 27. Benedict RHB, Brandt J: Hopkins Verbal Learning Test Revised and Brief
Visuospatial Memory Test Revised: Professional Manual Supplement. Odessa, Fl:
Received: 15 February 2012 Accepted: 22 June 2012 Psychological Assessment Resources, Inc; 2007.
Published: 16 July 2012 28. Parmenter BA, Testa SM, Schretlen DJ, Weinstock-Guttman B, Benedict RHB:
The utility of regression-based norms in interpreting the minimal
References assessment of cognitive function in multiple sclerosis (MACFIMS). J Int
1. Hauser SL, Oksenberg JR, Hauser SL, Oksenberg JR: The neurobiology of Neuropsychol Soc 2010, 16:6–16.
multiple sclerosis: genes, inflammation, and neurodegeneration. Neuron 29. Horsfield MA, Rovaris M, Rocca MA, et al: Whole-brain atrophy in multiple
2006, 52:61–76. sclerosis measured by two segmentation processes from various MRI
2. Trapp BD, Nave KA: Multiple sclerosis: an immune or neurodegenerative sequences. J Neurol Sci 2003, 216:169–177.
disorder? Annu Rev Neurosci 2008, 31:247–269. 30. Cohen J, Cohen P: Applied Multiple Regression/Correlation Analysis for the
3. Rao SM, Leo GJ, Bernardin L, Unverzagt F: Cognitive dysfunction in multiple Behavioral Sciences. 2nd edition. Hillsdale, NJ: Lawrence Erlbaum; 1983.
sclerosis. I. Frequency, patterns, and prediction. Neurology 1991, 41:685–691. 31. Benedict RHB: Effects of using same vs. alternate form memory tests in
4. Benedict RHB, Cookfair D, Gavett R, et al: Validity of the Minimal short-interval, repeated assessment in multiple sclerosis. J Int
Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS). Neuropsychol Soc 2005, 11:727–736.
J Int Neuropsychol Soc 2006, 12:549–558. 32. King MT: A point of minimal important difference (MID): a critique of
5. Patti F, Amato M, Trojano M, et al: Cognitive impairment and its relation terminology and methods. Expert Rev Pharmacoecon Outcomes Res 2011,
with disease measures in mildly disabled patients with relapsing- 11:171–184.
remitting multiple sclerosis: baseline results from the Cognitive 33. Langdon DW, Thompson AJ: Multiple sclerosis: a preliminary study of
Impairment in Multiple Sclerosis (COGIMUS) study. Mult Scler 2009, selected variables affecting rehabilitation outcome. Mult Scler 1999, 5:94–100.
15:779–788. 34. Amato MP, Ponziani G, Pracucci G, Bracco L, Siracusa G, Amaducci L: Cognitive
6. Glanz BI, Holland CM, Gauthier SA, et al: Cognitive dysfunction in patients impairment in early-onset multiple sclerosis: Pattern, predictors, and
with clinically isolated syndromes or newly diagnosed multiple sclerosis. impact on everyday life in a 4-year follow-up. Arch Neurol 1995, 52:168–172.
Mult Scler 2007, 13:1004–1010. 35. Jean VM, Paul RH, Beatty WW: Psychological and neuropsychological
7. Benedict RHB: Integrating cognitive function screening and assessment into predictors of coping patterns by patients with multiple sclerosis. J Clin
the routine care of multiple sclerosis patients.: CNS Spectrums; 2005. in press. Psychol 1999, 55:21–26.
8. Benedict RH, Zivadinov R: Risk factors for and management of cognitive 36. Schultheis MT, Garay E, Millis SR, DeLuca J: Motor vehicle crashes and
dysfunction in multiple sclerosis. Nat Rev Neurol 2011. violations among drivers with multiple sclerosis. Arch Phys Med Rehabil
9. Wechsler D: Wechsler Adult Intelligence Scale. 4th edition. New York: Pearson; 2008. 2002, 83:1175–1178.
10. Lezak M: Neuropsychological Assessment. 2nd edition. New York: Oxford 37. Goverover Y, Kalmar J, Gaudino-Goering E, et al: The relation between
University Press; 1995. subjective and objective measures of everyday life activities in persons
11. Langdon DW, Amato MP, Boringa J, et al: Recommendations for a Brief with multiple sclerosis. Arch Phys Med Rehabil 2005, 86:2303–2308.
International Cognitive Assessment for Multiple Sclerosis (BICAMS). 38. Nortvedt MW, Riise T, Myhr KM, Nyland HI: Quality of life in multiple
Multiple Sclerosis Journal, in press. sclerosis: measuring the disease effects more broadly. Neurology 1999,
12. Rao SM: A Manual for the Brief, Repeatable Battery of Neuropsychological Tests 53:1098–1103.
in Multiple Sclerosis: National Multiple Sclerosis Society; 1991. 39. Goverover Y, Genova HM, Hillary FG, DeLuca J: The relationship between
13. Smith A: Symbol digit modalities test: Manual. Los Angeles: Western neuropsychological measures and the Timed Instrumental Activities of
Psychological Services; 1982. Daily Living task in multiple sclerosis. Mult Scler 2007, 13:636–644.
14. Delis DC, Kramer JH, Kaplan E, Ober BA: California Verbal Learning Test 40. Schultheis MT, Weisser V, Ang J, et al: Examining the relationship between
Manual: Second Edition, Adult Version. San Antonio, TX: Psychological cognition and driving performance in multiple sclerosis. Arch Phys Med
Corporation; 2000. Rehabil 2010, 91:465–473.
Benedict et al. BMC Neurology 2012, 12:55 Page 8 of 8
http://www.biomedcentral.com/1471-2377/12/55

41. Beatty WW, Goretti B, Siracusa G, Zipoli V, Portaccio E, Amato MP: Changes
in neuropsychological test performance over the workday in multiple
sclerosis. Clin Neuropsychol 2003, 17:551–560.
42. Goretti B, Portaccio E, Zipoli V, Razzolini L, Amato MP: Coping strategies,
cognitive impairment, psychological variables and their relationship with
quality of life in multiple sclerosis. Neurol Sci 2010, 31:S227–S230.
43. Rao SM, Leo GJ, Ellington L, Nauertz T, Bernardin L, Unveragt F: Cognitive
dysfunction in multiple sclerosis. II. Impact on employment and social
functioning. Neurology 1991, 41:692–696.
44. LaRocca NG, Kalb RC, Gregg K: A program to facilitate retention of
employment among persons with multiple sclerosis. Work 1996, 7:37–46.
45. Benedict RHB, Wahlig E, Bakshi R, et al: Predicting quality of life in multiple
sclerosis: Accounting for physical disability, fatigue, cognition, mood
disorder, personality, and behavior change. J Neurol Sci 2005, 231:29–34.
46. Benedict RHB, Zivadinov R: Predicting neuropsychological abnormalities
in multiple sclerosis. J Neurol Sci 2006, 245:67–72.
47. Parmenter BA, Zivadinov R, Kerenyi L, et al: Validity of the Wisconsin Card
Sorting and Delis-Kaplan Executive Function System (DKEFS) sorting
tests in multiple sclerosis. J Clin Exp Neuropsychol 2007, 29:215–223.
48. Morrow SA, Drake AS, Zivadinov R, Munschauer F, Weinstock-Guttman B,
Benedict RHB: Predicting loss of employment over three years in multiple
sclerosis: Clinically meaningful cognitive decline. Clin Neuropsychol, . in press.
49. Benedict RHB, Zgaljardic DJ: Practice effects during repeated
administrations of memory tests with and without alternate forms. J Clin
Exp Neuropsychol 1998, 20:339–352.
50. Boringa JB, Lazeron RH, Reuling IE, et al: The brief repeatable battery of
neuropsychological tests: normative values allow application in multiple
sclerosis clinical practice. Mult Scler 2001, 7:263–267.
51. Benedict RH, Smerbeck A, Parikh R, Rodgers J, Cadavid D, Erlanger D: Reliability
and equivalence of alternate forms for the Symbol Digit Modalities Test:
implications for multiple sclerosis clinical trials. Mult Scler 2012, in press.
52. Benedict RHB, Groninger L: Preliminary standardization and validation of
a new visuospatial memory test with six alternate forms. Clin
Neuropsychol 1995, 9:11–16.
53. Benedict RHB, Schretlen D, Groninger L, Dobraski M, Shpritz B: Revision of
the brief visuospatial memory test: studies of normal performance,
reliability, and validity. Psychol Assess 1996, 8:145–153.
54. Green MF, Nuechterlein KH: The MATRICS initiative: developing a
consensus cognitive battery for clinical trials. Schizophr Res 2004, 72:1–3.
55. Rey A: L’Examen Clinique en Psychologie. Paris: Press Universitaire de France; 1964.
56. Benedict RHB, Schretlen D, Brandt J, Groninger L: Revision of the Hopkins
verbal learning test: reliability and normative data. Clin Neuropsychol
1998, 12:43–55.
57. Shapiro AM, Benedict RHB, Schretlen D, Brandt J: Construct and concurrent
validity of the Hopkins Verbal Learning Test - Revised. Clin Neuropsychol
1999, 13:348–358.
58. Trojano M: Multiple sclerosis: the evolving diagnostic criteria for multiple
sclerosis. Nat Rev Neurol 2011, 7:251–252.

doi:10.1186/1471-2377-12-55
Cite this article as: Benedict et al.: Brief International Cognitive
Assessment for MS (BICAMS): international standards for validation. BMC
Neurology 2012 12:55.

Submit your next manuscript to BioMed Central

and take full advantage of:

• Convenient online submission

• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution

Submit your manuscript at

www.biomedcentral.com/submit