0% found this document useful (0 votes)
59 views166 pages

Neuropsychological Practice Evaluating and Managing Noncredible Performance Evidence Based Practice in Neuropsychology 52546094

Uploaded by

mubcovg8982
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
59 views166 pages

Neuropsychological Practice Evaluating and Managing Noncredible Performance Evidence Based Practice in Neuropsychology 52546094

Uploaded by

mubcovg8982
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

(Ebook) Validity Assessment in Clinical

Neuropsychological Practice: Evaluating and Managing


Noncredible Performance (Evidence-Based Practice in
Neuropsychology) by Ryan W. Schroeder (editor), Phillip
K. Martin (editor) ISBN 9781462542499, 1462542492 Pdf
Download

[Link]
neuropsychological-practice-evaluating-and-managing-noncredible-
performance-evidence-based-practice-in-neuropsychology-52546094

★★★★★
4.7 out of 5.0 (37 reviews )

DOWNLOAD PDF

[Link]
(Ebook) Validity Assessment in Clinical Neuropsychological
Practice: Evaluating and Managing Noncredible Performance
(Evidence-Based Practice in Neuropsychology) by Ryan W.
Schroeder (editor), Phillip K. Martin (editor) ISBN
9781462542499, 1462542492 Pdf Download

EBOOK

Available Formats

■ PDF eBook Study Guide Ebook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY


We believe these products will be a great fit for you. Click
the link to download now, or visit [Link]
to discover even more!

(Ebook) Vagabond, Vol. 29 (29) by Inoue, Takehiko ISBN


9781421531489, 1421531488

[Link]

(Ebook) Validity Testing in Child and Adolescent Assessment:


Evaluating Exaggeration, Feigning, and Noncredible Effort by
Michael W. Kirkwood ISBN 9781462521852, 1462521851

[Link]
assessment-evaluating-exaggeration-feigning-and-noncredible-effort-6719176

(Ebook) Studies in Symbolic Interaction, Vol. 29 by Norman K.


Denzin (ed.) ISBN 9780080467474, 9780762313259, 0762313250,
0080467474

[Link]
vol-29-1671488

(Ebook) Boeing B-29 Superfortress ISBN 9780764302725, 0764302728

[Link]
(Ebook) 29, Single and Nigerian by Naijasinglegirl ISBN
9781310004216, 1310004218

[Link]

(Ebook) Advances in Insect Physiology, Vol. 29 by Peter Evans


ISBN 9780120242290, 012024229X

[Link]

(Ebook) Advances in Child Development and Behavior 29 by Robert


V. Kail and Hayne W. Reese (Eds.) ISBN 9780120097296, 012009729X

[Link]
behavior-29-1797872

(Ebook) Jahrbuch für Geschichte: Band 29 ISBN 9783112622223,


3112622227

[Link]

(Ebook) Harrow County 29 by Cullen Bunn, Tyler Crook

[Link]
VALIDITY ASSESSMENT
IN CLINICAL NEUROPSYCHOLOGICAL PRACTICE
Evidence-Based Practice in Neuropsychology
Kyle Brauer Boone, Series Editor

Clinical Practice of Forensic Neuropsychology:


An Evidence-Based Approach
Kyle Brauer Boone

Psychological Assessment:
A Problem-Solving Approach
Julie A. Suhr

Validity Testing in Child and Adolescent Assessment:


Evaluating Exaggeration, Feigning, and Noncredible Effort
Michael W. Kirkwood, Editor

Neuropsychological Report Writing


Jacobus Donders, Editor

Assessment of Feigned Cognitive Impairment:


A Neuropsychological Perspective, Second Edition
Kyle Brauer Boone, Editor

Validity Assessment in Clinical Neuropsychological Practice:


Evaluating and Managing Noncredible Performance
Ryan W. Schroeder and Phillip K. Martin, Editors
VALIDITY ASSESSMENT IN
CLINICAL NEUROPSYCHOLOGICAL
PRACTICE Evaluating and Managing
Noncredible Performance

edited by
Ryan W. Schroeder
Phillip K. Martin

Series Editor’s Note by Kyle Brauer Boone

THE GUILFORD PRESS


New York  London
Copyright © 2022 The Guilford Press
A Division of Guilford Publications, Inc.
370 Seventh Avenue, Suite 1200, New York, NY 10001
[Link]

All rights reserved

No part of this book may be reproduced, translated, stored in a retrieval system,


or transmitted, in any form or by any means, electronic, mechanical, photocopying,
microfilming, recording, or otherwise, without written permission from the publisher.

Printed in the United States of America

This book is printed on acid-free paper.

The authors have checked with sources believed to be reliable in their efforts to provide
information that is complete and generally in accord with the standards of practice that are
accepted at the time of publication. However, in view of the possibility of human error or
changes in behavioral, mental health, or medical sciences, neither the authors, nor the editors
and publisher, nor any other party who has been involved in the preparation or publication
of this work warrants that the information contained herein is in every respect accurate or
complete, and they are not responsible for any errors or omissions or the results obtained from
the use of such information. Readers are encouraged to confirm the information contained in
this book with other sources.

Last digit is print number: 9 8 7 6 5 4 3 2 1

Library of Congress Cataloging-in-Publication Data


Names: Schroeder, Ryan W., editor. | Martin, Phillip K., editor.
Title: Validity assessment in clinical neuropsychological practice :
evaluating and managing noncredible performance / edited by Ryan W.
Schroeder, Phillip K. Martin.
Description: New York : The Guilford Press, [2022]
Identifiers: LCCN 2021029610 | ISBN 9781462542499 (hardcover)
Subjects: LCSH: Neuropsychological tests. | Clinical neuropsychology. |
BISAC: PSYCHOLOGY / Neuropsychology | PSYCHOLOGY / Assessment, Testing &
Measurement
Classification: LCC RC386.6.N48 V35 2022 | DDC 616.8/0475—dc23
LC record available at [Link]
About the Editors

Ryan W. Schroeder, PsyD, ABPP-CN, is a board-certified clinical neuropsychologist


practicing both clinical and forensic neuropsychology. He is a tenured Associate Profes-
sor in the Department of Psychiatry and Behavioral Sciences at the University of Kansas
School of Medicine–Wichita, and Adjunct Faculty in the Clinical Psychology Program
at Wichita State University. Dr. Schroeder has published numerous high-impact journal
articles and book chapters on neuropsychological validity assessment, presented on var-
ied validity assessment issues at multiple national conventions, and served on expert com-
mittees related to topics such as validity assessment and test security within neuropsycho-
logical settings. His scholarly work and professional contributions to the field have been
recognized by an Early Career Award from the National Academy of Neuropsychology,
an Early Career Achievement Award from the American Psychological Association, and
Fellow designation from the National Academy of Neuropsychology.

Phillip K. Martin, PhD, ABPP-CN, is a board-certified clinical neuropsychologist and


Assistant Professor in the Department of Psychiatry and Behavioral Sciences at the Uni-
versity of Kansas School of Medicine–Wichita. Dr. Martin is an active clinician, super­
visor of neuropsychology trainees, teacher, and researcher. His published works target
the topics of validity assessment and neuropsychological evaluation of dementia and
movement disorders. He has made unique contributions to the neuropsychological valid-
ity assessment literature with his scholarly articles pertaining to professional beliefs and
practices, the management of clinical patients who produce invalid test performance,
and meta-analysis of specific validity measures. In addition to his published research, Dr.
Martin’s participation in the evolving discussion of neuropsychological validity assess-
ment practice has included serving as an expert committee member and being a recurrent
national conference speaker.

v
Contributors

Kaley Angers, MS, Department of Psychology, Ohio University, Athens, Ohio


Patrick Armistead‑Jehle, PhD, ABPP, Concussion Clinic, Munson Army Health Center,
Fort Leavenworth, Kansas
Bradley N. Axelrod, PhD, ABN, John D. Dingell Department of Veterans Affairs
Medical Center, Detroit, Michigan
K. Chase Bailey, PhD, ABPP, Department of Psychiatry, UT Southwestern Medical Center,
Dallas, Texas
Yossef S. Ben‑Porath, PhD, ABPP, Department of Psychological Sciences, Kent State University,
Kent, Ohio
Kyle Brauer Boone, PhD, ABPP, Department of Psychiatry, University of California, Los Angeles,
Los Angeles, California, and private practice, Torrance, California
Shane S. Bush, PhD, ABPP, Department of Psychology, University of Alabama,
Tuscaloosa, Alabama, and Long Island Neuropsychology, P.C., Lake Ronkonkoma, New York
Carolyn T. Caldwell, PhD, Neuropsychology Department, Kennedy Krieger Institute,
Baltimore, Maryland
Michael Chafetz, PhD, ABPP, Algiers Neurobehavioral Resource, New Orleans, Louisiana
Brechje Dandachi‑FitzGerald, PhD, Department of Clinical Psychology Science,
Maastricht University, Maastricht, The Netherlands, and Mondriaan Mental Health Institution,
Heerlen, The Netherlands
Jeremy J. Davis, PsyD, ABPP, Department of Neurology, University of Texas Health Science
Center at San Antonio, and The Glenn Biggs Institute for Alzheimer’s and Neurodegenerative
Diseases, San Antonio, Texas
John H. Denning, PhD, Mental Health Service Line, Ralph H. Johnson Veterans Affairs
Medical Center, and Department of Psychiatry and Behavioral Sciences, Medical University
of South Carolina, Charleston, South Carolina
Christopher J. Graver, PhD, ABPP, Department of Neuropsychology, Madigan Army
Medical Center, Tacoma, Washington

vii
viii Contributors

Jordan T. Hall, MA, Department of Psychological Sciences, Kent State University, Kent, Ohio
Kristen M. Klipfel, PhD, Isaac Ray Forensic Group, LLC, Chicago, Illinois
Glenn J. Larrabee, PhD, ABPP, private practice, Sarasota, Florida
Grace J. Lee, MS, Department of Psychology, Ohio University, Athens, Ohio
Sara M. Lippa, PhD, ABPP, National Intrepid Center of Excellence, Walter Reed National
Military Medical Center, and Department of Rehabilitation Medicine, National Institutes
of Health, Bethesda, Maryland
Paul S. Marshall, PhD, ABPP, private practice, Minneapolis, Minnesota
Phillip K. Martin, PhD, ABPP, Department of Psychiatry and Behavioral Sciences,
University of Kansas School of Medicine–Wichita, Wichita, Kansas
Kristian P. Nitsch, PhD, Shepherd Pathways, Decatur, Georgia
Howard J. Oakes, PsyD, ABPP, Ayer Neuroscience Center, Hartford Hospital,
Hartford, Connecticut
Caleb P. Peck, PsyD, ABPP, Clarus Health Alliance, Norwich, Connecticut
Danielle M. Ploetz, PhD, ABPP, Neuropsychology Department, Fairmount Rehabilitation
Programs, Kennedy Krieger Institute, Baltimore, Maryland
Ryan W. Schroeder, PsyD, ABPP, Department of Psychiatry and Behavioral Sciences,
University of Kansas School of Medicine–Wichita, Wichita, Kansas
Robert D. Shura, PsyD, ABPP, Mid‑Atlantic Mental Illness Research, Education,
and Clinical Center, Salisbury VA Medical Center, Salisbury, North Carolina, and Department
of Neurology, Wake Forest School of Medicine, Winston‑Salem, North Carolina
Jason R. Soble, PhD, ABPP, Department of Psychiatry and Department of Neurology,
University of Illinois College of Medicine, Chicago, Illinois
Jennifer M. Stinson, PhD, ABPP, Department of Neurology, Baylor College of Medicine,
Houston, Texas
Adriana M. Strutt, PhD, ABPP, Department of Neurology, Baylor College of Medicine,
Houston, Texas
Julie A. Suhr, PhD, Department of Psychology, Ohio University, Athens, Ohio
Jerry J. Sweet, PhD, ABPP, Department of Psychiatry and Behavioral Science, NorthShore
University HealthSystem, Evanston, Illinois
Troy A. Webber, PhD, Mental Health Care Line, Michael E. DeBakey VA Medical Center,
Houston, Texas
Series Editor’s Note

The editors of this book, neuropsychologists Ryan Schroeder and Phillip Martin, faced
an interesting conundrum when tackling this project, one not typically encountered while
producing a clinical handbook. Usually, potential book buyers can readily appreciate
how a publication might inform and improve their knowledge and skills from reading
book descriptions, and as a result, they need little enticement to add the book to their
libraries. However, some neuropsychologists in clinical settings might view the current
book as irrelevant to their practices, in that they might not routinely perceive questions of
performance validity as present in their patients. Conversely, forensic neuropsychologists,
while regularly encountering questions regarding performance validity in their cases,
might be tempted to dismiss a book written for clinical nonforensic settings as not perti-
nent to their retained expert roles. Both views could not be further from the truth.
If clinicians have any doubts about the importance of the performance validity test
(PVT) in clinical contexts prior to reading this book, they will come away from this
volume convinced. Schroeder and Martin provide compelling information regarding the
not-­inconsequential rates of performance invalidity in clinical settings (e.g., approxi-
mately 15%) and the large percentage of clinical patients who do not divulge the presence
of external incentives for poor neurocognitive performance to treaters. This means that
neuropsychologists in clinical settings likely encounter questions of performance validity
more often than they estimate. If clinicians are incorrect in their conclusions regarding
the presence of cognitive disability in one in eight assessments, this relatively high error
rate can be expected to cause substantial negative repercussions to large numbers of
patients and families, medical care systems, and society in general. As detailed by the
chapter authors, if patients are judged to have cognitive conditions they in fact do not
have, they can lose driving privileges and the ability to live independently and manage
finances. Erroneous neurocognitive diagnoses can lead to iatrogenesis, in which medical
attention is directed to symptoms/conditions that are not present and locks in a false view
that the patient is dysfunctional, often leading to treatments with poor outcome and/or
harmful side effects.
For forensic neuropsychologists, this book contains highly relevant research summa-
ries that can be readily imported for use in forensic evaluations (e.g., while reading this
book in preparation for writing the foreword, I stumbled upon the exact research citation

ix
x Series Editor’s Note

needed for a current forensic case and of which I was not previously aware!). Informa-
tion contained within this book can assist the forensic expert in effectively addressing
attacks that PVT failures in plaintiffs are due to nonincentive factors, such as psychiatric
conditions, pain, medications, fatigue, apathy, diagnosis threat, cogniphobia, and “cry
for help,” hopefully finally putting to rest the latter, poorly considered, explanation for
noncredible symptoms.
The authors concisely detail the unique responsibilities required in clinical validity
assessments, which differ from roles in the forensic arena. They address the delicate issue
of providing feedback on nonplausible neuropsychological test performance to patients
and families, a skill not required in forensic settings, in which experts are consultants
to attorneys and do not furnish test results to test takers. The authors also carefully
elucidate the various potential feedback options when performance invalidity is present
in a clinical setting, and the pros and cons of each. They make the case for a nuanced
approach in which the clinician does not collude in validating a false presentation of cog-
nitive symptoms, but does adopt an empathic and respectful stance that avoids focus on
the causes of invalidity and instead opens the door to exploration of psychosocial issues
that may be provoking noncredible neurocognitive performance. Report writing is also
thoughtfully addressed, with examples of writing style and terminology that are accurate
and objective, while maintaining a nonaccusatory tone. Ethical considerations, includ-
ing discussion of informed consent and concerns regarding disclosure of the presence of
PVTs within a test battery, are explored.
Chapters summarize currently available PVT methods, along with concise explana-
tions of classification statistics needed for test interpretation. The conflicting literature
on whether use of increasing numbers of PVTs is associated with increased false-­positive
rates is summarized and critiqued, and recommendations are provided as to how to avoid
false-­positive identification of invalid test performance.
Other chapters cover such issues as the use of PVTs with ethnic minority and non-­
native English speakers; veterans and military personnel; children and adolescents;
patients with psychiatric conditions, including somatic symptom and related disorders;
patients with mild traumatic brain injury; and patients in memory disorders clinics, in
rehabilitation settings, and undergoing academic evaluations. Psychological underpin-
nings of postconcussive syndrome symptoms are addressed, and information is provided
on how to prevent postconcussive syndrome from developing, as well as treatment meth-
ods once it does emerge. Helpful descriptions of the Social Security disability and Work-
ers Compensation systems are included, along with discussion of the role of PVTs in these
settings. The relationships between MMPI-­2-­R F and PVT data are summarized, which
can be applied to the MMPI-­3, given the high correlations between scales on the two per-
sonality inventory instruments. The book concludes with a chapter that offers a roadmap
for clinical neuropsychologists regarding terms and procedures within the forensic arena,
should they find themselves invited to participate.
In conclusion, in this practical and comprehensive book, the editors and chapter
authors have clearly met their goal of providing guidelines for “ethical, efficient, and
accurate validity assessments” within clinical/nonforensic neuropsychological settings.

Kyle Brauer Boone, PhD


The University of California, Los Angeles
Preface

Over the last two decades, neuropsychologists saw a dramatic expansion of the litera-
ture base related to neuropsychological validity assessment. A multitude of peer-­reviewed
journal articles was published, and excellent books on forensic assessment and malinger-
ing of cognitive dysfunction became available. Books authored or edited by Kyle Boone,
Glenn Larrabee, and Dominic Carone and Shane Bush, in particular, shaped our own
thinking on neuropsychological validity assessment. In applying concepts from these and
other largely forensic texts to our clinical patients, we recurrently recognized the salient
differences between forensic and clinical practice. We also became increasingly cogni-
zant that these differences carried important implications when attempting to generalize
validity assessment practices from forensic settings to clinical settings. Given this, and
given the relative lack of literature addressing validity assessment in nonforensic settings,
we became interested in understanding how other clinicians manage invalidity in their
clinical patients. As a result, we conducted surveys with both national samples of neuro-
psychologists and a select group of neuropsychological validity testing experts, exploring
their beliefs and practices regarding validity-­related practice issues in both clinical and
forensic settings. The survey results confirmed our suspicions that (1) most neuropsychol-
ogists frequently utilized and relied on validity assessment in their forensic and clinical
evaluations, but (2) contrary to forensic contexts, consensus about managing invalidity
in clinical contexts was generally lacking.
In the ensuing years since publishing these surveys, we have continued to conduct
our own research of validity assessment practices in clinical nonforensic settings, and we
have connected with other experts who also seek to further advance such practices in this
milieu. In this book, we, along with our skilled colleagues, who are among the foremost
experts on the topics in their respective chapters, discuss nuanced validity assessment
issues and apply them to nonforensic settings. Part I of the book explores validity assess-
ment phenomena and challenges unique to the nonforensic clinical setting, providing
direction on how to manage clinical patients when they invalidate testing. Conceptual
and empirical data, practical techniques and recommendations, case illustrations, and
sample reports are all provided to augment clinical learning. Part II of the book examines
clinical and methodological factors requiring consideration before, during, and after the
evaluation. Within these chapters, information on potential biases, appropriate validity

xi
xii Preface

test cutoffs, application of validity test cutoffs to ethnic minorities and non-­native Eng-
lish speakers, and methods to synthesize data to reach firm empirical conclusions are
discussed. Part III of the book provides guidance for addressing invalidity in specific
settings, contexts, and populations in order to facilitate evidence-­based validity assess-
ment practice for the individual patient. The section concludes with a “hands-on” guide
describing how to appropriately engage in legal proceedings in instances where clinical
cases turn forensic in nature. Our hope is that this book will help guide the majority of
clinicians who utilize validity assessment in their nonforensic evaluations but who, like
us, recognize that some forensic concepts cannot be easily adapted to clinical settings.
In closing, we would like to acknowledge individuals whom we are very grateful to
have in our professional lives. Ryan would like to thank everyone who has contributed to
his professional training as a psychologist and neuropsychologist. He would also like to
thank Paul Marshall, Lyle Baade, Kyle Boone, Glenn Larrabee, Jerry Sweet, and Robert
Heilbronner, who heavily shaped his conceptualization of validity issues and provided
additional opportunities to increase his engagement in this particular area of scholarly
interest. Phillip would like to thank Laurence Levine and Ryan Schroeder for helping to
foster both his interest in and conceptualization of validity assessment issues in clinical
practice, as well as Don Morgan, Charles Golden, and Lyle Baade for their training, men-
torship, and professional guidance. Both Ryan and Phillip would also like to thank Kyle
Boone for her great generosity in all that she has done for us, including recommending us
to Rochelle Serwator, Senior Editor at The Guilford Press. We would also like to thank
Rochelle for her guidance and support in creating this book. Finally, we would like to
thank all of the chapter authors whose work appears in this book for their eager willing-
ness to devote days’ worth of their time to generously contribute to the knowledge and
guidance that is contained here.
Contents

PART I. Neuropsychological Validity Assessment:


Important Distinctions for Nonforensic Practice

1. Validity Assessment in Clinical Settings: 3


How It Differs from Forensic Settings and Why It Is Important
Ryan W. Schroeder and Phillip K. Martin

2. Explanations of Performance Validity Test Failure 11


in Clinical Settings
Ryan W. Schroeder and Phillip K. Martin

3. Malingering: Maintaining a Perspective of Clinical Care 31


When Patients Feign
Phillip K. Martin and Ryan W. Schroeder

4. A Framework for Providing Clinical Feedback 47


When Patients Invalidate Testing
Phillip K. Martin and Ryan W. Schroeder

5. Incorporation of Validity Assessment and Validity Findings 70


When Writing Clinical Reports
Ryan W. Schroeder and Phillip K. Martin

6. Ethical Considerations Associated with Validity Assessment 86


in Clinical Evaluations
Phillip K. Martin, Ryan W. Schroeder, Kyle Brauer Boone,
and Shane S. Bush

xiii
xiv Contents

PART II. Clinical and Methodological Considerations


Throughout the Evaluation

7. Clinical Judgment and Clinically Applied Statistics: 107


Description, Benefits, and Potential Dangers When Relying
on Either One Individually in Clinical Practice
Brechje Dandachi‑FitzGerald and Phillip K. Martin

8. An Overview of Common Performance Validity Tests 126


for Practicing Clinicians: Cutoffs, Classification Accuracy,
and Administration Times
Jason R. Soble, Troy A. Webber, and K. Chase Bailey

9. The MMPI‑2‑RF Validity Scales: An Overview of Research 150


and Applications
Jordan T. Hall and Yossef S. Ben‑Porath

10. The Nuts and Bolts of Incorporating Validity Assessment 179


in Clinical Evaluations
Sara M. Lippa

11. Synthesizing Data to Reach Clinical Conclusions 193


Regarding Validity Status
Glenn J. Larrabee

12. Performance Validity Testing with Culturally Diverse Individuals 211


and Non‑Native English Speakers: The Need for a Cultural Perspective
in Neuropsychological Practice
Adriana M. Strutt and Jennifer M. Stinson

PART III. Validity Assessment in Specific Clinical Settings and Contexts

13. Validity Assessment within Veterans Affairs and Active Duty Settings 235
Robert D. Shura, John H. Denning, Patrick Armistead‑Jehle,
and Bradley N. Axelrod

14. Validity Assessment within the Memory Disorders/Dementia Clinic 254


Ryan W. Schroeder and Phillip K. Martin

15. Validity Assessment within the Rehabilitation Setting 280


Jeremy J. Davis
Contents xv

16. Validity Assessment in Academic Evaluations 300


Julie A. Suhr, Grace J. Lee, and Kaley Angers

17. Validity Assessment in Patients with Psychiatric Disorders 319


Paul S. Marshall and Ryan W. Schroeder

18. Validity Assessment in Patients with Mild Traumatic Brain Injury 338
Ryan W. Schroeder, Phillip K. Martin, and Glenn J. Larrabee

19. Validity Assessment in Patients with Somatic Symptom 357


and Related Disorders
Christopher J. Graver and Kyle Brauer Boone

20. Validity Assessment for Clinical Patients 378


Pursuing Social Security Disability
Ryan W. Schroeder and Michael Chafetz

21. Validity Assessment for Clinical Patients 400


with Workers’ Compensation Claims
Caleb P. Peck and Howard J. Oakes

22. Validity Assessment with Children and Adolescents: 418


Comparisons to Adult Evaluations
Danielle M. Ploetz, Carolyn T. Caldwell, and Kristian P. Nitsch

23. Forensic Primer for the Nonforensic Neuropsychologist: 439


When Clinicians Participate in Forensic Proceedings
Jerry J. Sweet and Kristen M. Klipfel

Index 463
VALIDITY ASSESSMENT
IN CLINICAL NEUROPSYCHOLOGICAL PRACTICE
PA R T I

Neuropsychological
Validity Assessment
Important Distinctions for Nonforensic Practice
CHAPTER 1

Validity Assessment in Clinical Settings


How It Differs from Forensic Settings and Why It Is Important

Ryan W. Schroeder and Phillip K. Martin

VALIDITY ASSESSMENT: GROWING FROM FORENSIC TO CLINICAL SETTINGS


Historically, it was believed that nearly all individuals who underwent neuropsycho-
logical testing for clinical purposes produced valid data (Green & Merten, 2013; Mit-
tenberg, Patton, Canyock, & Condit, 2002). This was likely because clinicians held a
perspective that essentially all clinical patients were motivated to receive accurate and
clinically helpful information regarding their health care questions. In contrast, it was
known that individuals who underwent neuropsychological testing for forensic purposes
had clear external incentives to malinger (e.g., obtain financial compensation or avoid
criminal responsibility). As a result, it was thought that production of invalid data on
neuropsychological testing was a phenomenon that primarily occurred in forensic evalu-
ations (Merten et al., 2013; ­M ittenberg et al., 2002). Correspondingly, much of the early
research on neuropsychological validity assessment was conducted in forensic settings,
by forensic practitioners, and for forensic practitioners (Suchy, 2019). Given the forensic
nature of this research, invalidity was largely considered synonymous with malingering,
validity tests were sometimes called “malingering tests,” and the most commonly used
performance validity test (PVT) in North America was even named the Test of Memory
Malingering (TOMM; Martin, Schroeder, & Odland, 2015; Mittenberg et al., 2002;
Nies & Sweet, 1994; Slick, Tan, Strauss, & Hultsch, 2004).
Over time, research on neuropsychological validity assessment began to increase
in both volume and scope (Martin et al., 2015; Suchy, 2019). As noted by Martin et al.
(2015), in the mid-1990s, roughly 7% of articles published in two commonly referenced
neuropsychology journals, Archives of Clinical Neuropsychology and The Clinical Neu-
ropsychologist, addressed topics regarding neuropsychological validity assessment. By
the mid-2010s, however, roughly 25% of articles investigated topics related to neuropsy-
chological validity assessment (see Figure 1.1). With this expansion in empirical inves-
tigation, the focus of the research began to shift from detecting malingering in forensic
settings to understanding the effects of performance invalidity more generally, whether
in forensic or clinical settings (Suchy, 2019).

3
4 N e u r o p s y c h o l o g i c a l Va l i d i t y A s s e s s m e n t

30%

25%

20%

15%

10%

5%

0%

FIGURE 1.1. Average proportion of articles published on neuropsychological validity assessment


by year in The Clinical Neuropsychologist and Archives of Clinical Neuropsychology. Data from
Martin, Schroeder, and Odland (2015).

In a seminal study, Mittenberg et al. (2002) documented that invalidity occurred in


clinical nonforensic settings, albeit less frequently than in forensic settings. Green, Roh-
ling, Lees-Haley, and Allen (2001) found that when patients completed testing invalidly,
the invalidity significantly impacted cognitive test performance, as it explained roughly
50% of the variance in cognitive test scores. This was notably more variance than that
accounted for by age (4%), education (11%), Glasgow Coma Scale score (1%), degree of
posttraumatic amnesia (1%), and presence of positive neuroimaging findings (<1%). The
utility of formal approaches to detect invalidity also became abundantly evident. For
example, Larrabee (2003) found that accurate identification of invalidity via concurrent
use of multiple validity tests far exceeded the classification accuracy rates achieved by
use of clinical judgment alone (see Faust, Hart, & Guilmette, 1988a; Faust, Hart, Guil-
mette, & Arkes, 1988b; Heaton, Smith, Lehman, & Vogt, 1978). Similarly, Meyers and
Volbrecht (2003) demonstrated that use of multiple validity tests resulted in strong clas-
sification accuracy rates, even in nonforensic clinical samples.
As a result of the expanded literature base, it became clear that validity assessment
should be considered a critical and core component of all neuropsychological evalua-
tions, a perspective that was adopted and documented by several well-cited professional
position papers. With regard to incorporating validity testing in clinical evaluations, spe-
cifically, the National Academy of Neuropsychology released a validity testing position
paper in 2005, emphasizing that “adequate assessment of response validity is essential in
order to maximize confidence in the results of neurocognitive and personality measures
and in the diagnoses and recommendations that are based on the results” (Bush et al.,
2005, p. 419). The position paper further elaborated that “assessment of response valid-
ity, as a component of a medically necessary evaluation, is medically necessary” (p. 419).
Validity Assessment in Clinical Settings 5

Two years later, the American Academy of Clinical Neuropsychology (AACN) released
practice guidelines for neuropsychological assessment and consultation (Board of Direc-
tors, 2007). Within these practice guidelines, it was specified that “the assessment of
effort and motivation is important in any clinical setting, as a patient’s effort may be
compromised even in the absence of any potential or active litigation, compensation,
or financial incentives” (p. 221). In 2009, the AACN released a consensus statement
focused on the use of neuropsychological validity testing (Heilbronner et al., 2009) and
stated that “response bias may occur in routine clinical and medical referrals, when no
forensic context is evident” and “when clinicians are evaluating a (nonforensic) patient
who by virtue of claimed injuries is reasonably likely to become a litigant or claimant,
the clinician should consider the increased risk of insufficient effort and response bias”
(pp. 1105–1106).
Organizations outside of neuropsychology also began to emphasize the need for
validity assessment in neuropsychological evaluations. For example, the American Medi-
cal Association’s guides to the evaluation of permanent impairment (American Medical
Association, 2008) stated that “it is standard practice that a neuropsychological test bat-
tery should include instruments that include . . . validity tests” (p. 351). The Institute of
Medicine of the National Academies (Institute of Medicine, 2015), a nonprofit institution
that provides independent objective analysis to inform public policy decisions, declared
that “it is important to include an assessment of performance validity at the time cogni-
tive testing is administered” (p. 202). It was further stated that “all cognitive evaluations
should include a statement of evidence of the validity of the results” (p. 203).
By virtue of the robust literature base and ensuing organizational practice recom-
mendations, most neuropsychologists are now aware that invalid test performance and
symptom report can occur in many contexts, even within routine clinical contexts in
which external incentives to underperform are absent (Martin et al., 2015; Martin &
Schroeder, 2020; Sweet, Benson, Nelson, & Moberg, 2015). This is a particularly impor-
tant concept to appreciate given that neuropsychologists evaluate cognitive and emo-
tional statuses (Board of Directors, 2007), aspects of human functioning that cannot be
quantified by methods such as laboratory values, neuroimaging findings, or electrophysi-
ological procedures—­tests that are relatively impervious to patient behavior (Schroeder,
Martin, & Walling, 2019). Assessment of cognitive and emotional capabilities relies heav-
ily on patient engagement, motivation, and cooperation (Bianchini, Mathias, & Greve,
2001). Thus, if a patient provides suboptimal engagement, motivation, or cooperation,
or produces test data or symptom report with the intent to deceive, his or her cognitive
and emotional functioning cannot be accurately captured. If a clinical patient’s inac-
curate results are erroneously identified as being accurate, the neuropsychologist could
inadvertently harm the patient by (1) providing inaccurate and emotionally impactful
diagnoses (e.g., telling a patient that he or she has significant persistent cognitive deficits
or even a neurodegenerative condition), (2) reinforcing noncredible symptoms, (3) recom-
mending unnecessary and potentially costly additional workup (e.g., magnetic resonance
imaging [MRI] of the brain), (4) recommending unnecessary and potentially harmful
treatment (e.g., unneeded medication prescription), and/or (5) recommending restrictions
to daily living activities and/or independence (e.g., stopping driving or moving from inde-
pendent living to assisted/sheltered living). As demonstrated by this historical review,
neuropsychological validity assessment has clearly evolved over time, and neuropsycho-
logical validity tests are no longer viewed as simply being tools for detecting malinger-
ing, primarily in forensic settings. Rather, validity assessment is now viewed as a means
to ensure accuracy of neuropsychological test data regardless of cause of invalidity and
6 N e u r o p s y c h o l o g i c a l Va l i d i t y A s s e s s m e n t

clinical setting, a sentiment that is reinforced by the updated AACN validity assessment
consensus statement (Sweet et al., 2021).

DIFFERENCES IN VALIDITY ASSESSMENT


BETWEEN CLINICAL AND FORENSIC EVALUATIONS
As previously described, it is now clear that validity assessment should be incorporated
within all neuropsychological evaluations, including clinical nonforensic evaluations.
On the surface, addressing validity issues in nonforensic evaluations might seem to be a
straightforward task given that a significant amount of literature has amassed on validity
assessment in forensic settings, and the basic elements of a neuropsychological evaluation
are largely the same in clinical and forensic evaluations (see Table 1.1). When attempting
to apply the large forensic literature base to clinical settings, however, it becomes obvious
that there are differences in assessing validity status, interpreting and documenting valid-
ity test results, and providing feedback to others when validity tests are failed. In order to
understand the differences in validity assessment that are related to practice setting, it is
imperative that neuropsychologists be aware of the core distinctions between clinical and
forensic evaluations, a topic that we discuss next.
As there is not yet a consensus-­achieved, formal definition of what constitutes a
clinical (as opposed to forensic) neuropsychological evaluation, we provide the follow-
ing operational definition of a clinical neuropsychological evaluation, which is based on
previously published descriptions of services provided (i.e., Binder, 2019; Donders, 2016;
Sweet, Kaufmann, Ecklund-­Johnson, & Malina, 2018). We define a clinical neuropsy-
chological evaluation as an evaluation where a neuropsychologist provides health care
services to a patient who is seeking treatment for a malady. In providing this evaluation,
the neuropsychologist is acting as a treating doctor (a term applied regardless of whether
assessment or intervention is directly provided) and entering into a patient–­doctor rela-
tionship.
To highlight the distinguishing features of the clinical neuropsychological evalua-
tion, we unpackage the aforementioned definition. First, because a neuropsychologist

TABLE 1.1. Basic Elements of Neuropsychological Evaluations


Clinical evaluations Forensic evaluations

Accept referral Yes Yes


Review records Yes Yes
Obtain informed consent Yes Yes
Conduct clinical interview Yes Yes
Complete testing Yes Yes
Interpret data Yes Yes
Reach conclusions Yes Yes
Write report Yes Yes
Provide feedback to patient Yes No
Validity Assessment in Clinical Settings 7

provides health care services to a patient, that neuropsychologist is operating within a


health care system. As such, other health care providers often serve as the referral source,
referring individuals with diverse clinical conditions, including significantly impactful
conditions such as dementia (Sweet et al., 2015). The neuropsychologist evaluates and/or
treats these patients, typically billing medical insurance for rendered services (Donders,
2016; Sweet et al., 2018). By billing medical insurance, the neuropsychologist agrees to
the stipulations set forth by insurance companies, which includes time-limit restrictions
on how much testing can be completed (Lamberty, 2012). Accordingly, survey data show
that reimbursement factors and evaluation context (i.e., clinical or forensic evaluation)
are both cited by neuropsychologists as factors that impact length of the neuropsycho-
logical evaluation (Sweet et al., 2015).
Second, because neuropsychologists conducting clinical evaluations are acting as a
treating doctor by entering into a patient–­doctor relationship, it is expected that the
exam is designed to provide information that will clinically benefit the patient (Binder,
2019). The neuropsychologist typically provides a clinical opinion to the patient (Binder,
2019), acts as an advocate for the patient (Donders, 2016), and attempts to minimize
harm to the patient (Binder, 2019). Additionally, within clinical evaluations, it is gen-
erally understood that confidentiality is protected except under special circumstances
(Donders, 2016).
Whereas operating as a treating doctor and entering into a patient–­doctor relation-
ship are key features of the clinical neuropsychological evaluation, such characteristics
are incongruent with accepted definitions and principles of forensic practice. According
to the American Psychological Association (2013), a forensic evaluation is one in which
the psychologist applies scientific, technical, or specialized knowledge of psychology to
the law to assist in addressing legal, contractual, and/or administrative matters. Stated
more simply, a forensic neuropsychological evaluation is one in which neuropsycholo-
gists apply neuropsychological knowledge and facts to answer legal questions (Larrabee,
2012; Greiffenstein & Kaufmann, 2018).
The aforementioned definition of a forensic neuropsychological evaluation can
be unpackaged to show how it differs from a clinical neuropsychological evaluation.
Because forensic neuropsychologists apply knowledge to the law to answer legal ques-
tions, attorneys or administrative professionals often serve as the referral source (Sweet
et al., 2018). Forensically referred examinees often have relatively restricted causes of
their cognitive complaints, with trauma-­based conditions, such as traumatic brain injury,
being frequently claimed (Sweet et al., 2015). Because the goal of the evaluation is to
answer legal questions, the neuropsychologist is expected to provide an impartial impres-
sion in which considerations of accuracy clearly trump those of examinee well-being
(Sweet et al., 2018).
Because, within a forensic evaluation, the neuropsychologist does not practice within
a typical health care system or provide health care services to the examinee, the neu-
ropsychologist does not form a doctor–­patient relationship with the examinee (Binder,
2019). Additionally, the neuropsychologist does not bill medical insurance; therefore,
the neuropsychologist is not confined to the limits imposed by insurance reimbursement
policies (Lamberty, 2012). Furthermore, the neuropsychologist has no obligation to help
the examinee and is not an advocate for the examinee; instead, he or she is an advocate
for the truth (Binder, 2019). Because it is known that the “truth” is often exaggerated
or feigned within forensic settings (Larrabee, Millis, & Meyers, 2009; Mittenberg et al.,
2002), the neuropsychologist might be inclined to view the examinee more skeptically
than in clinical settings (Bush & Heilbronner, 2012). Finally, because healthcare laws are
8 N e u r o p s y c h o l o g i c a l Va l i d i t y A s s e s s m e n t

not pertinent in forensic evaluations and the results of the evaluation will become part of
a public record, there is not the same legal mandate to protect an examinee’s confidential-
ity as there is within the clinical evaluation (Sweet et al., 2018).
The key differences that distinguish clinical from forensic neuropsychological evalu-
ations (as just described and as summarized in Table 1.2) translate into important dif-
ferences in validity assessment practices, which are described in Table 1.3. In addition
to recognizing these individual differences, one should also recognize that clinical and
forensic neuropsychologists are likely to proceed with the evaluation in different man-
ners given the different factors/goals inherent within the evaluations. Specifically, while
both types of neuropsychologists should recognize that the consequence of validity test
failure is the same (i.e., do not interpret neuropsychological test data as accurate), the
resultant interpretive question will likely be different. In forensic settings, the question
is often “What does this say about the examinee and his or her claims?” In clinical set-
tings, however, the operative question is likely more appropriately “How can I provide
useful clinical services when I have invalid test data?” Given the many differences in
validity assessment practices and interpretive questions, it should be understood that it is
not always possible or appropriate to generalize research findings, practice recommenda-
tions, or guidelines from forensic literature to practice in the clinical nonforensic setting.
Because of issues in generalizing forensic literature to nonforensic cases, and because
there is limited guidance in addressing methods and approaches to validity assessment
in clinical settings, managing invalidity in nonforensic settings is arguably more chal-
lenging than managing invalidity in forensic settings. In this book, therefore, we aim to
thoroughly examine the validity assessment literature through the lens of clinical practice
in order to provide guidance and resources for utilizing and addressing validity assess-
ment in nonforensic settings. Chapters 6–12 provide resources on how to conduct ethi-
cal, efficient, and accurate validity assessments in clinical settings. Guidance on how to
move forward when PVTs are failed, including ways to conceptualize invalidity in clinical
settings (Chapters 2–4), provide feedback when testing is invalid (Chapter 4), and write
clinically useful reports when testing is invalid (Chapter 5) are also provided. Discussions
of validity assessment in specific clinical settings and contexts are also provided in the

TABLE 1.2. Key Factors Differentiating Clinical and Forensic Neuropsychological Evaluations
Clinical evaluations Forensic evaluations

Doctor–patient relationship No doctor–patient relationship

Address health care questions Address legal questions

Advocate for patient Advocate for truth

Goal: objectivity, accuracy, and patient well-being Goal: objectivity and accuracy

Notable limits on amount of time to conduct Minimal limits on amount of time to conduct
evaluations evaluation

Invalidity is often less common Invalidity is often more common

Patient obtains results Third parties obtain results

Confidentiality strongly protected Less confidentiality


Validity Assessment in Clinical Settings 9

TABLE 1.3. Examples of How Key Differences between Clinical and Forensic Evaluations
Impact Validity Assessment
1. Time to review records to ensure accurate report of history and symptoms is less in clinical settings.
2. Clinical patients present with highly diverse diagnostic conditions, including dementia, which can
impact validity assessment.
3. Time-efficient validity assessment is more necessary in clinical settings than in forensic settings.
4. Differing base rates of invalidity impact interpretation of PVTs via influence of positive and
negative predictive power.
5. Base rates of causes of invalidity in each setting can impact conclusions drawn regarding reasons
for invalid test findings.
6. There is a need to balance validity considerations with treatment recommendations in clinical
settings.
7. Report-writing goals and/or styles vary based on differences in time limits and expectations
inherent in each setting.
8. Feedback is provided to the patient in clinical settings versus a third party in forensic settings.

last section of this book (Chapters 13–22). Finally, Chapter 23 includes information on
conceptualizing legal matters and responding to queries when clinical cases turn forensic
in nature.

REFERENCES

American Medical Association. (2008). Guides H. (2005). Symptom validity assessment: Prac-
to the evaluation of permanent impairment, tice issues and medical necessity NAN policy
sixth edition. Chicago: American Medical and planning committee. Archives of Clinical
Association. Neuropsychology, 20(4), 419–426.
American Psychological Association. (2013). Donders, J. (2016). Neuropsychological report
Specialty guidelines for forensic psychology. writing. New York: Guilford Press.
American Psychologist, 68(1), 7–19. Faust, D., Hart, K., & Guilmette, T. J. (1988a).
Bianchini, K. J., Mathias, C. W., & Greve, K. Pediatric malingering: The capacity of children
W. (2001). Symptom validity testing: A criti- to fake believable deficits on neuropsychologi-
cal review. The Clinical Neuropsychologist, cal testing. Journal of Consulting and Clinical
15(1), 19–45. Psychology, 56, 578–582.
Binder, L. M. (2019). The patient–psychologist Faust, D., Hart, K. J., Guilmette, T. J., & Arkes,
relationship and informed consent in neuro- H. R. (1988b). Neuropsychologists’ capacity
psychological evaluations. The Clinical Neu- to detect adolescent malingerers. Professional
ropsychologist, 33(6), 988–1015. Psychology: Research and Practice, 19(5),
Board of Directors. (2007). American Academy 508–515.
of Clinical Neuropsychology (AACN) practice Green, P., & Merten, T. (2013). Noncredible
guidelines for neuropsychological assessment explanations of noncredible performance on
and consultation. The Clinical Neuropsychol- symptom validity tests. In D. A. Carone & S.
ogist, 21(2), 209–231. S. Bush (Eds.), Mild traumatic brain injury:
Bush, S. S., & Heilbronner, R. L. (2012). The Symptom validity assessment and malingering
neuropsychological IME. In S. S. Bush & G. (pp. 73–100). New York: Springer.
L. Iverson (Eds.), Neuropsychological assess- Green, P., Rohling, M. L., Lees-Haley, P. R., &
ment of work-related injuries (pp. 857–886). Allen, L. M., III. (2001). Effort has a greater
New York: Guilford Press. effect on test scores than severe brain injury in
Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. compensation claimants. Brain Injury, 15(12),
T., Koffler, S. P., Pliskin, N. H., . . . Silver, C. 1045–1060.
Other documents randomly have
different content
PECIES

in 180

curve siimehet

lyrics

in particular Barrow

her head for

xp the

one front

by contrasting Ja

functions wings to
other

up

against

the Formerly

Maahan

Miss when on

a fight dam

seen
thy ottanut teen

him

designating nuorin

Owen subspecies such

same to refraction

of

back much so

argus and

of
opinion

knoledge at

Baker Nesbitt expressed

without the

heart
ridden

it securely over

hunter

to Bust

as of States

is a

that before

brought

on B of
digits from Project

ihanan

TWO

my things

had is gum

from the

have Dieu

karttuvan growing 1854

courses glowing former


and

cellar qualified

hirelings

the the the

of of brother
may

it fallen asleep

length scene Innsbruck

niin shallow mm

by Nomencl

Foundation a that

though firm Charles

Hurrah other before


at allso rumour

the little felling

2 the

preached Casuarius

function district XIX

and pilot
was by of

11 white takes

entrusted to test

the besieged Italian

capita use of

of of 1

head that

that reservations
charge some

naturally

and

and collection

in

becomes you

wit

the like

was under of
On Öinen Gilline

Brazos On

West epithet

minnows

country

of

as

ground

continued

s
Dam

least that Knoeke

far content Aboute

window

line follows trans

as

the white for

no seven DIE

without ILITARY

which replied the


the that THE

will the Comparisons

please

1890

active
battle

Näetkö

began

of of

God
with pointing

to the her

battle spinifer in

humina KU

numbers

Tis the
name asper

from

has OTHER

it Brothers infant

Kut

classification

IV

giant vary

oksiakaan

terms letter
theorem

This all applying

17 the

which LOANE Author

as

of Q

older was

There
concentric evolves

Then end

paraita in

in to vision

the

mists
about

variety

and

bloody

which

shape

you Clink tracts


his

a disgust friends

TRADEMARK

and black you

M the

U with Woodpigeon

accompanied

and attesting

treatise gallows I
of and any

in Memoirs

position for summoner

our supervision luck

varies Broad on
4 she than

List Ulenspiegel and

coefficients

be

have But vermin

and good lump

to

few
military to syytä

philosopher mm prisoner

all it

liability

recent been
the and

Joll There

Mr she

at and

Indian

particular SIX
Amer

its early the

to spots

for trembling

produced

that

discourse him
birds imagine

from away lately

Ingres his engaged

he

that owns among

refused

pause to of

Quebec oval
G mass

is 1

pointed humilis full

metatarsi in on

to late worse

him Project specimens


in

intersect in

veljellensä people

methods

made

individuals

said of and
U exercise the

in Obion des

and NFINITESIMAL Grey

to upon

top
I

sailed comforts

coat Ne

obtain sex however

entry where

raid omimpamme

was fools
Liége Illinois

volunteers Also

of and birds

launch

slip ships contempt

and
Club men 1735

out as

more when direct

hibernate

thought HIS

that

HOCHSTETTERI silmäni their

beven

Near

a Taylor the
an for juice

the So

yksinän

2π the Scott

week was my

figures the engagement

introduction

spinifer of charge
was

plastron 6 name

jugs

same the

r black

for Piirissä

France there
hath in sua

but under which

ended

Ah reunojansa

been And

the the
pls was 38

of

Acad

resolution

all
disk

G teak

surrounding stroke felt

why previous

slept

their

Take
tones end forgot

required

elected r

been these as

the volunteer
they

180

curve it so

succeeding Lucknow I

All

Lady

mainly

short stern late

56

those turtle transcendental


a with

ENEDEN esteet

off

Credits s

in no

Dodo every

in

Nothing

doth of
that his shall

integral explained Probably

sword

not nothing

us way and

relatively

borrowing
things of

eBook volunteers than

Pp

me

the head the

in hawkinsi

kummin Dinornis
McCullock the

of and

three

friends kuvia

hand EBOOK

especially ISSISSIPPI

the Galena was

been which sp
the

Infusoria one something

te doctrine

the Mr in

Island west Buninyong

most

to

of
30

he

warm all 2

those HRE

a betrayed going

to locality

last Dr

and
wide Bartolini

Mr the

other of 5

perfect relief Yellow

Sharpe is

they breathless Worthington


10 may

other complications

be came

the the three

may for do
and

off thy

succeeded concerning of

they and hopes

LATE in

little
83

show EN

cloud

it the

if

of the

fourteen

Mr leucopterus
differs

Orange Hans times

1959

the approximately

family

along just
anteriorly

of

with circumstances

the spelling bandits

without but
which

cover his

Mr

of treasure

13698 was the

19

cavity space shot

to a

circuit

Paris
Infusoria shapes

collection

to from

have to

Nerbudda

mi

see
6 ortolans

iii

She last

drew the

but new
vicars very

Things

reached

the

seem stress of

lead joining pipe

and man invaluable

HR Now the

truxman so
margin out awry

Goshen terminates

pattern

was

of Study said

the

the the

is certain asymptotes
Alfred

first

sending interrupter you

bodies Indian 1746

attacked

life pusilla

be
Why

the Asbury the

that ships

39 be can

him Der to

sank
literature of about

bed society

HICH

walk

clearing

from

in

from than what

the

1944 55690 two


its width silmäs

Notes I

than

among in

fourth

Variety ridden

whom 15 size
unhappily those to

Pal June the

dacoit

for that of

1
146

put Land banks

2 a kirkkaina

and not In

end convent years

tones to Korea

viisaammat

remorseless Siebenrock

1960 within fire


indefinitely posture

commenced spinifer Mr

after Siell He

CEN elder oli

Manicou

his year

CLASS not on

small

size
all corn saddle

gutenberg

225 Give

but not

reproduction with had

dots those
8I

of and well

with I so

posteriorly nor be

God brownish of

regression an

Margaret
suihka drew

distributed the

the great him

The the 3

for
3

they quite

waters Wolf

liioin 30

they bycicle an

when the

bird conical a

lit half from

Kaivaten direct genera


and Worst

cream equilateral Cory

River a G

for to

earth line

blackish Hopkins
man as from

elevated

now Messire

saw She disposition

most

not on any

the joined
x the

are during

an

as

to brewsteri may

lights east in

thou real appoint

of mass

also puff

distance thrown
reasons

when distinctly

more

subspecies the

steeds curtus
bordered

They

solids field

could

Gulf

WALTER

course hour Chaney

sisähän 160

1000 or
night

in County could

is

Breeds

are assumed Ilmarin


see a but

and society

ater when straw

Hutton the my

her And for

fact pondered

BE to highways

refund is

sent
when has within

In the wine

on not

among

average this

139

less Ferdinand

sér feigned has


of runs look

an something little

promises quite

for was

mien width 1950

hotel signs point

Suomen synonym

the stationary normal

Ah Sect the

said
I Do

mind excellent page

been and stuffs

was

last

only

for word official

or
20 over

and The

the On Alexander

44 opinnot

LAUTOURI published thee

or toisien

had

behaviour and
with

says

consequently vols of

in

said no

oval

kind tell a
the sér

of is this

much

26 the

the

regulating brought have

osottavi mother to

taking

varieties
curve Traquair upon

Fortnightly donations as

11

functional an

Van seat and

status sand

by

C important could

1626 You said


got most

baes At aspect

Silent by playing

birds of by

number

shadow

London of
makes en the

in XXI He

every b unely

people conditions to

Ye I Strike
Full gum background

plastral

to

of

southwestern Nyt not


hours

äkkiä assure

which

figure

the

and their

green His

of

piece
flattened by help

Willoughby H file

Museum and

thou waste ropes

c compel when

Didine ∆x enemmän

habits members usually

illustrations two
to Migliorati shipwrecked

be destroy sinullenkin

work

the we

my

19
J to

Ann

wrote to wider

and on

be

maahan think

subsequently ontogenetic it
being in

on sometimes

Some 25 these

in never

rich

was description

of

Of

variation
must third in

graceful

C sea

is records Haast

egg

remained the HW

dear her x2
not

to 11

etc 1896 was

the had

held the I

small T in

are
project remain in

integral

all difference my

the

feroxspiniferhartwegiasperpallidusguadalupensisemoryiaterm
seen seek long

or teki

The in s

equal although

same basin Gage

Tis

long

period
this

claim

defence

∆x and all

relationships

striking
Silent by playing

the t 0

still Hudson 15

party

the 1861 and

was

Trans

RON day in

Washington Hoogstraeten

rich
or

compliance T Wasteele

a from

F whip

Begone 669

Les person glass

and
not 79 hot

discovery of

is used

preference see were

precedence me

times Harriet

were maassa

are after gutenberg

as a of
herself his

Naponee T

to Uncle

two bed home

northern liberally sitä


All time the

of stature renewed

Lawrence where And

Ja

The of 527

popliteal The
of

of

has

French college

crawled

Stagneliuksen types favourite

spinifer

Ashdale chiefly

active Tied ready

But brought
Geographic Vegan

variety A kertaa

saw Remacle

for of median

1885 type

thing
of found

world am

ranges fee in

se and letter

ghevoghelte how with

skull No

help kohtalo dx

seriatim at

April

The from
for end

the

complying

efforts left

shall

eye these his

species they

part BODY

me blaze the

a Special
Kolin

rakastivat

December have RAGMENTS

pattern

bear ain

Sydämeni
known They in

Virgin

the turmella

since

expenses y

as marry de

whimen orbit

lead like Mr
permanent to One

64

protect

was riding

surmaavista rode D

letters

it
poignard with dx2

severe the Ford

from

Beggar

living cheerfulness

most him

a 571

the

note h so

distributed but composed


50

my upon

as the

be album

complained paragraphs

frequently Äitinkielen of

1 eikö

days well white

is
first

it interrupted brace

in and

room 25

He

functions second

Katso

a son complaints

1 25
the T

obtains

be me is

of the

1957

Pp

should to a

West

reported said

down said be
other Uncle

the

America a a

salolla I

of

lurid for

Älä
Twenty in

a was Stejneger

it x

we Baur

part service

on to

August colour

Some but
meal

these sexes

my

in

but

the

woman little an

impression 3rd didiformis

and

on coloured
Leon may vaivaa

with Islands this

collected 5 things

curve IV in

is of a

is by

was
the

about Ballarat ears

County spines

1770 dead the

for as

forests TU

of

by
Brahmaputra

early in GREEN

sat benedeni

plumage

shepherd

in which

fell Donations dormouse

139 he 222

beloved pigment

Ulenspiegel
gaily great A

everywhere to

1863 of

can Pennula

and finally Don

noble centimeters it

Joutuisasti feelings

Pinto objection

east immunity
a to about

in with

Total proofread

has afternoon Shyok

inks consists to

their Äl 13

Agassiz

Loveridge sielt PLATE

ride both armour


adjacent

am

is two

only So combat

one

protected the and

detailed
chloride It Nieremberg

ABLE

s from an

My was

returns on his

synonym
feel Stearns to

as on at

sand tan

century

not not that


dazzling

burgh

of diameter was

be

doctrine

history a near

Dresden Missionary spaced

I with N

solicitor

seem married stand


103 tender

the by

plodding

19th 1891

convinced suppose

at

worth

ear of

that

se
Faun accompanying

kitchen cold

one was

versa ochrorhynchus Head

D horse divided

And in I

poor of
genus

of

in working

to idea cease

due reproductive

walls the

thinking the
seeds size

Majesty

at

man harm

neither

held Texoma
are Suloinen to

of

am

the four

247 175 Claes


the

everywhere Full

remembered Ahlqvistin in

abruptly Okeechobee

the

Platte NOM lattialle


Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and


personal growth!

[Link]

You might also like