Uncertainty in Laboratory Medicine

October 28th, 2019

CERTAINTY IS AN ILLUSION
 CERTAINTY IS AN ILLUSION
….and despite significant advances in diagnostic
testing, physicians still face uncertainty in
interpretation.
As the historic paradigm of estimating pretest
probability, followed by laboratory tests to refine
the likelihood of disease, frequently no longer
applies, new approaches are needed to remind
clinicians that results should be considered in
relation to the clinical impression and context.

Whyte MB, Vincent RP. Emerg Med J. 2016

 THE DIAGNOSTIC PROCESS
The diagnostic process is a complex, patient-
centered, collaborative activity that involves
information gathering and clinical reasoning
with the goal of determining a patient’s health
problem.

Improving diagnosis in health care. National Academies of

Sciences, Engineering and Medicine, 2015
 INFORMATION GATHERING
The goal of information gathering in the diagnostic
process is to reduce diagnostic uncertainty enough
to make optimal decisions for subsequent care (J
Kassirer, 1989)
There are four types of information gathering
activities in the diagnostic process: 1) taking a
clinical history and interview, 2) performing a
physical exam; 3) obtaining diagnostic testing; and
4) sending a patient for referrals or consultations.
 CLINICAL REASONING
Clinical reasoning is «the cognitive process that
is necessary to evaluate and manage a patient’s
medical problems».
Clinical reasoning occurs within clinicians’
minds (facilitated or impeded by work system)
and involves judgment under uncertainty, with a
consideration of possible diagnoses that may
explain symptoms and signs, the harm and
benefits of diagnostic testing……….
 UNCERTAINTY

Well defined,
measurable,
traceable,
Complex,
Limited traceability
More complex and
“personalized”
 UNCERTAINTY

Well defined,
measurable,
traceable,
Complex,
Limited traceability
More complex and
“personalized”
Measurement uncertainty
 ISO 15189 and MEASUREMENT
UNCERTAINTY
ISO 15189: 2012,5.5.1.4 requires that
“(medical laboratories)….shall determine
measurement uncertainty for each
measurement procedure in the examination
phase used to report measured quantity values
on patients’ samples”
Additionally, “Upon request, the laboratory shall
make its estimates of measurement uncertainty
available to laboratory users”
MU ESTIMATES : WHAT IS THE VALUE
?
▪ Indicate that multiple values are possible for a given
measurement;
▪ Provide evidence that the term “true value” of a
quantity is a theoretical concept;
▪ Quantify the quality of a result relative to its
suitability for use in making medical decisions;
▪ Assume that known medically significant bias is
eliminated
▪ Assist in identifying technical steps to reduce MU
 MEASUREMENT UNCERTAINTY and
CLINICAL-LABORATORY COMMUNICATION
The admission of uncertainty forms the starting
point for a more open conversation between
laboratory professionals and clinicians (and
patients too)
PADOVA’S LABORATORY REPORTS

TE

RCV
TE
RCV
Why measurement uncertainty
should be adopted in Medical
Laboratories ?
 MU ESTIMATES
WHAT IS THE VALUE ?

▪ To be used to determine if medically

allowable analytical performance
specifications should be/are really achieved
(in routinary practice)
▪ To support interpretation of patient results,
particularly close to medical decision limits
To provide evidence To provide objective
of the compliance information for an
with analytical appropriate
performance interpretation of
characteristics laboratory results

Plebani M Clin. Biochem. 2018

 - Benchmark (performance
characteristics)
- Driver for harmonization
- Driver for comparability and
- TO COMPLY: - Objective information for
with established analytical interchangeability
appropriate interpretation/ of laboratory results
performance characteristics
utilization of laboratory
- TO MONITOR: results
Imprecison (IQc) and bias
(EQAs)
- TO IDENTIFY:
Sources of uncertainty
What is measurement
uncertainty ?
 What is measurement
uncertainty ?
A “non-negative parameter characterizing the
dispersion of the quantity values being
attributed to a measurand, based on the
information used ”

result ± U
Non negative Dispersion
Measurement result parameter characterizing
the dispersion of X
JCGM 200:2008. International vocabulary of metrology – Basic and general concepts and
associated terms (VIM), 3rd edition 2008
 UNCERTAINTY IN LABORATORY
MEDICINE
Uncertainty is a property of a measurement result
which expresses lack of knowledge of the true
value of the result and incorporates the factors
known to influence it.
Uncertainty, therefore, is a quantification of doubt
about the measurement result as is caused by the
interplay of errors which create dispersion around
the estimated value of the measurand: the smaller
the dispersion, the smaller the uncertainty.
 Main guidelines available for estimating measurement
uncertainty for medical laboratories

Nordtest TR
537 ed. 3.1
Eurachem ISO/TS
CITAC Guide
GUM 1st CG 4, 2012 20914:2019
edition EA4-16 (first edition 7-
CLSI EP29-A
(Cofrac) 2019)
1993 2004 2012
1980 1995 2008 2014

INC-1 GUM 3rd CAP 15189

GUM 2nd print program
print
 Approaches to measurement uncertainty
estimation
GUM – JCGM 200:2008
Measurement uncertainty should be calculated by Type A and Type B
uncertainties.
Type A evaluations include any statistical analysis of a series of observation;
Type B evaluations include any methods for evaluating uncertainty from
distribution of error “a priori” known.

CLSI EP29 - 2012

Measurement uncertainty should be calculated either by Top Down or
Bottom up approaches.

NORDTest – 2012
Proposed an approach based on precision and systematic errors that may
be estimated by IQC and EQAs results.
 STATE OF THE ART

….that can be
monitored in
laboratories by

EQAs/PT

IQC

Modified from Menditto et al. Accred Qual Assur 2007; 12:45.

 Sources of uncertainty can be
combined
• Uncertainty of measurement comprises, in general,
many components, each one definable as standard
uncertainty (u).
• Different standard uncertainty may be finally combined
into quantities called combined uncertainty (uc)
• Combined uncertainty, should be expanded to the
expanded uncertainty, with a given confidence
(usually approximately 95%)
 Many possible sources of
uncertainty….
a) incomplete definition of the measurand;
b) imperfect reaIization of the definition of the measurand;
c) nonrepresentative sampling — the sample measured may not represent
the defined measurand;
d) inadequate knowledge of the effects of environmental conditions on the
measurement or imperfect measurement of environmental conditions;
e) personal bias in reading analogue instruments;
f) finite instrument resolution or discrimination threshold;
g) inexact values of measurement standards and reference materials;
h) inexact values of constants and other parameters obtained from
external sources and used in the data-reduction algorithm;
i) approximations and assumptions incorporated in the measurement
method and procedure;

JCGM 100:2008. Guide to the expression of uncertainty in measurement. 2008.

Document prepared by Technical Committee ISO/TC 212, Clinical laboratory testing and
in vitro diagnostic test systems.
 MU AND FIT-FOR-PURPOSE OF TEST
RESULTS
“ a “one size fits all” calculation of MU is inappropriate; rather
MU should be calculated depending on how the “true” value is
obtained and applied depending on the type of comparison
required for correct result interpretation.
Example scenarios of the components of MU calculation are
given ranging from the simplest comparison of a previous result
from the same patient (e.g. serial troponin measurements
within a short time period on the same analyzer within one
calibration), to repeat measurements over multiple calibrations,
to the interpretation of results against a population reference
interval or a clinical decision limit ”

Tate J and Plebani M. CCLM 2016; 54:1277

 MU AND FIT-FOR-PURPOSE OF TEST
RESULTS
Test Purposes and Uncertainty: components to be included
Components to be included in
Test purpose Examples
measurement uncertainty
e.g. tumour
Test results are primarily used for markers, Imprecision only
Jones GR. CCLM 2016; 54:1303
monitoring patients over time immunosuppressi Tate J and Plebani M. CCLM 2016; 54:1277
ve drugs.
Test result if used in comparison
with a reference interval either
established in the same laboratory e.g. hormones Imprecision only
Jones GR. CCLM 2016; 54:1303
or verified by the laboratory by
appropriated procedures

Test result is usually compared Imprecision, bias and bias

e.g. glucose, ions uncertainty
with a clinical decision point
Jones GR. CCLM 2016; 54:1303

Padoan A. Clin Biochem. 2018 Jul;57:41-47

Document prepared by Technical Committee ISO/TC 212, Clinical laboratory testing and
in vitro diagnostic test systems.
ISO20914:2019
flow chart
 ISO 20914:2019
The Change of View

Long term precision

Uncertainty of calibrator
 ISO 20914:2019 MU Calculation
1) Absence of medically significant bias and lack of calibration uncertainty

2
𝑢 𝑦 = 𝑢𝑅𝑊

2) Absence of medically significant bias and data on calibration uncertainty

present

2 2
𝑢 𝑦 = 𝑢𝑐𝑎𝑙 + 𝑢𝑅𝑊

3) Presence of medically significant bias, correction for bias uncertainty, and

data on calibration uncertainty present

2 2 2
𝑢 𝑦 = 𝑢𝑏𝑖𝑎𝑠 + 𝑢𝑐𝑎𝑙 + 𝑢𝑅𝑊
 THE MOST SIGNIFICANT UNCERTAINTY
CONTRIBUTIONS TO OVERALL MU

• Long-term imprecision data obtained for IQC

materials for a period sufficient to include all
changes to measuring conditions (URW)
• Uncertainty of end-user calibrator values
(Ucal)- obtainable from the manufacturer or
established by a laboratory that develops its
own measuring system
 Uncertainty of end-user calibrator
Because only few manufacturers provide
uncertainty of their calibrators

ISO 20914
Ask manufacturers Use pragmatic,
for this requirement in alternative approaches
supply contracts

Use intermediate
imprecision only
 FIT-FOR-PURPOSE OF TESTS
The different test purposes were evaluated and measurement
uncertainty values were estimated differently on the fit for test
purposes (mainly used for diagnosis or for patients’
monitoring)

The purpose is mainly monitoring:

only imprecision was used for calculating
measurement uncertainty

Other purposes (e.g. diagnosis): both imprecision

and bias were used for calculating measurement
uncertainty
 After ISO 20914:2019 …

A new practical approach should be evaluated …

 ISO20914:2019
Maximum allowable MU flow chart
 MAXIMUM ALLOWABLE MU
• The magnitude of MU should be suitable for a
result to be used in a medical decision and
ideally as small as technically possible
• ….estimating the expanded uncertainty of the
results produced is of very limited value
unless it can be compared with an upper limit
of allowable expanded uncertainty based on
the quality of results required for medical use

ISO/TS 2914: 2019

 MAXIMUM ALLOWABLE MU
Such limits should be based on models defined
by the 2014 EFLM Consensus Conference
including clinical outcomes, a selected
proportion of biological variation, or, when
information derived from the first two models
are lacking, state-of-the-art of the
measurement performance

ISO/TS 2914: 2019

Should MU be communicated and
how in Laboratory Reports?
MEASUREMENT UNCERTAINTY (MU)
AIMS OF MU

Within the laboratory To users

To provide evidence of the To provide objective

compliance with information for an
analytical performance appropriate interpretation
characteristics by: of laboratory results with
• Setting appropriate quality the aim of improving
specifications patient care and safety
• Monitoring imprecision (IQC)
and bias (EQAs)
• Aiding in the identification of
sources of uncertainty
 RESULTS NOTIFICATION
MU and RESULTS
NOTIFICATION

Result + the range of

Result + MU as Result + MU as
the result due to
number percentage
MU

e.g. 50 ± 0.5 mg/L e.g. 50 ± 1 % e.g. 50 mg/L

(49.5-50.5 mg/L)
• MU should be reported considering the measurand
concentration (different concentration, different MU)
• Reporting the result + range facilitate the clinician
interpretation
 MU and LABORATORY REPORTS
• Including information on the reliability of
results in the laboratory report may lead to a
more careful evaluation of their effective
value in diagnosing and monitoring diseases.
• Although interest in evidence-based medicine
has increased in recent years, evidence-based
strategies have been inconsistently adopted in
patient care.
Plebani M. Clin Chem Lab Med 2007
 UNCERTAINTY

Well defined,
measurable,
traceable,
Complex,
Limited traceability
More complex and
“personalized”
• Request appropriateness
• Quality of biological samples (Pre-pre-analytical phase)
• Appropriate intepretative criteria (measurement units,
reference range, decision limits, refernce change value)
• Post-post-analytical quality (timeliness, right
interpretation/utilization of laboratory information,
outcomes)
 INTERFERING SUBSTANCES and MU
Sources of uncertainty may arise from interfering
substances that modify the interaction of the analyte
with the measuring system and/or the signal generated
by the measurement process.
Examples include patient antibodies to the analyte or
reagent, spectrofotometric interference by free
haemoglobin, or cross-reactivity of structurally related
molecules
These pre-measurement sources of uncertainty are
generally individual sample-specific and not included in
the estimation of MU for typical human samples.
ISO/TS 2914: 2019
COMBINED UNCERTAINTY AND PRE-
ANALYTICAL ERRORS
“However, it seems quite difficult to incorporate the
pre- and post-analytical uncertainty into an MU
calculation. The alternative way is to identify and
continuously reduce the risk of errors in the extra-
analytical phases through a risk management process
that, according to ISO 15189, takes into consideration
all steps of the cycle, namely the steps that are more
vulnerable to error and risk of errors”
Tate J and Plebani M. CCLM 2016; 54:1277
Uncertainty and pre-analytical factors

“However, some laboratorians believe that

searching for pre-analytical quality, e.g. by rejecting
haemolysed samples, should delay/damage patients
care. If so, pre-analytical uncertainty should be
considered and notified to clinicians.
But which degree of uncertainty should be
“permitted” and how should it be “calculated” ? This is
clearly a patient safety issue”.
 Extra-analytical variability

Errors
Extra-analytical
errors should not
be considered as
sources of Extra-analytical
uncertainty
Sources of
uncertainty

Amenable to Random occurring

standardization/harmonization

Reduce variability of these If the magnitude of effect is

sources by appropriate clearly estimable, it can be
standardization approaces included in MU estimation
 HOW SHOULD UNCERTAINTY OF LABORATORY
INFORMATION BE REDUCED/LIMITED?

Clinical
decision
support
Standardization
Clinicians cooperation
Harmonization
Guidelines
 UNCERTAINTY

Well defined,
measurable,
traceable,
Complex,
Limited traceability
More complex and
“personalized”
ONLY UNCERTAINTY IS A SURE THING
The reality is that doctors continually have to
make decisions on the basis of imperfect data
and limited knowledge, which leads to
diagnostic uncertainty, coupled with the
uncertainty that arises from unpredictable
patient response to treatment and from health
care outcomes that are far from binary.
Simpkin AL, Schwartzstein RM. N Engl J Med 2016
 DIAGNOSTIC UNCERTAINTY
“…. laboratory uncertainty is a small part in the
whole clinical reasoning that leads to decision-
making and includes past experience of physicians,
the pre-test probability of a disease or the disease
prevalence, the uncertainty originating from the
measuring procedure and from the interpretation
of the results in view of the patient’s clinical
parameters or comorbidities and in differential
diagnosis”.
Padoan et al. Clin. Chem. Lab. Med. (2019).
REDUCING DIAGNOSTIC UNCERTAINTY

Ironically, as tests have become more precise

and “precision medicine” has become major
pre-occupation, there is growing awareness and
appreciation of the pervasiveness of
uncertainty in medicine.
……more intelligent test selection, timing, and
interpretation, and using a more balanced
understanding of their benefits, harms, costs,
and limitations.
Schiff GD et al Ann Int Med 2018
Diagnostic uncertainty
Added information

Patient with
symptoms, YES Diagnostic
Clinical
condition or uncertainty tests
need
NO
Diagnosis,
Diagnostic error

treatment or NO
Uncertainty on
intervention tests result

YES
YES
Correct
diagnosis? • Measurement
uncertainty influence
correct interpretation of
NO test result
• Lack of information on
patient (e.g co-
Patient with morbidities)
• Discrepancy on test
resolved result (for multiple
symptoms testing)
[email protected]