PS1e - CMC Session: Statistical Considerations

When Assessing Product Stability and/or Shelf Life

Stability Studies in the IVD Industry

Jeffrey Budd, PhD
Principal Biostatistician
Beckman Coulter
jrbudd@beckman.com
 In Vitro Diagnostics (IVD)
• We supply laboratory test results that doctors
use to diagnose, treat and monitor patients
• We need to ensure that tests remain accurate
even as material used in these tests age
– reagents used to detect and measure substances
– calibrators used to convert from instrument signal
to substance concentration
– control material used to monitor proper system
operation

9/26/2017 IVD Stability Studies 2

 IVD Stability Guidelines
• The two most active organizations in providing
guidelines for the IVD industry are
– International Organization for Standardization
(ISO)
– Clinical and Laboratory Standards Institute (CLSI)
• The most influential guideline for IVD stability
is EP251, published in 2009
– This guideline is currently being revised

9/26/2017 IVD Stability Studies 3

 Types of Stability Studies
• Shelf life
– Original packaging, specified storage conditions
• In-use
– After opening, reconstituting, thawing
• Transportation simulation
– Product exposed to potential extreme conditions
• Performance monitoring
– Is stability behavior maintained over life cycle?

9/26/2017 IVD Stability Studies 4

 Stability Considerations
• Product storage conditions
– Maximize stability (room temp, refrigerate, freeze)
– If range of temperature what is test temperature?
• Acceptance criteria
– What is clinical need, considering intended use?
• Number of lots (3?)
• Mix of shelf life, in-use, transport simulation
– Beginning or end of shelf life?

9/26/2017 IVD Stability Studies 5

 Types of Stability Studies
• Classical
– eg, result measured each month over 13 months
• Isochronous
– eg, each month test material placed in stability
condition, all measured together at 13 months
• Matching
– eg, each month, test compared to reference
• Accelerated (Arrhenius, other options)

9/26/2017 IVD Stability Studies 6

 Time Point Value Assignment
• Factors to consider
– Within run*
– Between vial*
– Between run
– Between day
– Calibration to calibration
– Reagent lot to lot
– Calibrator lot to lot
– Instrument to instrument
– Drift over time*
* Currently considered in EP25-A
9/26/2017 IVD Stability Studies 7
 Designing a stability study
• Minimize systematic influences
– Use same instrument(s), reagent lot(s), calibrator
lot(s) across the study period
– Be aware of potential drift due to these factors
• Sample random factors (eg, calibrations, runs)
• Determine uncertainty at each time point
– CVadj = sqrt(CVcal2/#cals + CVBR2/#runs + CVWR2/#reps)
• Determine sample size given proposed # points
• Use mean of results at each time point
cal = calibrations, BR=between run, WR=within run, reps=replicates
9/26/2017 IVD Stability Studies 8
 Plot Replicates or Not?
Slope = -0.0223
At 365 days:
Fit = -8.16%
95%CI = -9.12%
10% at 399 days

Slope = -0.0223
At 365 days:
Fit = -8.16%
95%CI = -9.48%
10% at 384 days

9/26/2017 IVD Stability Studies Not! 9

 How to determine baseline
• Some suggest that more robust testing be
conducted at day zero to establish baseline
• However, there is no more robust method
than using all the data in the study (via the
regression): set baseline = zero intercept
Akbas (2016)
• This modifies the determination from
measuring change from a set value to
measuring the percent change over time

9/26/2017 IVD Stability Studies 10

 Control Material Matching
• Place control test material in intended use
condition (eg, 4°C)
• Place additional control reference material in
known, unchanging state (eg, -70°C)
• At each time point measure the difference in
results between the two conditions
• Compare drift in this difference to %criteria
• Eliminates the effects of factors: run, day,
calibration, instrument, reagent lot, cal lot
9/26/2017 IVD Stability Studies 11
 Calibrator Matching
• Use unchanging, internal working calibrators
as reference calibrator set at each time point
• Place product calibrator set to be tested in
their standard storage condition
• At each time point run both reference and test
calibrator sets
– In same run, on same instrument, using the same
reagent lot
– Can do multiple repeats if needed

9/26/2017 IVD Stability Studies 12

 Calibrator Matching
18000
Reference calibrator points
with fitted calibration curve
16000
Direction of
degradation
14000

12000 2) Find signal level of

medical decision point Test calibrator points
20000
0 Reference calibrator points

Syste …
from reference curve with fitted calibration curve
System Signal

10000
0
8000

6000

4000
3) Using signal level,
1) Set concentration read concentration off
2000
of interest (eg, medical test calibration curve
decision point)
0
0 20 40 60 80 100 120
Concentration

9/26/2017 IVD Stability Studies 13

 Example:

9/26/2017 IVD Stability Studies 14

 Current Practice (EP25, Ed. 1)
• Determine sample size based mainly on
repeatability (within-run imprecision)
• Plot all replicates (y-axis typically in units)
– Difference from T0 point determined
• Fit regression line to data
– If regression p-value < 0.05 then stability is good
– If regression p-value ≥ 0.05 then use 95% CI of the
regression fit

9/26/2017 IVD Stability Studies 15

 Example of Current Practice

95% CI on Regression Fit

9/26/2017 IVD Stability Studies 16

 Future Practice (EP25, Ed. 2)*
• Determine sample size based on all relevant
variance components
• Plot one estimate per time point
– y-axis can be in units or percentage
– Difference from intercept (β0) determined
• Fit regression line to data
– Use 95% CI on percent of change from T0 = β0

* Based on Holland (2017)

9/26/2017 IVD Stability Studies 17
 Same Example: Future Practice

95% CI on %Change

9/26/2017 IVD Stability Studies 18

 Outcome Predictability

9/26/2017 IVD Stability Studies 19

 Old versus New
• Difference from T0 • % difference from β0
• p-value >0.05 (pass) • Equivalence test
• No confidence • Can state confidence in
statement on outcome outcome
• Unpredictable outcome • Outcome is predictable
• Plot all replicates at Ti • Plot mean at each Ti
• Underpowered study by • Fully powered study
missing variance that covers all variance
components components

9/26/2017 IVD Stability Studies 20

 References
1) Pierson-Perry, J. et al. (2009), “Evaluation of Stability of In Vitro Diagnostic
Reagents: Approved Guideline”, CLSI EP25-A.

2) Akbas, N., Budd J., and Klee, G. (2016), “Multiple Calibrator Measurements
Improve Accuracy and Stability Estimates of Automated Immunoassays”,
Scandinavian Journal of Clinical and Laboratory Investigation, 76, 177-180.

3) Holland, M., Kraght, P., Akbas, N., Budd J., and Klee, G. (2017), “Improved
Statistical Methods for Evaluation of Stability of In Vitro Diagnostic
Reagents”, Statistics in Biopharmaceutical Research,
http://dx.doi.org/10.1080/19466315.2017.1305287 (Accepted)

9/26/2017 IVD Stability Studies 21