Applied R&M Manual for Defence Systems

Part D - Supporting Theory

CHAPTER 11

MAINTAINABILITY DEMONSTRATION PLANS

CONTENTS

1 INTRODUCTION 2

2 CONCEPTS 3

3 TEST METHODS 4

4 TASK SELECTION METHODS 7

Issue 1 Page 1
Chapter 11, Leaflet 0
Maintainability Demonstration Plans

1 INTRODUCTION
1.1 The purpose of conducting a Maintainability Demonstration is to provide evidence that
a specified Maintainability parameter (e.g. MTTR, MART, MDT, etc.) will be attained during
operation. This is achieved by undertaking a demonstration test where maintenance tasks are
performed and the time required to complete the task is recorded. The data collected is used
to determine whether the Maintainability is acceptable. This decision is reached once a
significant number of tasks have been performed; this number is governed by the
Maintainability parameter being demonstrated, and the particular test method chosen.

1.2 The demonstration test is defined by one or more numerical requirements and risk
levels that govern the decision criteria of the demonstration test. There are many test methods
available, each with a different specification for the following:
a) Type of maintainability parameter.
b) Accept and reject criteria.
c) Associated risk levels.

Test methods are detailed in MIL-HDBK-470A1, BS6548 : Part 62 and the maintainability
demonstration model MDEM3 which supports NES 10174.

1.3 The basis for test methods is hypothesis testing, which is described in PtDCh7.
Typically, each test method has a null (H0) and an alternative (H1) hypothesis, and producer's
(α) and consumer's (β) risks. For example, a test specification might be:

H0 : Mean Active Repair Time = 30 minutes;

H1 : Mean Active Repair Time = 60 minutes;

with α = 0.10 and β = 0.10.

The demonstration test for this specification will be such that the probability of rejecting a
system whose MART is 30 minutes is 0.10, while the probability of accepting a system whose
MART is 60 minutes is 0.10.

1.4 The Maintainability parameter should be specified in the system specification and
should be representative of the desired system characteristics when in-service. Obviously the
parameter must be a measure which the producer can influence through design. This chapter
discusses the sampling and statistical evaluation procedures required to demonstrate
conformance to the requirement. Leaflet D11/1 describes the various test methods available.

Page 2
 Applied R&M Manual for Defence Systems
Part D - Supporting Theory
2 CONCEPTS

2.1 Hypothesis Testing

2.1.1 The procedure for hypothesis testing is to establish the appropriate hypothesis and its
alternative before the demonstration is conducted. Then the hypothesis can be tested with the
appropriate statistics determined from the sample data.

2.1.2 The first step is to set up the null hypothesis H0, that is there is no real change or
difference between the sample and the population, and to test the null hypothesis against an
alternative hypothesis, of which there are many alternatives. For example, suppose the
required Mean Active Repair Time (MART) for a system is μ0 (population mean). We test a
sample of 30 repair tasks to obtain an observed mean μ. The null hypothesis is that the mean
of the sample equals the MART requirement. The alternative hypothesis H1 is that the sample
mean is greater than μ0:

Null hypothesis H0 : μ = μ0

Alternative hypothesis H1 : μ > μ0

2.2 Producer’s and Consumer’s Risks

2.2.1 One cannot expect the sample mean to equal exactly the expected population mean.
Therefore, we must allow for variation between the means. The variation is described by two
types of errors:
a) Type I error - the test concludes that the sample mean was not equal to the
requirement (population), when in fact μ = μ0. The probability of making a type I
error is α (producer's risk).
b) Type II error - the test concludes that the sample mean was equal to the
requirement (population) when in fact μ ≠ μ0. The probability of making a type II
error is β (consumer's risk).

These errors can be summarised by the table below:

Decision H0 is true H0 is false

Accept H0 No error Type II error

Reject H0 Type I error No error

Table 1: Summary of Errors Associated with Hypothesis Testing

2.2.2 Figure 1 is a visual presentation of the Type I and Type II errors. For a given
decision point, Type I error (α) is part of the population distribution below the decision point.
These are test results which belong to the population distribution, with mean μ0, but would be
rejected. Type II error (β) is part of the sample distribution above the decision point. It is

Issue 1 Page 3
Chapter 11, Leaflet 0
Maintainability Demonstration Plans
evident from Figure 1, that changing the producer's (α) or consumer's (β) risk will change the
decision point. For a fixed sample size, as β decreases, α increases.

Figure 1: Type 1 and Type II Errors

2.2.3 The values for the producer's and consumer's risk are sometimes given in the
requirements specification. However, often the onus is put on the contractor to develop a plan
which will be acceptable to the customer. The implications as to whether to minimise type I
or type II errors needs to be considered. For example, in the example described in 2.1.2, a
type I error to reject H0 when, in fact, H0 is true would mean the failure of the Maintainability
Demonstration, the possibility of re-designs leading to programme delay and financial loss to
the contractor. On the other hand, a type II error, accepting H0 when it is false, would mean
that the system maintainability was not as good as required, leading to higher support costs
in-service. In most engineering situations, a type II error is least desirable and should be
minimised.

3 TEST METHODS

3.1 Test Parameters

3.1.1 Each test method has a Maintainability parameter which it is designed to demonstrate,
assumptions and the required sample size and selection method. Table 2 presents a summary
of the test methods available for planning a demonstration. The choice of test method will
depend on a number of factors, including the Maintainability parameter, and any statistical
assumptions related to the maintainability parameter of interest.

Test Plan Test Parameter Assumptions Sample Size Sample Selection Method
MIL-HDBK-470A Mean Log-normal distribution and No minimum in MIL- Natural occurring failures or
Test 1 - A & BS prior knowledge of variance HDBK-470A, 30 in stratification in MIL-HDBK-
6548 : Part 6 Test 1 BS 6548 470A, or simple random
sampling in BS 6548

Table 2: Summary of Available Maintainability Demonstration Test Methods

(part 1 of 2)

Test Plan Test Parameter Assumptions Sample Size Sample Selection Method
MIL-HDBK-470A Mean No distribution assumption, prior No minimum Natural occurring failures or
Test 1 - B knowledge of variance stratification
MIL-HDBK-470A Critical Percentile Log-normal distribution and No minimum in MIL- Natural occurring failures or
Test 2 & BS 6548 (Fractile) prior knowledge of variance HDBK-470A, 20 in stratification in MIL-HDBK-
: Part 6 Test 4 BS 6548 470A, or simple random
sampling in BS 6548

Page 4
 Applied R&M Manual for Defence Systems
Part D - Supporting Theory
Test Plan Test Parameter Assumptions Sample Size Sample Selection Method
MIL-HDBK-470A Critical Maintenance None No minimum Natural occurring failures or
Test 3 Time or Manhours stratification
MIL-HDBK-470A Median Log-normal distribution Must be 20 Natural occurring failures or
Test 4 stratification
MIL-HDBK-470A Chargeable Maintenance Only valid if the Central Limit Minimum of 50 Natural occurring failures
Test 5 Downtime/ Flight Theory applies. No assumption
regarding probability distribution
MIL-HDBK-470A Manhour Rate Non-statistical test No minimum Natural occurring failures
Test 6
MIL-HDBK-470A Manhour Rate None Minimum of 30 Natural occurring failures or
Test 7 stratification
MIL-HDBK-470A Mean and Percentile/ Log-normal distribution. See also Sequential test Natural occurring failures or
Test 8 Dual Percentile Leaflet 6/1. simple random sampling
MIL-HDBK-470A Mean ( Corrective Task Only valid if the Central Limit Minimum of 30 Natural occurring failures or
Test 9 Time, Preventive Theory applies. No assumption stratification
Maintenance Time, regarding probability distribution
Downtime)/ Mmax ( 90
or 95 Percentile)
MIL-HDBK-470A Median ( Corrective None Minimum of 50 Natural occurring failures or
Test 10 Task(ct) Time, stratification
Preventive
Maintenance(pm) Time ),
Mmaxct ( 95 Percentile),
Mmaxpm ( 95
Percentile)
MIL-HDBK-470A Mean ( Preventive None None All
Test 11 Maintenance Task Time
). Mmax (Preventive
Maintenance Task Time
at any Percentile)
BS 6548 : Part 6 Mean No distribution assumption, prior Minimum of 30 Simple random sampling
Test 2 knowledge of variance
BS 6548 : Part 6 Mean None Minimum of 50 Simple random sampling
Test 3
BS 6548 : Part 6 Proportion of Corrective Log-normal distribution Minimum of 20 Simple random sampling
Test 5 Task Times above a
specified value
BS 6548 : Part 6 Proportion of Corrective None No minimum Simple random sampling
Test 6 Task Times above a
specified value
BS 6548 : Part 6 Proportion of Corrective None Sequential test Simple random sampling
Test 7 Task Times above a
specified value
NES 1017/MDEM Details not available at the time of writing

Table 2: Summary of Available Maintainability Demonstration Test Methods

(part 2 of 2)

3.1.2 The mission profile of the equipment is often the main criterion for selecting a
particular Maintainability parameter. If the equipment is mission critical, then equipment
downtime will determine the Maintainability parameter to be demonstrated. However, if the
equipment is not mission critical, then manpower may be the more important characteristic.
Often emphasis is placed on corrective maintenance as this is unscheduled and could result in
an interruption to the mission; whereas preventive maintenance can be scheduled during
periods of non-use. However, for equipment in continuous use, then the total maintenance
time is important. For one shot devices, such as a missile system, corrective and preventive
maintenance must be considered separately.

Issue 1 Page 5
Chapter 11, Leaflet 0
Maintainability Demonstration Plans
3.1.3 If the requirement for a system is either operational or intrinsic Availability, given
that:

MTBM
Operational Availability =
MTBM + MDT

Where MTBM = Mean Time Between Maintenance

MDT = Mean Down Time

and

MTBF
Intrinsic Availability =
MTBF + MTTR

Where MTBF = Mean Time Between Failure

MTTR = Mean Time Between Repair

then the maintainability parameter to be demonstrated would be MDT and MTTR

respectively.

3.1.4 A Maintenance Free Operating Period (MFOP) is a requirement which can be applied
to military platform and is defined as a period where the platform is maintenance free.
However, the MFOP will be followed by a Maintenance Recovery Period (MRP), which is
the downtime during which the appropriate preventive or corrective maintenance is done to
recover the system to its fully serviceable state so that it is capable of achieving the next
MFOP. For this type of requirement, a maximum downtime or 0.95 probability of completing
the maintenance within a specific time would be the most appropriate Maintainability
parameter. It is important to note that the Maintainability parameter of interest may vary
depending on the maintenance level. Typically, maintenance levels nearer the operational
front line will tend to have Maintainability parameters which define equipment downtime,
whereas more remote levels will have parameters which define maintenance manhours.

3.2 Choosing a Test Method

3.2.1 The test method to be used to demonstrate Maintainability is often determined by the
Maintainability parameter of interest. However, if prior knowledge exists, perhaps from a
Maintainability estimation, then a sequential test can result in a significant reduction in
sample size, otherwise a fixed sample size test would be required. The benefit of a fixed
sample size is that the number of tasks is known prior to the demonstration which is therefore
easier to plan.

3.2.2 Analysis has shown that in many situations a log-normal distribution provides a good
estimation of corrective maintenance repair times. However, it is not safe to assume that
every system will have repair times which are log-normally distributed. For equipment with a
large amount of electronics or a high degree of built-in diagnostics, the distribution should be
tested through use of goodness-of-fit tests such as Chi-square or Kolomogorov-Simirnov (see
PtDCh7).

Page 6
 Applied R&M Manual for Defence Systems
Part D - Supporting Theory
4 TASK SELECTION METHODS

4.1 General
4.1.1 Task selection methods are only required when failure simulation is
used to generate maintenance tasks, rather than for naturally occurring
failures. The two widely used methods are:
Table 3:
Calculations a) Stratified sampling
of Relative b) Non-stratified sampling
Frequency
and Sample The object of both methods is to determine a hypothetical task population
Size which will be representative of the total population of the tasks for the
for an system.
Example
Coolant 4.2 Non-Stratified Sampling
System
4.2.1 This sampling method enables representative tasks to be selected
based on the relative frequency of task occurrence and is probably the most commonly used
approach in the UK. Table 3 shows the computations used on a example coolant system. The
following presents a step by step approach based on NES 1017 and MIL-HDBK-470A:
a) Identify the major items which make up each equipment within the whole
equipment.
b) Subdivide each major item to a unit level at which maintenance will be performed,
as defined in the maintenance plan.
c) Identify the maintenance tasks associated with each unit defined in b. Note that
these are tasks and not failure modes, and the same task may be required for
different failure modes of the same unit.
d) From the failure rate prediction (see PtCCh36) assign a failure rate for each unit
maintenance task as identified in b.
e) Determine the number of units in each major item
f) Determine the resulting failure rate for each unit. This is the product of the unit
failure rate from d., the number of units from e. and their duty cycle.
g) Sum the failure rates for the equipment identified in f.
h) Determine the relative frequency of each unit, by dividing the failure of each unit
by that of the whole equipment.
i) For a Fixed Sample Test Method, the number of tasks per unit level task can then
be calculated by multiplying the relative frequency determined in h. by the sample
size specified by the selected test.
a) For a Sequential Test Method, the relative frequencies of each unit level task are
added to determine a cumulative frequency range for each unit level task. For
example, in Table 2 the first unit range is 0.0 to 0.024, and the next 0.024 to 0.108.
Using random numbers, maintenance tasks are selected whose range of frequency
includes the random number obtained. This task is demonstrated and the process
repeated until an accept/reject decision is reached.

Issue 1 Page 7
Chapter 11, Leaflet 0
Maintainability Demonstration Plans
4.2.2 A Failure Mode and Effects Analysis (FMEA) (see PtCCh33), down to a level at
which at which maintenance is to be performed, could be utilised for this task selection
method. The FMEA will provide the failure modes, which result in the maintenance tasks for
consideration, and the failure rates for the relative frequency of tasks.

4.3 Stratified Random Sampling

4.3.1 The object of stratification is to divide the system into groups with similar task
characteristics. The maintenance tasks within each group should be of a similar task type
(e.g. remove and replace, clean and grease etc.), have similar task characteristics (e.g.
significant diagnosis time, or short setting to work time, etc.), and similar total repair times.
Stratification ensures that the tasks which are selected are not biased towards one task type,
characteristic or repair time.

4.3.2 Engineering judgement plays an important part in stratification, as two very different
maintenance tasks, such as replacing a PEC board and renewing a mechanical valve may have
similar repair times, but it would inappropriate to group the two tasks. The stratification
process based on MIL-HDBK-470A is illustrated in Table 3 and is summarised by step by
step approach below.
a) Steps a., b. and c. are as presented in Section 4.2.1 above.
b) Identify for each maintenance task the estimated maintenance time from the
maintainability estimation (See PtCCh37).
c) Steps d., e. and f. are as in Section 4.2.1 above
d) Group maintenance tasks which have similar characteristics, such as similar
maintenance actions and estimated maintenance times.
e) For each grouping, determine the total failure rate by summing the product of the
unit failure rate, and the number of units for all tasks within the group.
f) Determine the relative frequency of occurrence for each task grouping by dividing
the sum of the total failure rate into the individual total failure rate for each group.
g) For a Fixed Sample Test Method a sample of maintenance tasks equal to four times
the sample sized defined by the test method is allocated to the task groups
appropriate to its relative frequency of occurrence. (i.e. for a test method requiring
a minimum sample size of 30, the population sample size would be 4 x 30 =120)
h) Allocate the maintenance tasks among the task groups in accordance with the
relative frequency of occurrence of maintenance group. The task which is actually
demonstrated, is then selected from the maintenance tasks allocated to the group.
Note: Once the maintenance task has been demonstrated it is not returned to the
sample and therefore is only demonstrated once.
i) For a Sequential Test Method, the relative frequency of each unit level task is
added to the previous, in order to determine a cumulative frequency range for each
unit level task. Using random numbers, maintenance tasks are selected whose
range of frequency includes the random number obtained. This task is
demonstrated and the process repeated until an accept/reject decision is reached.
The demonstrated task is then returned to the sample.

Page 8
 Applied R&M Manual for Defence Systems
Part D - Supporting Theory

Table 4: Stratification Process for an Example Coolant System

(Part 1 of 2)

Table 4: Stratification Process for an Example Coolant System

(Part 2 of 2)

4.4 Maintenance Task Selection

4.4.1 The methods listed above describe how to determine the sample sizes, but not how to
select the sample tasks for demonstration. NES 1017 states that for each item requiring a
demonstrated task, a FMEA should be consulted to determine the predominant failure mode
to be simulated in the item. Otherwise, if a number of failure modes are possible, to employ a
further simple random sampling method to determine which failure mode to use.

4.4.2 MIL-HDBK-470A suggests a similar method based on the frequency of occurrence of

failure modes. Table 3 indicates the allocation of maintenance tasks for each group of similar
tasks, based on their estimated frequency of occurrence. The population allocation for the
Thermostatic Control Valve (TCV) is three, which means that at least three failure modes
must be considered, from which only one will be selected for simulation. To select the failure
mode for simulation, a random sampling procedure is used based on the relative frequency of
occurrence of the failure modes.

4.4.3 Even when the failure mode to be simulated has been chosen, there will still be
different ways of inducing the failure. Some methods of failure inducement will result in
different symptoms, which may be either easier or more difficult to detect. This will not
actually affect the maintenance action which takes place, but there is a possibility that the
maintenance time will be affected.

Issue 1 Page 9
Chapter 11, Leaflet 0
Maintainability Demonstration Plans

Page 10
 Applied R&M Manual for Defence Systems
Part D - Supporting Theory
LEAFLET 11/0

REFERENCES

1. MIL-HDBK-470A. Volume 1. 04 August 1997. Designing and Developing

Maintainable Products and Systems. Volume 1. US Department of Defense.

2. BS6548 : Part 6: 1995. Maintainability of Equipment. Part 6. Guide to Statistical

Methods in Maintainability Evaluation. BSI Standards.

3. DGSS/ADPSS/26/5/7. Issue 3. June 1996. The Maintainability Demonstration

(MDEM) Model Guide to PC Operators. MoD(PE).

4. NES 1017. Issue 3. January 1993. Requirements for Maintainability Demonstrations

of Naval Systems. MoD(PE).

RELATED DOCUMENTS

1. DEF-STAN 00-40. Reliability and Maintainability. MOD(PE).

2. Maintenance Free Operating Periods - The Designs Challenge. M N Relf. Paper

presented to 13th Advances in Reliability Technology Symposium (April 1998). BAe.

3. Delivering Failure Free Operations. C Hockley. Paper presented to 12 Technical

Proceedings of the SRD (June 1997).

Issue 1 Page 11
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

LEAFLET 11/1

MAINTAINABILITY DEMONSTRATION TEST METHODS

CONTENTS

1 INTRODUCTION AND ACKMOWLEDGEMENT 2

2 EXTRACT FROM MIL-HDBK-470A, APPENDIX B 2

B4.1 LIST OF SYMBOLS. 2

B4.2 TEST METHOD 1: TEST ON THE MEAN. 5

B4.3 TEST METHOD 2: TEST ON CRITICAL PERCENTILE. 9

B4.4 TEST METHOD 3: TEST ON CRITICAL MAINTENANCE TIME OR

MANHOURS. 11

B4.5 TEST METHOD 4: TEST ON THE MEDIAN (ERT). 15

B4.6 TEST METHOD 5: TEST ON CHARGEABLE MAINTENANCE DOWNTIME PER

FLIGHT. 18

B4.7 TEST METHOD 6: TEST ON MANHOUR RATE 23

B4.8 TEST METHOD 7: TEST ON MANHOUR RATE - (USING SIMULATED FAULTS)24

B4.9 TEST METHOD 8: TEST ON A COMBINED MEAN/PERCENTILE B4.

REQUIREMENT. 25

B4.10 TEST METHOD 9: TEST FOR MEAN MAINTENANCE TIME (CORRECTIVE,

PREVENTIVE, COMBINATION OF CORRECTIVE AND PREVENTIVE) AND MMAX. 36

B4.11 TEST METHOD 10: TESTS FOR PERCENTILES AND MAINTENANCE TIME
(CORRECTIVE PREVENTIVE MAINTENANCE). 38

B4.12 TEST METHOD 11: TEST FOR PREVENTIVE MAINTENANCE TIMES. 40

Issue 1 Page 1
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

5 INTRODUCTION AND ACKNOWLEDGEMENT

5.1 This leaflet duplicates Sections B4.1 to B4.12 of Appendix B to MIL-HDBK-470A.
This reference is a comprehensive description of the most frequently used maintainability
demonstration test methods, and includes many examples.

6 EXTRACT FROM MIL-HDBK-470A, APPENDIX B

6.1.1.1.1 List of Symbols. The following symbols and notations are common to test
methods 1 – 3 contained in this appendix.

X = the random variable which denotes the maintenance characteristics of interest (e.g.
X can denote corrective maintenance time, preventative maintenance time, fault
location time, man-hours per maintenance task, etc.).

Xi = the ith observation or value of the random variable X.

n = the sample size.

1 n
X = the sample mean X = ∑ ( X i ))
n i =1

E(random variable) = the expected value of the variable

σ2 = E[(lnX -θ)2] = the true variance of lnX

μ = E(X) = the true mean of X

d$ 2 = Var(X) = E[(X-μ)2] = the true variance of X

d$ 2 = the sample variance of X (i.e. d$ 2 =

1 n 1 ⎛ n 2 2⎞
∑
n − 1 i =1
( X i − X ) 2
= ⎜ ∑ X i − nX ⎟
n − 1 ⎝ i =1 ⎠

~
d2 = the prior estimate of teh variance of the maintenance time.

Xp = the (1-p)th percentile of X (i.e. X.05 = the 95th percentile of X).

~
M = X.05 = the median of X.

Y = ln(X) = the natural logarithm of X.

Y = the sample mean of Y.

θ = E(ln(X)) = the true mean of ln(X).

Page 2
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
~
σ2 = the prior estimate of the variance of the logarithm of maintenance times.

s2 = the sample variance of ln(X).

Zp = the standardised normal deviate exceeded with probability p (i.e.

⎛ 2⎞
⎜ -Z ⎟
∞ ⎜ ⎟
1 ⎜ 2 ⎟
∫ 2π
e ⎝ ⎠
dz = p)
Zp

Zα = Z(1-β) = standardised normal deviate exceeded with probabilities α and (1-β)

respectively.

α = the producer’s risk: the probability that the equipment will be rejected when it has a
true value equal to the desired value (H0).

β = the consumer’s risk: the probability that the equipment will be accepted when it has
a true value equal to the maximum tolerable value (H1).

H0 = the desired value specified in the contract or specification and is expressed as a

mean, critical percentile, or critical maintenance time.

H1 = the maximum tolerable value. Note: H0 < H1.

When X is a log-normally distributed random variable:

1 2
f(x) = e -1/2σ (1nx - θ) 2 , 0 < x < ∞
σx 2π

If Y = ln(X) the probability density of Y is normal with mean θ and σ2 variance

Y~N(θ.σ2)

Properties of the log-normal distribution:

⎛ σ2 ⎞
⎜⎜ θ + ⎟⎟
⎝ 2 ⎠
mean = μ = e

variance = d2 = e
(θ+σ ) ⎛⎜ e σ 2 − 1⎞⎟
2

⎝ ⎠

~
median = M = e ϑ

mode = M = e
(θ − σ )
2

(θ + Zpσ )
(1-p)th percentile = X p = e

TABLE B_VIII. Standardised Normal Deviates

Issue 1 Page 3
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

P Zp
0.01 2.33
0.05 1.65
0.10 1.28
0.15 1.04
0.20 0.84
0.30 0.52

The following symbols are common to test methods 4, 8-11 contained in this appendix.

Xci = Maintenance downtime per corrective maintenance task (of the ith task).

X pm j = Maintenance downtime per preventative maintenance task (of the ith task).

nc = Number of corrective maintenance tasks sampled.

npm = Number of preventative maintenance tasks sampled.

β = Consumer’s risk.

φ = That value, corresponding to risk, which is obtained from a table of normal

distribution for a one-tail test.

fc = Number of expected corrective maintenance tasks occurring during a representative

operating time (T).

fpm = Number of expected preventive maintenance tasks occurring during a

representative operating time (T).

T = Item representative operating time period.

Dt = Total maintenance downtime in the representative operating time (T).

X c , X pm , X p/c = Mean downtimes of sample. (Corrective, Preventive and combined

Corrective/Preventive Maintenance Times).

M'Max = Sample calculated maximum corrective maintenance downtime.

c

μc = Specified mean corrective maintenance time.

μ pm = Specified mean preventive maintenance time.

μ p/c = Specified mean maintenance time. (Taking both corrective and preventive
maintenance time into account)

Page 4
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
M Max = A requirement levied in terms of a maximum value of a percentile of task time (i.e.
95% of all corrective task times must be less than 60 minutes) usually taken as the
90th or 95th percentile.

M Max c = Specified M Max of corrective maintenance downtimes.

M Max pm = Specified M Max of preventive maintenance downtimes.

θc = E(lnXc) = Expected value of the logarithm of corrective maintenance tasks.

Log X c . Log X c = Log to the base 10 of X c , X c

i i

ln X ci .ln X c = Natural logs of X c , X c

i
~
M ct = Median value of corrective maintenance tasks.

~
M pm = Median value of preventive maintenance tasks.

6.1.1.1.2 TEST METHOD 1: Test On The Mean. This test provides for the demonstration
of maintainability when the requirement is stated in terms of both a required mean value (μ1 )
and a design goal value (μ 0 ) (or when the requirement is stated in terms of a required mean
value (μ1 ) and a design goal value (μ 0 ) is chosen by the contractor). The test plan is
subdivided into two basic procedures identified herein as Test Plan A and Test Plan B. Test
A makes use of the lognormal assumption for determining the sample size, whereas Test B
does not. Both tests are fixed sample tests (minimum sample size of 30), which employ the
Central Limit Theorem and the asymptotic normality of the sample mean for their
development.

ASSUMPTIONS

Test A - Maintenance times can be adequately described by a lognormal distribution. The

variance, σ2, of the logarithms of the maintenance times is known from prior information or
reasonable precise estimates can be obtained.

Test B - No specific assumption concerning the distribution of maintenance times are

necessary. The variance d2 of the maintenance times is known from prior information or
reasonably precise estimates can be obtained.

Hypotheses

H 0 : Mean = μ 0 (Equation B-2)

H1 : Mean = μ 1 .(μ 1 > μ 0 ) (Equation B-3)

Illustration: H 0 : μ 0 = 30 minutes

Issue 1 Page 5
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
H1 : μ 1 = 45 minutes

Note that μ 0 is normally the specified maintainability index value, and that μ1 is typically the
maximum acceptable value of the specified index.

SAMPLE SIZE - For a test with producer’s risk α and consumer’s risk (β) the sample size for
Test A is given by:

(Z α μ 0 + Zβ μ1 ) 2 ~2
n= 2
(e σ - 1) (Equation B-4)
(μ1 - μ 0 )

where ~
σ 2 is a prior estimate of the variance of the maintenance times and Z α and Zβ are
standardised normal deviates. The sample size for Test B is given by:
2
⎛ Z α + Zβ ⎞
⎜ ⎟
⎜ μ -μ ⎟
n =⎜ 1~ 0 ⎟ (Equation B-5)
d
⎜⎜ ⎟⎟
⎝ ⎠
~
where d 2 is a prior estimate of the variance of the maintenance times. Z α and Zβ are
standardised normal deviates.

Decision Procedure - Obtain a random sample of n maintenance times, X1, X2 …… Xn and

compute the sample mean.

1 n
X= ∑ Xi
n i =1
(Equation B-6)

and the sample variance

1 ⎛ n 2⎞
d̂ 2 = ⎜⎜ ∑ X i 2 - n X ⎟⎟ (Equation B-7)
n - 1 ⎝ i =1 ⎠

d̂
Test A: Accept if X ≤ μ 0 + Z α (Equation B-8)
n

d̂
Test B: Accept if X ≤ μ 0 + Z α (Equation B-9)
n

Reject otherwise.

Discussion - By the central limit theorem, the sample mean X is appropriately normal for
large n with mean E(X) and variance Var (X) . In Test A, under the log-normal assumption
⎛ ~ 2 ⎞⎟
⎜ 2θ + σ
2 2 ⎜ ⎟ σ ~2 ~2
Var X = d where d = e ⎝ ⎠
(e − 1) = μ 2 (e σ - 1) . Thus the sample size n, can be

Page 6
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

computed using a prior estimate of ~

σ 2 . In Test B, a prior estimate of d 2 is assumed to be
available to calculate the sample size. A critical value C is chosen such that
μ 0 + Zα Var X = C = μ 1 - Z β Var X .

If μ = μ 0 , then P( X > C) = α and if μ = μ 1 then P( X ≤ C) = β .

Example – It is desired to test the hypothesis that the mean corrective maintenance time is
equal to 30 minutes against the alternate hypothesis that the mean is 45 minutes with
α = β = 0.05.

Then H 0 : μ 0 = 30 minutes

H1 : μ 1 = 45 minutes

Test A: Under the log-normal assumption with prior estimate of ~

σ 2 = 0.6, the sample size

using equation B-4 is: n c =

[1.65(30) + 1.65(45)]2 (e 0.6 - 1) = 56.
(45 − 30) 2
~ ~
Test B: Under the distribution-free case with a prior estimate of d 2 = 900. (or d = 30) the
sample size using equation B-5 is:
2
⎡ ⎤
⎢ 3.29 ⎥
nc = ⎢ ⎥ = 43
⎢ ⎛ 45 - 30 ⎞ ⎥
⎢ ⎜⎝ 30 ⎟⎠ ⎥
⎣ ⎦

Operating Characteristic (OC) Curve – The OC curve for Test B for this example is given in
Figure B-4. It gives the probability of acceptance for values of the mean maintenance time
from 20 to 60 minutes. The OC curve for Test A for this example is given in Figure B-3. It
gives the probability of acceptance for various values of the mean maintenance time. Thus, if
the true value of μ is 40 minutes, then the probability that a demonstration will end in
acceptance is 0.21 as seen from Figure B-3.

Issue 1 Page 7
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

0.9

0.8

0.7
Probability of acceptance

0.6

0.5

0.4

0.3

0.2

0.1

0
0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Critical Maintenance Time Percentile

Figure B-3. OC Curve for Test A

0.9

0.8

0.7
Probability of acceptance

0.6

0.5

0.4

0.3

0.2

0.1

0
20 25 30 35 40 45 50 55 60
Mean Maintenance Time (Minutes)

Figure B-4. OC Curve for Test B

Page 8
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
6.1.1.1.3 TEST METHOD 2: Test On Critical Percentile. This test provides for the
demonstration of maintainability when the requirement is stated in terms of both a required
critical percentile value (T1) and a design goal value (T0) [or when the requirement is stated in
terms of a required percentile value (T1) and a design goal value (T0) is chosen by the system
developer]. If the critical percentile is set at 50 percent, then this test method is a test of the
median. The test is a fixed sample size test. The decision criterion is based upon the
asymptotic normality of the maximum likelihood estimate of the percentile value.

ASSUMPTIONS

Maintenance times can be adequately described by a log-normal distribution. The variance,

σ2, of the logarithms of the maintenance times is known from prior information or reasonably
precise estimates can be obtained.

HYPOTHESES

H0 : (1-p)th percentile, Xp = T0 (Equation B-10)

or P[X > T0] = p

H1 : (1-p)th percentile, Xp = T1 (Equation B-11)

or P[X > T1] = p, (T1 > T0)

Illustration: H0 : 95th percentile = Xp = X.05 = T0 = 1.5 hours

ln T0 = 0.4055

H1 : 95th percentile = Xp = X.05 = T1 = 2 hours

ln T1 = 0.6932

SAMPLE SIZE - To meet specified α and β risks, the sample size to be used is given by the
formula

⎛ 2 + Z p 2 ⎞ ⎛ Z α + Zβ ⎞ 2
n =⎜ ⎟~σ2 ⎜ ⎟ (Round up to next integer) (Equation B-12)
⎜ 2 ⎟ ⎜⎝ lnT1 − lnT0 ⎟⎠
⎝ ⎠

where:
~
σ 2 is a prior estimate of σ2, the true variance of the logarithms of the maintenance
times.

Zp is the standardised normal deviate corresponding to the (1-p)th percentile.

DECISION PROCEDURE - Compute:

1 n
Y= ∑ ln X i
n i =1
(Equation B-13)

Issue 1 Page 9
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
1 ⎡n 2⎤
s2 = ⎢ ∑
n − 1 ⎣ i=1
(lnX i ) 2 − n Y ⎥ (Equation B-14)
⎦
1/ 2
⎡1 Zp ⎤
2
X * = ln T0 + Z α s ⎢ + ⎥ (Equation B-15)
⎢⎣ n 2(n − 1) ⎥⎦

Accept if Y + Z p s ≤ X * (Equation B-16)

Reject otherwise.

Discussion - This test is based upon the fact that under the log-normal assumption, the (1-
(θ + Zpσ)
p)th percentile value is given by X p = e . Taking logarithms gives ln Xp = θ + Zpσ, and
using maximum likelihood estimates for the normal parameters θ and σ, the (1-p)th percentile
n −1
maximum likelihood estimate is ln X̂ p = Y + Z p σ . ln Xp is approximately normal. To
n
meet the producer's risk requirements, a critical value X* is chosen for the sample estimate of
the (1-p)th percentile Xp. Note Y = θˆ is an estimate for θ.

Example - The following hypotheses are to be tested at α = β = .10

H0 : 95th percentile = X.05 = 1.5 hours = T0; ln T0 = .4055

H1 : 95th percentile = X.05 = 2.0 hours = T1; ln T1 = .6932.

A prior estimate of ~
σ 2 is equal to 1.0. Using equation B-12,

⎛ 2 + (1.65) 2 ⎞ (2.65) 2
n c = ⎜⎜ ⎟1.0
2 ⎟ (ln2.0 − ln1.5) 2
⎝ ⎠

The critical value X* is given by equation B-15,

1/2
⎡1 Zp ⎤
2
X * = ln T0 + Z α s ⎢ + ⎥
⎢⎣ n 2(n − 1) ⎥⎦

1/2
⎡ 1 (1.65) 2 ⎤
= ln 1.5 + 1.28s ⎢ + ⎥
⎣187 372 ⎦

= .4055 + .1437s

OC Curve - The OC curve for Test Method 2 for this example is given in Figure B-5. It gives
the probability of acceptance for various values of the 95th percentile of the maintenance time
distribution. If the true value of X0.05 is 1.7 hours, then the probability that a demonstration
will end in acceptance is 0.57 as seen from Figure B-5.

Page 10
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

0.9

0.8

0.7
Probability of acceptance

0.6

0.5

0.4

0.3

0.2

0.1

0
1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2
95th Percentile Maintenance Time (Minutes)

Figure B-5. OC Curve for Test Method 2

6.1.1.1.4 TEST METHOD 3: Test On Critical Maintenance Time or Manhours. This

test provides for the demonstration of maintainability when the requirement is specified in
terms of both a required critical maintenance time (or critical manhours) (XP1) and a design
goal value (XP0) (or when the requirement is stated in terms of a required critical maintenance
time (XP1) and a design goal value (XP0) is chosen by the system developer). The test is
distribution-free and is applicable when it is desired to establish controls on a critical upper
value on the time or manhours to perform specific maintenance tasks. In this test both the
null and alternate hypothesis refer to a fixed time and the percentile varies. It is different
from Test Method 2 where the percentile value remains fixed and the time varies.

ASSUMPTIONS - No specific assumption is necessary concerning the distribution of

maintenance time or manhours.

HYPOTHESES

H0 : T = XP0

(P1 > P0) (Equation B-17)

H1 : T = XP1 (Equation B-18)

For a specified α and β.

Illustration

Issue 1 Page 11
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
H0 : 30 minutes = X0.50 = 50th percentile (median)

H1 : 30 minutes = X0.75 = 25th percentile.

SAMPLE SIZE, n, AND ACCEPTANCE NUMBER, c -

The normal approximation to the binomial distribution is employed to find n and c when P0 is
not a small value. Otherwise, the Poisson approximation is employed. The equations for n
and c are as follows:

For 0.20 < p0 < 0.80 (pi = 1-Qi)

2
⎡ Zβ p1Q1 + Z α p 0 Q 0 ⎤
n =⎢ ⎥ (Use next higher integer value) (Equation
⎢⎣ p1 − p 0 ⎥⎦
B-19)

⎡Z p p Q +Z p p Q ⎤
c = n⎢
β 0 1 1 α 1 0 0 ⎥ (Use next lower integer value) (Equation B-20)
⎢ Z p Q +Z p Q ⎥
⎣ α 0 0 β 1 1 ⎦

For P0 < 0.20, n and c can be found from the following two equations:

c e − np0 (np 0 ) r
∑ r!
≥1 − α (Equation B-21)
r =0

c e − np1 (np1 ) r
∑ r!
≤β (Equation B-22)
r =0

Table B-IX provides sampling plans for various α and β risks and ratios p1 / p 0 when
p 0 < 0.20.

Decision Procedure - Random samples of maintenance times are taken, yielding n

observations X1, X2, ... Xn. The number of such observations exceeding the specified time T
is counted. This number is called r.

Accept H0 if r ≤ c (Equation B-23)

Reject H0 if r > c (Equation B-24)

Example - A median value of 30 minutes is considered acceptable whereas if 30 minutes is

the 25th percentile then this is considered unacceptable. The following hypotheses result:

H0 : 30 minutes = X0.50 = 50th percentile (median)

H1 : 30 minutes = X0.75 = 25th percentile.

α = β = .10

Page 12
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
Then, Zα = Zβ = 1.28, p0 = 0.50, p1 = 0.75. Using equations B-19 and B-20:
2
2
⎡ (.75)(.25) + (.50)(.50) ⎤
n = (1.28) ⎢ ⎥ ≈ 23
⎢⎣ (.25) ⎥⎦

Issue 1 Page 13
 and
Table B-IX. Sampling Plans for Specified 0p, p1 , α and β when p0 is Small (e.g. p0 <0.20)

Page 14
⎣⎢
c = 23 ⎢
= 0.10 = 0.20
Chapter 11, Leaflet 1

p1
k=— = 0.05 = 0.10 = 0.20 = 0.05 = 0.10 = 0.20 = 0.05 = 0.10 = 0.20
p2

c D c D c D c D c D c D c D c D c D

1.5 66 54.1 54 43.4 39 30.2 51 43 40 33 29 23.2 36 31.8 27 23.5 17 14.4

Maintainability Demonstration Plans

2 22 15.7 18 12.4 14 9.25 17 12.8 14 10.3 10 7.02 12 9.91 9 7.29 6 4.73

2.5 13 8.46 10 6.17 8 4.7 10 7.02 8 5.43 6 3.9 7 5.58 5 3.84 3 2.3

3 9 5.43 7 3.98 6 3.29 7 4.66 5 3.15 4 2.43 4 3.09 3 2.3 2 1.54

1.28 (.50)(.50) + 1.28 (.75)(.25)

4 6 3.29 5 2.61 4 1.97 4 2.43 3 1.75 2 1.1 3 2.3 2 1.54 1 0.824

⎦⎥
⎡1.28(0.5) (.75)(.25) + 1.28(.75) (.50)(.50) ⎤
5 4 1.97 3 1.37 3 1.37 3 1.75 2 1.1 2 1.1 2 1.54 1 0.824 1 0.824

⎥ ≈14
10 2 0.818 2 0.818 1 0.353 1 0.532 1 0.532 1 0.532 1 0.824 1 0.824 0 0.227

To find the sample size n, for given p 0 , p1 , and , divide the appropriate D value by p 0 and use the greatest integer less than the quotient.
Example: p0 = 0.05, p1 = 0.20, = 0.10, = 0.05. Then k = 0.20/0.05 = 4 and n = D/0.05 = 2.43 / 0.05 = 48 . The acceptance number is c = 4.
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
OC Curve - The OC curve for Test Method 3 for this example is given in Figure B-6. It gives
the probability of acceptance for values of probability p, varying from 0.3 to 1.0. Here Xp is
the (1-p)th percentile. Thus, if the true value of the given critical maintenance time is the
40th percentile, i.e., if the value of p is 0.6, then the probability that a demonstration will end
in acceptance is 0.61 as seen from Figure B-6.

0.9

0.8

0.7
Probability of acceptance

0.6

0.5

0.4

0.3

0.2

0.1

0
0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Critical Maintenance Time Percentile

Figure B-6: OC Curve for Test Method 3

6.1.1.1.5 TEST METHOD 4: Test on the Median (ERT). This method provides for
demonstration of maintainability when the requirement is stated in terms of an Equipment
Repair Time (ERT) median, which will be specified in the detailed equipment specification.

ASSUMPTION

This method assumes the underlying distribution of corrective maintenance task times is
lognormal.

SAMPLE SIZE

The sample size required is 20. This sample size must be used to employ the equation
described in this test method.

TASK SELECTION AND PERFORMANCE - Sample tasks are selected in accordance with
the stratification procedure. The duration of each task is recorded and used to compute the
following statistics:

Issue 1 Page 15
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
nc
∑ (LogX ci )
Log MTTR G = i=1 (Equation B-25)
nc

nc
∑ (log X ci ) 2
i =1
S= − (log MTTR G ) 2 (Equation B-26)
nc

(Note: All logarithms in equations B-25 and B-26 are to be taken to the base 10.)

Where: MTTRG is the measured geometric mean time to repair. It is the equivalent to the
~
M ct used in other plans included in this document.

DECISION PROCEDURE - The equipment under test will be considered to have met the
maintainability requirement (ERT) when the measured geometric mean-time-to-repair
(MTTRG) and standard deviation (S) as determined in equation B-26 above satisfies the
following expression:

Accept if log MTTRG ≤ log ERT + 0.397(S) (Equation B-27)

where:

log ERT = logarithm of the equipment repair time

log MTTRG = the value determined in accordance with equation B-25

S = the value determined in accordance with equation B-26.

DISCUSSION - The value of equipment repair time (ERT) to be specified in the detailed
equipment specification should be determined using the following expression:

ERT (specified) = 0.37 ERTmax (Equation B-28)

ERTmax = the maximum value of ERT that should be accepted no more than 10 percent
of the time.

0.37 = σ value resulting from application of "student’s t" operating characteristic that
assures a 95 percent probability that an equipment having an acceptable ERT will not be
rejected as a result of the maintainability test when the sample size is 20, and assuming a
population standard deviation (σ) of 0.55.

DERIVATION OF CRITERIA - The following are brief explanations of the derivations of

various criteria specified herein, and are intended for information purposes only. The
acceptance criterion, log MTTRG ≤ log ERT + 0.397(S), assures a probability of 0.95 of
accepting an equipment or system as a result of one test when the true geometric mean-time-
to-repair is equal to the specified equipment repair time (that is, a probability of 0.05 of
rejecting an equipment or system having a true MTTRG equal to the specified ERT). This
was derived by using conventional methods for establishing acceptance criteria. The
conventional methods for determining acceptance based on the measured mean of a small

Page 16
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
sample (that is, sample size less than 30), and when the true standard deviation (σ) of the
population can only be estimated, is to compare the measured mean with the desired mean
using the expression:

(x − x 0 )
t= n c −1 (Equation B-29)
S

where:

Σ i (x i − x ) 2
S = or the standard deviation of the sample
nc

x = the sample or measured mean

x0 = the specified or desired mean

nc = the sample size

xi = the value of one measurement of the sample.

The decision to accept the product will be made when the test results give a value of t, as
calculated for the above expression, numerically less than or equal to a value of t obtained
from "student's t" distribution tables at the established level (that is, 0.99, 0.95, 0.90, etc.) of
acceptance and the appropriate sample size. The "student's t" distribution tables (for a single
tailed area) give a value to t = 1.729 at the 0.95 acceptance level when the sample size is 20
(that is, 19 degrees of freedom). The table for single tailed area is used since only values of
MTTRG lower than the specified ERT is acceptable. To apply the expression for "t" to the
maintainability test, let x 0 = log ERT (specified), x = log MTTRG (measured), S = the
measured standard deviation of the logarithms of the sample of measured repair time, and nc =
the sample size of 20. The measured MTTRG is then compared to the desired ERT by
calculating the value of t using the expression below:

(log MTTR G − log ERT )

t= 19
S

The equipment under test can be acceptable if the value of t calculated from the expression
above is equal to or less than +1.729 (the value of t from the "student's t" distribution tables at
an acceptable level of 0.95 when the sample size is 20). Therefore, the equipment should be
accepted when:

(log MTTR G − log ERT )

19 ≤ + 1.729
S

Upon re-arranging and simplifying the above expressionm, the acceptance criterion is
obtained as shown below:

1.729(S)
log MTTR G − log ERT ≤
19

Issue 1 Page 17
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
log MTTR G ≤ log ERT + 0.397(S) 1

6.1.1.1.6 TEST METHOD 5: Test on Chargeable Maintenance Downtime per Flight.

Because of the relatively small size of the demonstration fleet of aircraft and administrative
and operational differences between it and fully operational units, operational ready rate or
availability cannot be demonstrated directly. However, a contractual requirement for
chargeable downtime per flight can be derived analytically from an operational requirement
of operational ready rate (ORR) or availability. This chargeable downtime per flight can be
thought of as the allowable time (hours) for performing maintenance given that the aircraft
has levied on it a certain availability or operational ready requirement. The requirement for
chargeable downtime per flight will be established using the procedure presented within this
section.

DEFINITIONS - the following definitions apply to this test method:

A = Availability - A measure of the degree (expressed as a probability) to which an aircraft is

in the operable and commitable state at the start of a mission, when the mission is called for at
an unknown (random) point in time. For this test method, availability is considered
synonymous with operational readiness. The aircraft is not considered to be in an operable
and commitable state when it is being serviced and is undergoing maintenance.

TOT = Total Active Time in Hours.

Active Time = That time during which an aircraft is assigned to an organisation for the
purpose of performing the organisational mission. It is time during which:

1. The aircraft is flying or ready to fly.

2. Maintenance is being performed.
3. Maintenance is delayed for supply or administrative reasons.

DUR = Daily Utilisation Rate - The number of flying hours per day.

AFL = Average Flight Length - Flying hours per flight.

NOF = Number of Flights per day.

DT = Downtime - Time (in hours) during which the aircraft is not ready to commence an
assigned mission (i.e. have the flight crew aboard the aircraft).

CMDT = Chargeable Maintenance Downtime - Time (in hours) during which maintenance
personnel are working on the aircraft, except when the only work being down would fall
under the non-chargeable maintenance downtime (NCMDT) category.

NCMDT - Non-chargeable Maintenance Downtime - Time (in hours) during which the
aircraft is not available for immediate flight but the only maintenance being performed is not
chargeable. It would include the following:

1
Reference - “Introduction to Mathematical Statistics”. P Hoel. J Wiley and Sons Inc., 2nd Edition,
1954, pp. 222-229.

Page 18
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
1. To correct maintenance or operational errors not attributable to technical orders,
contractor furnished training or faulty design.

2. Miscellaneous tasks such as keeping of records or taxiing or towing the aircraft to

or from other than the work centre area.

3. Repair of accident or battle damage.

4. Modification tasks.

5. Maintenance caused by test instrumentation.

DDT = Delay Downtime - Downtime (in hours) during which maintenance is required but no
maintenance is being performed on the aircraft for supply or administrative reasons. It would
include the following:

1. Supply Delay Downtime

a. Not Operationally Ready Supply (NORS) time.

b. Item obtainment time from other than the work centre area.

2. Administrative Delay Downtime

a. Personnel breaks such as coffee or lunch.

b. No maintenance people available for administrative reasons.

α = The producer's risk: The risk that the producer (or supplier) must take that the hypothesis
that a true mean = M0 will be rejected even though it is true. The desirable value of α must be
determined by judgement and agreed upon by the procuring activity and the systems
developer. All other things being equal, a smaller value of α will require a larger sample size.

M = The maximum mean chargeable maintenance downtime per flight.

M0 = The required mean CMDT per flight.

M-M0 = The difference between the maximum mean (M) of the parameter being tested and
the specified mean (M0). This value must be determined in conjunction with a value for β,
the consumer's risk. M is a value, greater (or worse) than the specified mean, which the
consumer is willing to accept, but only with a small risk or probability (β). If the true mean is
in fact equal to the value of M selected, the hypothesis the true mean = M0 will be accepted,
although erroneously, 100 β percent of the time.

β = The consumer's risk. The risk, which the consumer is willing to take, of accepting the
hypothesis that the true mean = M0 when in fact the true mean = M. All other things being
equal, a smaller value of β will require a larger sample size.

σ = The true standard deviation of the parameter (CMDT per flight) being tested. This value,
unless it is a specification requirement, will not be known, but an estimate must be made. (It
is assumed that both M and M0 will have the same value of σ.) The developer's

Issue 1 Page 19
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
maintainability math model, previous models, or previous data may be used. All other things
being equal, a larger value of σ will require a larger sample size.

ASSUMPTIONS - This method requires no assumption as to the probability distribution of

chargeable downtime per flight. The method is valid only if the Central Limit Theorem
applies, which means that the sample size (number of flights) must be large enough for this
theorem to apply. The sample size must be at least 50, but the actual size is to be determined
in accordance with equation B-39.

DERIVATION OF CMDT PER FLIGHT FROM AVAILABILITY - The requirement for

CMDT per flight which will be demonstrated will be determined using the following
mathematical derivation:

CMDT + NCMDT + DDT

A =1 − (Equation B-30)
TOT

A(TOT) = TOT - CMDT - NCMDT - DDT (Equation B-31)

CMDT = TOT - A(TOT) - NCMDT - DDT (Equation B-32)

CMDT TOT − A(T0T) − NCMDT − DDT

= (Equation B-33)
NOF NOF

TOT (DUR )
but, NOF = (Equation B-34)
24(AFL)

therefore,

CMDT 24(AFL) A(24)(AFL) NCMDT DDT

= − − − (Equation B-35)
NOF DUR DUR NOF NOF

CMDT
= CMDT per flight, which will be demonstrated.
NOF

Values for DUR an AFL should be those planned for the aircraft during operational use.
NCMDT DDT
Values for and are a function of the operational environment. They should
NOF NOF
be provided to the system developer in the RFP or, if not, must be provided by the developer
in his proposal. The value for availability or operational ready rate should be provided in the
RFP.

⎛ CMDT ⎞
Example - Following is an example of how a requirement for CMDT per flight ⎜ ⎟
⎝ NOF ⎠
will be derived:

Required A = 0.75

DUR = 2 hours per day

AFL = 4 hours per flight

Page 20
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
NCMDT
= 0.2 hours per flight
NOF

DDT
= 1.0 hours per flight
NOF

Then,

CMDT 24(4) (0.75)(24)(4)

= − − 0.2 −1.0
NOF 2 2

CMDT
= 48 - 36 - 0.2 - 1.0
NOF

CMDT
= 10.8 hours per flight
NOF

SAMPLE SIZE - Since the Central Limit Theorem is applied, the expected distribution of the
means will take on a normal distribution as in Figure B-7. If the true mean is equal to M0 and
a particular α is desired, the upper distribution (the mean of the distribution will equal M0)
will apply. It is on this basis that an acceptance rule is generated to the effect that if X is
Z σ
found to be equal to or less than the value M 0 + α the item is to be accepted.
n

zα ′σ
Where: M+
m
zασ
zασ corresponds to the value M0 +
m
m
M0 zασ
M0 +
m

(M − M0 )

zα ′σ
m

M0 M zα ′σ ∞
M+
m

Figure B-7. Distribution of Means

Issue 1 Page 21
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
If the true mean is equal to M (which is greater than M0) the distribution of means will take
on a normal distribution with a mean of M as shown in the lower distribution. The value to
Z σ
be used as an acceptance criterion, M 0 + α , corresponds and is equal to a value:
n

Z α' σ
M+ ; where α' is a new confidence level
n

Zα σ Z α' σ
M0 = =M+ ; (Equation B-36)
n n

where M = M0 + (M - M0) (Equation B-37)

Zα σ Z α' σ
M0 + = M0 + M − M0 + (Equation B-38)
n n

or simplifying, the sample size (n) requirement is:

(Z α − Z α' ) 2 Z α − Z (1−β) ) 2
n= 2
= 2
(Equation B-39)
⎛ M − M0 ⎞ ⎛ M − M0 ⎞
⎜ ⎟ ⎜ ⎟
⎝ σ ⎠ ⎝ σ ⎠

If this expression should result in n less than 50, then a sample of 50 must be used.

α = Probability of rejection if true mean equals M.

1-α' = β = Probability of acceptance if true mean equals M.

Zα' Z(1-β) = Standardised normal deviate as defined.

See table below for relationships between Zw and α and β, where w = α or 1 - β.

Zw .01 .05 .1 .15 .2 .3 .7 .8 .85 .9 .95 .99

2.33 1.65 1.28 1.04 .84 .52 -.52 -.84 -1.04 -1.28 -1.65 -2.33

Zw = Zα or Z(1-β)

Example - Suppose for a requirement of M0 = 2.0, the following statistical test parameters
were agreed to by the procuring activity and the system developer:

M − M0
α = 0.10; Z α =1.28; β = 0.10; Z1−β = 1.28; M − M 0 = 0.30; σ = 1.0; = 0.3
σ

(1.28 + 1.28) 2 (2.56) 2 6.57

Using equation B-39; n = = = = 73
(0.3) 2 (0.3) 2 0.09

Page 22
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
Decision Procedure - The chargeable maintenance downtime (Xi) after each flight will be
measured and, at the end of the test, the total chargeable downtime will be divided by the total
number of flights to obtain ( X ) the sample mean CMDT and the sample standard deviation
(s) of CMDT.
NOF
∑ Xi
i =1
X= (Equation B-40)
NOF

NOF (X i − X) 2 1 ⎡ NOF 2 2⎤
s= ∑ NOF −1
= ∑
(NOF −1 ⎢⎣ i=1
X i − (NOF)X ⎥ (Equation B-41)
i =1 ⎦

ZαS
Accept if: X ≤ M0 + (Equation B-42)
NOF

ZαS
Reject if: X > M0 + (Equation B-43)
NOF

6.1.1.1.7 TEST METHOD 6: Test on Manhour Rate 2 - This test for demonstrating
manhour rate (manhours per flight hour) is based on a determination during Phase II test
operation of the total accumulative chargeable maintenance manhours and the total
accumulative demonstration flight hours. The demonstrated manhour rate is calculated as:

Total Ch arg eable Ma int enance Manhours

Manhour Rate = (Equation B-44)
Total Demonstration Flight Hours

If the demonstrated manhour rate is less than or equal to the manhour rate requirement plus a
maximum value (ΔMR), by which the demonstrated manhour rate will be permitted to differ
from the required manhour rate, then the requirement has been met. ΔMR will be provided,
by the procuring activity, as a percentage of the system manhour rate requirement and will be
determined based upon such considerations as the expected Phase II duration, and prior
experience with similar systems. It is recognised that this demonstration method is non-
statistical in nature and does not allow the determination of quantitative producer's and
consumer's risk levels. It is for this reason that the ΔMR is provided (in a subjective manner)
to minimise the producer's risk.

Normally, all maintenance performed by approved test maintenance personnel during Phase II
and documented in appropriate maintenance reports will be the source of data for identifying
chargeable maintenance manhours. The procuring activity may elect to terminate the
demonstration prior to Phase II completion if sufficient data are collected to project that the
requirement will be met.

The manhour rate requirement must pertain to the aircraft configuration provided for in the
contract. For Phase II flights conducted with a configuration other than this, an appropriate

2
Test Method 6 is intended for use with aeronautical systems and subsystems.

Issue 1 Page 23
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
amount of chargeable manhours will be included in calculating the total chargeable manhours.
This amount will be based upon the predicted manhour rate associated with the equipment not
installed.

Care must be exercised in assuring that the predicted manhour rate pertains to flight time and
not equipment operating time. Appropriate ratios of equipment operating time to flight time
must therefore be developed.

6.1.1.1.8 TEST METHOD 7: Test on Manhour Rate - (Using Simulated Faults) 3 . - This
test for demonstrating manhour rate (manhours per operating hour) is based on (a) the
predicted total failure rate of the equipment used in the formulation of Table B-V (see Section
3.5.2 of Mil-Hdbk-470A, appendix B) and (b) the total accumulative chargeable maintenance
manhours and the total accumulative simulated demonstration operating hours. The
demonstrated manhour rate is calculated as:
n

Total Chargeable Maintenance Hours i=1

∑ X ci + (PS)
Manhour Rate = = (Equation B-45)
Total Operating Time T

Where:

Xc = Manhours for corrective maintenance task i.

i

n = Number of corrective maintenance tasks sampled; n must not be less than 30.

MTBF = MTBF of the unit.

(PS) = Estimated average total manhours which would be required for preventive
maintenance during a period of operating time equal to n.(MTBF) hours.
n
∑ X ci
i =1
= X c = Average number of corrective maintenance manhours per
n
corrective maintenance task.

T = Operating time.

Discussion = When maintenance tasks are simulated, T = n(MTBF), where 1/MTBF = λT, the
total failure rate of the equipment in question.
n n
∑ X ci + (PS) ∑ X ci + (PS) 1 ⎡ (PS) ⎤
i =1
= i =1 = Xc + (Equation B-46)
T ⎢
n • (MTBF) MTBF ⎣ n ⎥⎦

3
Test Method 7 is intended for use with ground electronic systems where it may be necessary to simulate
faults.

Page 24
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

All components of (B-46) with the exception of X c can be considered constants. X c can be
considered a normally distributed variable when n is large (due to the Central Limit Theorem)
d2
with Variance = .
n

1 ⎡ PS ⎤
If X c is normally distributed it can be shown that the function ⎢ X c + ⎥ is also
MTBF ⎣ n ⎦
2
⎛ 1 ⎞⎛ d ⎞
normally distributed around the mean of the manpower rate with Variance = ⎜ ⎟ ⎜ ⎟ ;
⎝ n ⎠ ⎝ MTBF ⎠
assuming d = dˆ .

Decision Procedure - Therefore, if the manhour rate requirement = μR:

Accept if:

PS d̂
X c ≤ μ R (MTBF) − + Zα (Equation B-47)
n n

where α denotes producer's risk.

6.1.1.1.9 TEST METHOD 8: Test on a Combined Mean/Percentile Requirement. - This

test provides for the demonstration of maintainability when the specification is couched in
terms of a dual requirement for the mean and either the 90th or 95th percentile of maintenance
times when the distribution of maintenance time is lognormal.

ASSUMPTIONS - For use as a dual mean and 90th or 95th percentile requirement, the mean
must be greater than 10 and less than 100 units of time; the ratio of the 90th percentile
maximum value to the value of the mean must be less than two (2); the ratio of the 95th
percentile maximum value to the value of the mean must be less than three.

Maximum Ratio of Percentile to Mean

90th Percentile Value 2
95th Percentile Value 3
Distribution assumptions are as defined above.

DISCUSSION - The test method actually demonstrates the 61st percentile value of
maintenance time in combination with either the 90th or 95th percentile values of
maintenance time rather than the mean value of maintenance time in combination with either
the 90th or 95th percentile values of maintenance time. However, because of the particular
characteristic of the lognormal distribution once a 61st percentile value of maintenance time
less than X1 and a 90th or 95th percentile value less than X2 has been demonstrated, for all
practical purposes, a mean value of less than approximately X1 and a 90th or 95th percentile
value less than X2 have likewise been demonstrated.

A dual requirement on maintainability, assuming a lognormal distribution of repair times, of a

maximum value of the Mean in conjunction with either the maximum value of the 90th or
95th percentile of repair time (to be referred to as MMax) results in the definition of various
combinations of θs and σs which are acceptable to the dual requirement. (A complete

Issue 1 Page 25
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
technical description of a lognormal distribution is provided by knowledge of θ and σ, hence
all possible lognormal distributions acceptable to the requirements are defined through
definition of all possible acceptable values of θ and σ.) See Figure B-8A which defines the
acceptable combinations of θ and σ for a Mean of 30 minutes and a 95th percentile (MMax) of
60 minutes.

For the lognormal distribution, it is also possible to structure a dual requirement made up of
the maximum values of two percentiles (for example, the 61st percentile of repair time shall
be a maximum of 30 minutes and the 95th percentile of repair time shall be a maximum of 60
minutes). This dual requirement also results in the definition of various combinations of
acceptable values of θ and σ. See Figure 8-B If a dual percentile requirement could be
structured such that the set of acceptable values of θ and σ defined were almost identical to
the set of values of θ and σ defined for a given dual Mean and percentile requirement then a
demonstration of that dual percentile requirement would in reality also demonstrate the
attainment of the dual Mean and MMax requirement. For this particular instance it has been
found that under the assumption listed above, almost identical acceptable values of θ and σ
are provided for a combined Mean and MMax requirement and a combined 61st percentile
(where the value of the 61st percentile is taken equal to the specified value of the Mean) and
MMax requirement. See Figure 8-B which defines the values of θ and σ acceptable to a dual
61st percentile (where the value of the 61st percentile is taken equal to a specified mean of 30
minutes) and 95th percentile (where the maximum value of the 95th percentile, MMax, is given
as 60 minutes) and Figure 8-C, which is the superimposition of Figure 8-A on Figure 8-B.

1¼ Q
let μ1 = required value of mean
(maximum allowable)

let Xp1 = maximum value of 95th

percentile - Mmax
Q
ln μ1≥θ + .5σ2

ln Mmax ≥ θ + 1.65σ
lnM define all possible values of θ & σ
¾Q
ma lnμ which are capable of meeting or
θ x =
CO θ+ 1 =θ bettering the requirement (defines
M 1.6 +. characteristics of all possible
5σ 5σ 2
AC BIN lognormal distributions capable of
A
CE T (wh meeting or bettering the
½Q
RE PTA ION en
μ
requirements).
QU BL S O 1 =3
IR E T F V 0, let lnμ1 = Q
EM O A M
EN DU LUE ma
TS AL S x =
¼Q 60
)

.2√Q .4√Q .6√Q .8√Q √Q 1.2√Q 1.4√Q 1.6√Q

σ

Figure B-8A. Acceptable Combinations of Dual Requirements

Therefore, tests performed to demonstrate the attainment of both the percentiles in question
actually demonstrates the attainment of values of θ and σ which are almost identically
acceptable to a dual requirement of the Mean and MMax. It follows then that an accept
decision relative to both percentiles would also approximately signify an accept decision for a
dual Mean and MMax requirement.

Page 26
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
Since both percentiles can be considered independent for practical purposes, the same
samples can be used for demonstration of both percentiles, therefore, if desired, the tests may
be run simultaneously.

1¼ Q
let μ1 = required value of
mean (max. allowable)

let Xp1 = maximum value of 95th

percentile - M max
Q lnμ = θ using the required value for the
1 +.28σ 2 (w
hen μ = mean as the maximum value of
1 30) the 61st percentile

ln μ1≥θ + .28σ
lnM
¾Q
ma ln Mmax ≥ θ + 1.65σ
θ = x
CO θ+
MB 1 .6 define all possible values of
AC IN 5σ θ & σ which are capable of
CE AT (w meeting the dual percentile
½Q I he
RE PTA ON nμ requirement.
QU BL S O 1 =3
IR E F 0, let lnμ1 = Q
EM T O VA M
EN DU LUE ma
TS AL S x =
¼Q 60
)

.2√Q .4√Q .6√Q .8√Q √Q 1.2√Q 1.4√Q 1.6√Q

σ

Figure B-8B. Values Acceptable to Dual Requirements of Maximum Values of Two

Percentiles

1¼ Q

Q lnμ = θ +
1 .28σ 2 (when μ
1 = 30)

lnM
¾Q
ma lnμ
x =
θ θ+ 1 =
CO 1.6 θ+
MB 5σ .5σ
AC IN 2
CE A T (w
½Q h
RE PTA ION en
μ
QU BL S O 1 =3
IR E T F V 0,
EM O A M
EN DU LUE ma
TS A x =
¼Q L S 60
)

.2√Q .4√Q .6√Q .8√Q √Q 1.2√Q 1.4√Q 1.6√Q

σ

Figure B-8C. Superimposition Figure B-8A and B-8B

PROCEDURE - Sample tasks are to be selected with respect to the procedure defined for
variable sample/sequential tests. The same sample tasks may be used simultaneously in the

Issue 1 Page 27
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
demonstration of both the Mean and MMax requirements. Table B-X 4 , Table B-XI4, and Table
BII4 (which are based upon the sequential probability ratio of proportion) define the

4
Tables B-X, B-XI and B-XII are appropriate to Test Plans A1, B1 and B2, respectively.

Page 28
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

TABLE B-X. PLAN A1 : OBSERVATIONS EXCEEDING THE VALUE OF THE

MEAN (OR 61ST PERCENTILE VALUE)
# of Tasks # of Tasks
Accept Reject Accept Reject
Observed (N) Observed (N)
5 5 55 - -
6 6 56 13 -
7 - 57 - 21
8 - 58 - -
9 7 59 14 -
10 - 60 - 22
11 - 61 - -
12 0 - 62 - -
13 - 8 63 15 23
14 - - 64 - -
15 1 - 65 - -
16 - 9 66 16 -
17 - - 67 - 24
18 - - 68 - -
19 2 - 69 17 -
20 - 10 70 - 25
21 - - 71 - -
22 3 - 72 - -
23 - 11 73 18 -
24 - - 74 - 26
25 4 - 75 - -
26 - 12 76 19 -
27 - - 77 - 27
28 - - 78 - -
29 5 - 79 20 -
30 - 13 80 - 28
31 - - 81 - -
32 6 - 82 - -
33 - 14 83 21 -
34 - - 84 - 29
35 7 - 85 - -
36 - 15 86 22 -
37 - - 87 - 30
38 - - 88 - -
39 8 - 89 - -
40 - 16 90 23 31
41 - - 91 - -
42 9 - 92 - -
43 - 17 93 24 -
44 - - 94 - 32
45 - - 95 - -
46 10 - 96 25 -
47 - 18 97 - 33
48 - - 98 - -
49 11 - 99 - -

Issue 1 Page 29
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

TABLE B-XI. PLAN (B1) : OBSERVATIONS EXCEEDING Mmax - 90 Percentile

# of Tasks # of Tasks
Accept Reject Accept Reject
Observed (N) Observed (N)
2 2 52 - -
3 - 53 - 5
4 - 54 - -
5 - 55 - -
6 - 56 - -
7 - 57 - -
8 - 58 - -
9 - 59 - -
10 - 60 - -
11 - 61 - -
12 - 62 - -
13 - 63 - -
14 3 64 - -
15 - 65 2 -
16 - 66 - -
17 - 67 - -
18 - 68 - -
19 - 69 - -
20 - 70 - -
21 - 71 - -
22 - 72 - -
23 - 73 - 6
24 - 74 - -
25 - 75 - -
26 0 - 76 - -
27 - - 77 - -
28 - - 78 - -
29 - - 79 - -
30 - - 80 - -
31 - - 81 - -
32 - - 82 - -
33 - - 83 - -
34 - 4 84 - -
35 - - 85 3 -
36 - - 86 - -
37 - - 87 - -
38 - - 88 - -
39 - - 89 - -
40 - - 90 - -
41 - - 91 - -
42 - - 92 - -
43 - - 93 - 7
44 - - 94 - -
45 - - 95 - -
46 1 - 96 - -
47 - - 97 - -
48 - - 98 - -
49 - - 99 - -

Page 30
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

TABLE B-XII. PLAN (B2): OBSERVATIONS EXCEEDING Mmax - 95 Percentile

# of Tasks # of Tasks
Accept Reject Accept Reject
Observed (N) Observed (N)
2 2 52 3
3 - 53 -
4 - 54 -
5 - 55 -
6 - 56 -
7 - 57 0 -
8 - 58 - -
9 - 59 - -
10 - 60 - -
11 - 61 - -
12 - 62 - -
13 - 63 - -
14 - 64 - -
15 - 65 - -
16 - 66 - -
17 - 67 - -
18 - 68 - -
19 - 69 - -
20 - 70 - 4
21 - 71 - -
22 - 72 - -
23 - 73 - -
24 - 74 - -
25 - 75 - -
26 - 76 - -
27 - 77 - -
28 3 78 - -
29 - 79 - -
30 - 80 - -
31 - 81 - -
32 - 82 - -
33 - 83 - -
34 4 84 - -
35 - 85 - -
36 - 86 - -
37 - 87 - -
38 - 88 - -
39 - 89 - -
40 - 90 - -
41 - 91 - -
42 - 92 - -
43 - 93 - -
44 - 94 - -
45 - 95 - -
46 - 96 - -
47 - 97 - -
48 - 98 - -
49 - 99 1 -

accept/reject criteria for the values of the required mean, Mmax (when defined as the maximum
90th percentile value). The number of observations greater than and less than the required

Issue 1 Page 31
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
values of the Mean and Mmax must be cumulated separately and compared to the decision
values shown in teh tables applicable to the two requirements. When one plan provides an
accept decision, attention to that plan is discontinued. The second plan continues until a
decision is reached. The equipment is rejected when a decision to reject on either plan has
occurred regardless of the status of the other plan. The equipment is accepted only when an
accept decision has been reached on both plans. If no accept or reject decision has been made
after 100 observations, the following rule applies:

Plan A1 - Accept only if 29 or less observations are more than the value of the
required Mean.

Plan B1 - Accept only if 5 or less observations are more than M Max c .

Plan B2 - Accept only if 2 or less observations are more than M Max c .

It is recognised and accepted that truncation will somewhat modify probability of acceptance
characteristics as described in the following subsection.

The OC Curve - The operating characteristic curve for the test procedure may be determined
by mapping the probability of acceptance for various selected points on a diagram of the
acceptable and unacceptable regions such as Figure B-8D. (Note that any point can be
identified uniquely by the coefficient of Q, where Q = ln (required Mean), on the ordinate and
the coefficient of Q on the abscissa - let the coefficient of Q be denoted as (C) and the
coefficient of Q be denoted as (K) - for example, point B on Figure B-8D can be uniquely
located at C = 3/4, K = .4). Each point is also representative of a particular lognormal
distribution possessing unique percentiles for the values given for μ1 (required maximum
value for Mean) and MMax, respectively.

Page 32
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

μ1 = 30
Mmax = 60 at 95th percentile
lnμ1 = Q
1¼ Q
POINTS FOR WHICH
PROBABILITY OF
ACCEPTANCE
Q
WILL BE DETERMINED

POINT A POINT B POINT C

¾Q
lnμ
1 =
θ
lnM θ+
ma .5σ
2
x =
½Q θ+
1 .6
D E F G 5σ

ACCEPTABLE REGION
¼Q
H I J K L

.2√Q .4√Q .6√Q .8√Q √Q 1.2√Q 1.4√Q

σ

Figure B8-D. OC Curve for Test Method 8

The probability of acceptance relative to any point is equal to the compound probability of
passing the percentile test relative to μ1 (Test A1) and passing the percentile test relative to
MMax (Test B1 or B2).

Let PA1, PB1 and PB2 be the probability of passing test A1, B1 and B2 respectively for any
given unique combination of θ and σ (a particular point). PA1, PB1 and PB2 may be determined
by calculating YA1, YB1 and YB2 from the following equations:

Q (1 − C)
YA1 = (Equation B-48)
K

ln M Max − CQ
YB1 = YB 2 = (Equation B-49)
K Q

and entering FIGURE B-8E. Probability of Passing Test A

(for Test A1) with the calculated value of YA1 and FIGURE B-8F. Probability of Passing
Test B (for Test B1) or FIGURE B-8G. Probability of Passing Test B2

(for Test B2) with the calculated value of YB1 or YB2. The corresponding value of probability
of acceptance PA1 and PB1 or PB2 (whichever of the B tests are appropriate) is read from each
figure and PA1 and the appropriate PB1 or PB2 value are multiplied. The result of this
multiplication is the probability of acceptance of a unit having a particular θ and σ
characteristic defined by (C) and (K).

Issue 1 Page 33
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

0.9

0.8

0.7

Probability
0.6

0.5

0.4

0.3

0.2

0.1
0 1 2 3

YA1

FIGURE B-8E. Probability of Passing Test A

0.9

0.8

0.7

0.6
Probability

0.5

0.4

0.3

0.2

0.1

0
0 1 2 3

YB1

FIGURE B-8F. Probability of Passing Test B1

Page 34
 Applied R&M Manual for Defence Systems
Part D Supporting Theory

0.9

0.8

0.7

0.6

Probability
0.5

0.4

0.3

0.2

0.1

0
0 1 2 3

YB2

FIGURE B-8G. Probability of Passing Test B2

Repeating the above for a number of points, as in FIGURE B-9. OC Map Relative to a Given
Dual Requirement, defines an operating characteristic map relative to a given dual
requirement. Note that probabilities of acceptance always decrease as the point is located
upward or to the right and always increase as the point in consideration is located downward
or to the left on the figure. Hence, sufficient knowledge of test characteristics can be
generated by evaluating relatively few points.

1¼ Q
μ1 = 30
POINTS FOR WHICH
Mmax = 60 at 95th percentile
PROBABILITY OF ACCEPTANCE lnμ1 = Q
WILL BE DETERMINED
Q

POINT A POINT B POINT C

¾Q
lnμ
(1) (.55) (.02) 1 =θ
θ +.
5σ 2
½Q
D E F G
(1) (1) (.64) (.6)

ACCEPTABLE REGION
¼Q
H I J K L
(1) (1) (1) (.62) (.17)

.2√Q .4√Q .6√Q .8√Q √Q 1.2√Q 1.4√Q

Sigma (σ)

FIGURE B-9. OC Map Relative to a Given Dual Requirement

Issue 1 Page 35
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
6.1.1.1.10 TEST METHOD 9: Test for Mean Maintenance Time (Corrective,
Preventive, Combination of Corrective and Preventive) and Mmax. This method is
applicable to demonstration of the following indices of maintainability: Mean Corrective
Maintenance Time (μc), Mean Preventive Maintenance Time (μpm), Mean Maintenance Time
(includes preventive and corrective maintenance actions) (μp/c) and MMax (percentile of repair
time).

CONDITIONS OF USE - The procedures of this method for demonstration of μc, are based
on the Central Limit Theorem. No information relative to the variance (d2) of maintenance
times is required. It may therefore be applied whatever the form of the underlying
distribution, provided the sample size is adequate. The maximum sample size is set at 30.
The actual sample size (if greater than 30 are required) must be determined for each
equipment to be demonstrated, and is usually approved by the procuring activity.

Note: The procedure of this method for demonstrating MMaxC is valid for those cases
where the underlying distribution of corrective maintenance task times is
lognormal.

QUANTITATIVE REQUIREMENTS - Application of this plan requires identification of the

index or indices of interest and specification of quantitative requirements for each. When
demonstration involves μc or μpm, or a combination of both, consumer's risks need to be
specified. When demonstration involves M max , the percentile point which defines the
c
specified value of M max is specified. A minimum sample size of 30 corrective maintenance
c
tasks is required for demonstration of corrective maintenance indices. A minimum sample of
30 preventive maintenance tasks is required where demonstration of preventive maintenance
indices by sampling is permitted and is to be accomplished by this method.

TASK SELECTION AND PERFORMANCE - Sample tasks are selected in accordance with
the stratification procedures outlined in Section 3.5.2. The duration of each is recorded and
used to compute the following statistics:
nc
∑ X ci
X c = i=1
nc

n pm

∑ X pmi
i =1
X pm =
n pm

D t = f c X c + f pm X pm

f c X c + f pm X pm
Xp/c =
f c + f pm

Page 36
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
⎡ 2 ⎤
⎢ ⎛ nc ⎞ ⎥
⎜ ∑ ln X ⎟
⎢ nc nc ⎜ ci ⎟ ⎥
2 ⎝ i =1 ⎠
⎢ ∑ ln X c
i ∑ (ln X ci ) − ⎥
⎢ i =1 nc ⎥
M' max = Antilog ⎢ i=1 +ψ ⎥
c nc n c −1
⎢ ⎥
⎢ ⎥
⎢ ⎥
⎢⎣ ⎥⎦

where the Antilog is taken to the Base e and where ψ is the value of the independent variable
lognormal function which corresponds to the percentile point at which M Max has been
c
established. For the two most common percentile points, 90% and 95%, ψ is 1.282 and 1.645
respectively.

ACCEPT/REJECT CRITERIA - A table of the normal distribution function is consulted for

values of φ (for a one-tailed test) which corresponds to the specified level of consumer risk β.
Table XIII provides values of φ which correspond to the most commonly used values of β.

TABLE B-XIII. vs.

φ β
0.84 20%
1.04 15%
1.28 10%
1.65 5%

Accept/reject criteria is computed for each specified index in accordance with the following.

Test for Mean Corrective Maintenance Time (μc) - The accept/reject value for μc is:

φd̂ c
Xc + d̂ c = standard deviation of sample of corrective maintenance tasks.
nc

φd̂ c
Accept if μc (specified) ≥ X c +
nc

φd̂ c
Reject if μc (specified) < X c +
nc

Test for Mean Preventive Maintenance Time (μpm) - The accept/reject value for μpm is:

φd̂ pm
X pm + d̂ pm = standard deviation of sample of preventive maintenance tasks.
n pm

Issue 1 Page 37
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

φd̂ pm
Accept if μpm (specified) ≥ X pm +
n pm

φd̂ pm
Reject if μpm (specified) < X pm +
n pm

Test for the Mean of all Maintenance Actions (μp/c) - The accept/reject value of μp/c is:

n pm (f c d̂ c ) 2 + n c (f pm d̂ pm ) 2
X p/c + φ
n c n pm (f c + f pm ) 2

n pm (f c d̂ c ) 2 + n c (f pm d̂ pm ) 2
If μp/c (specified) ≥ X p/c + φ . Accept
n c n pm (f c + f pm ) 2

n pm (f c d̂ c ) 2 + n c (f pm d̂ pm ) 2
If μp/c (specified) < X p/c + φ . Reject
n c n pm (f c + f pm ) 2

Test for M Max - The accept/reject value for M Max is:

c c

⎡ 2 ⎤
⎢ ⎛ nc ⎞ ⎥
⎜ ∑ ln X ⎟
⎢ nc nc ⎜ ci ⎟ ⎥
2 ⎝ i =1 ⎠
⎢ ∑ (ln X c )
i ∑ (ln X ci ) − ⎥
⎢ i =1 nc ⎥
M' max = Antilog ⎢ i=1 +ψ ⎥
c nc n c −1
⎢ ⎥
⎢ ⎥
⎢ ⎥
⎢⎣ ⎥⎦

where Antilog is to the Base e.

Accept if M Max (specified) ≥ M 'Max

c c

Reject if M Max (specified) < M 'Max .

c c

6.1.1.1.11 TEST METHOD 10: Tests for Percentiles and Maintenance Time (Corrective
Preventive Maintenance). This method employs a test of proportion to demonstrate
~ ~
achievement of M ct , M pm , M Max and M Max when the distribution of corrective and
c pm
preventive maintenance repair times is unknown.

CONDITIONS OF USE - This method is intended for use in cases where no information is
available on the underlying distribution of maintenance task times. The plan holds the

Page 38
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
confidence level at 75% or 90% as may be desired and requires a minimum sample size (N)
of 50 tasks.

QUANTITATIVE REQUIREMENTS - Application of this method requires specification of

~ ~
M ct , M pm , M Max (95th percentile) or M Max pt (95th percentile) and selection of 75% or
ct
90% confidence level.

TASK SELECTION AND PERFORMANCE - Sample tasks are selected in accordance with
the stratification procedures. The duration of each task will be compared to the required
~ ~
value(s) of the specified index or indices ( M ct , M pm , M Maxct and M Max pm ) and recorded as
greater than or less than each index.

ACCEPT/REJECT CRITERIA - The item under test shall be accepted when the number of
observed task times which exceed the required value of each specified index is less than or
equal to that shown in the Table (B-XIV or B-XV) corresponding to each index for the
specified confidence level.

Test for the Median - Table B-XIV is a test of the median for corrective and preventive
maintenance tasks. The acceptance level is shown for two confidence levels and a sample
size (N) of 50 tasks.
~ ~
TABLE B-XIV 5 Acceptance Table for M ct or M pm ; Sample Size = 50

Confidence Level
75% 90%
Acceptance Level
22 20

Test for M maxc and M Max pm - Table B-XV is a test for M Max c and M Max pm at the 95th
percentile. The acceptance level is shown for two confidence levels and a sample size (N) of
50 tasks.
TABLE B-XV Acceptance Table for M Maxc or M Maxpm ; Sample Size = 50

Confidence Level
75% 90%
Acceptance Level
1 0

5
NOTE: Reference for Tables BXIV and B-XV - “Introduction to Statistical Analysis” by Dixon &
Massey. Page 230. McGraw-Hill Company. 2nd Edition. 1957.

Issue 1 Page 39
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
6.1.1.1.12 TEST METHOD 11: Test for Preventive Maintenance Times. This method
provides for maintainability demonstration when the specified index involves μpm and/or
M Max pm and when all possible preventive maintenance tasks are to be performed.

CONDITIONS OF USE - All possible tasks are to be performed and no allowance need to
made for underlying distribution.

QUANTITATIVE REQUIREMENTS - Application of this plan requires quantitative

specification of the index or indices of interest. In addition, the percentile point defining
M Max pm must be stipulated when M Max pm is of interest.

TASK SELECTION AND PERFORMANCE - All preventive maintenance tasks will be

performed. The total population of PM tasks will be defined by properly weighing each task
in accordance with relative frequency of occurrence as follows: Select the particular task for
which the equipment operating time to task performance is greatest and establish that time as
the reference period. Determine the frequency of occurrence (fpm) of all other tasks during the
reference period, where the frequency of occurrence of a given task is a fractional number, the
frequency shall be set at the nearest integer. The total population of tasks consists of all tasks
with each repeated in accordance with its frequency of occurrence during the reference
period.

ACCEPT/REJECT CRITERIA

Test for μpm - the mean is computed as follows:

k
∑ f pm i (X pm i )
i =1
μpm (Actual) = k
∑ f pm i
i =1

Where: f pm i is the frequency of occurrence of the ith task in the reference period.

K is the number of different PM tasks.

∑ f pmi is the total number of PM tasks in the population.

Accept if: μpm (required) ≥ μpm (actual)

Reject if: μpm (required) < μpm (actual)

Test for M Max pm - The PM tasks shall be ranked by magnitude (lowest to highest value). The
equipment shall be accepted if the magnitude of the task time at the percentile of interest is
equal to or less than the required value of M Max pm .

Page 40
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
LEAFLET 11/2

EXAMPLE MAINTAINABILITY DEMONSTRATION PLAN

7 INTRODUCTION
7.1 This leaflet provides an example of how a maintainability demonstration plan can be
developed for a system.

8 EXAMPLE

8.1 Problem
A new communications system has the following maintainability requirements:

The Mean Time to Restore Service (MTRS) is the time taken to diagnose a fault, repair it and
restore the system to the level of functionality prior to the fault condition. The MTRS at first
line, using a trained maintainer shall not exceed the following values:
a) 20 minutes for system failures.
b) 3 hours for major failures.
c) 5 hours for minor failures.

It is proposed that this value will be verified by a Maintainability Demonstration Test. The
communication system has a high level of BITE and electronic equipment and there is little
confidence in the maintainability prediction carried out in development. Due to the nature of
the equipment there is very little preventive maintenance, and the specification does not
include a preventive maintenance requirement.

8.2 Solution
8.2.1 Test Method

The test method will be based on Test Method 9 of MIL-HDBK-470A. This allows the
statistical test to be valid without the need for any assumptions on the distribution of the
repair times or their variance. The minimum sample size is set at 30 and the acceptance of
the maintainability demonstration test will be deemed to take place when the following
conditions are satisfied:
~
φd c
μc ≥ Xc +
nc

Issue 1 Page 41
Chapter 11, Leaflet 1
Maintainability Demonstration Plans
Where μc = MTRS

nc
∑ X ci
Xc = i =1
nc
Xci = Individual task duration
nc = Number of demonstrated task.
φ = Value corresponding to the specified level of consumers risks
(with a consumer risk of 20%, φ = 0.84).
~
dc = Standard deviation of sample of tasks.

Hence to pass the demonstration it is required that:

~
φd c
μc ≥ Xc + ≤ 20 minutes for system failures.
nc

A consumers risk (β) of 20% will be applied. The duration of each task will be recorded and
used to calculate the following:
a) MTRS - Minor Failure.
b) MTRS - Major Failure.
c) MTRS - System Failure.

8.2.2 Accept/Reject Criteria

To determine the value of Φ which corresponds to the specified level of confidence, a table of
the normal distribution function is consulted. Table 1 provides values of φ which correspond
to the most commonly used values of β.

φ β
0.84 20%
1.04 15%
1.28 10%
1.65 5%

Table 5: Versus

~
φd c
μc ≥ Xc + ≤ 3 hours for major failures
nc
≤ 5 hours for minor failures.

Page 42
 Applied R&M Manual for Defence Systems
Part D Supporting Theory
CHAPTER 11/3

MAINTAINABILITY DEMONSTRATION PLANS

Due to the quantity of graphical content contained within this document it is not
possible at present to display Chapter 11 as a web page. Please use the link
opposite.

Issue 1 Page 43
Chapter 11, Leaflet 1
Maintainability Demonstration Plans

Page 44