THE INTERNATIONAL JOURNAL OF AVIATION PSYCHOLOGY, 20(1), 25–47

Copyright © 2010 Taylor & Francis Group, LLC

ISSN: 1050-8414 print / 1532-7108 online
DOI: 10.1080/10508410903416136

Language Error in Aviation

Maintenance: Quantifying the Issues
and Interventions in Four World Regions
Jiao Ma,1 Colin G. Drury2, and Clara V. Marin2
1Department of Aviation Science

Saint Louis University, St. Louis, Missouri

2Department of Industrial and Systems Engineering

University at Buffalo, SUNY, Buffalo, New York

There has been a great increase in contract maintenance among major airlines, to a
current level of about 50%, with the fastest growing segment of the worldwide main-
tenance, repair, and overhaul (MRO) market outside the United States. Although
English is the language of aviation, it is certainly not the native language of most of
the world. This study assisted the Federal Aviation Administration (FAA) in estab-
lishing a method for determining whether language barriers result in maintenance de-
ficiencies. This article examines language error broadly before quantifying the issues
and potential interventions for one class of language errors, those related to written
documentation. Two studies of language errors in aviation maintenance collected
data from 941 participants at 19 maintenance sites in 4 world regions: Asia, Latin
America, Europe, and the United States. One survey was administered to assess the
incidence of seven language error scenarios, and the factors leading to such errors
and their detection. Language errors were found to be relatively common but largely
detected before they could propagate through the maintenance system. An interven-
tion test was conducted on the same sample to quantify the effectiveness of language
error interventions, including use of European Association of Aerospace Manufac-
turers (AECMA) Simplified English, translation into the native language, use of an
English-speaking coach, and provision of a local language glossary. The quantitative
recommendations to both MROs and regulatory bodies for the effective reduction of
language errors included the use of translation and language training as the only two
effective interventions.

Correspondence should be sent to Jiao Ma, Department of Aviation Science, Saint Louis Univer-
sity, 3450 Lindell Boulevard, St. Louis, MO 63013. E-mail: [email protected]
26 MA, DRURY, & MARIN

In June 2005, the Federal Aviation Administration (FAA) Office of the Inspec-
tor General reported that the country’s air carrier industry is in an era of transition
(FAA, 2005b). Part of the cause of the transition is record financial losses for Fed-
eral Aviation Regulations (FAR) Part 121 carriers, and part of the solution is seen
as contract maintenance, popularly known as “outsourcing.” The volume of con-
tract maintenance has been increasing each year, with the percentages for Part 121
operators now exceeding 50%. For comparison, the percentage of outsourcing in
1996 was 37% (FAA, 2003).
In aviation, contract maintenance has been advocated and widely used, as it
avoids tying up capital in maintenance facilities, and can reduce costs by opening
the airline’s maintenance operation to outside competition. The formation of
global maintenance, repair, and overhaul (MRO) networks involves U.S. and for-
eign airlines, as well as repair stations (Sparaco, 2002). In addition to offshore
MROs, there are many within the United States where non-native English speakers
(NNESs) form part of the labor pool. The difficulty of moving between languages
creates an additional potential for error. The language of aviation is primarily Eng-
lish, both in flight operations and in maintenance. However, it is certainly not the
native language of most of the world (Weber, 1997). Aviation maintenance techni-
cians (AMTs) must pass their examinations in English, and maintenance docu-
mentation in use at the FAA-approved facilities is in English (FAA, 2005a). This
poses a second-language or translation burden for NNESs that can potentially in-
crease their workload, their performance time, their error rate, or even all three
measures.
In a 2001 report to the Secretary of Transportation by the Aircraft Repair and
Maintenance Advisory Committee (ARMAC), many of these issues were raised in
considering changes to the domestic and foreign FAR Part 145 (ARMAC, 2001).
They recommended that “the FAA should establish a method for determining
whether language barriers result in maintenance deficiencies.”
This article reports studies that were performed in direct response to these con-
cerns that NNESs, in repair stations in the United States and abroad, might be
prone to an increased error rate that could potentially affect airworthiness. It
started from an overall examination of the roles of communication and language in
aviation, using a series of preliminary studies before presenting two main studies
of maintenance language errors, concentrating on written documentation.

PRELIMINARY STUDIES

Before conducting our own study to quantify levels and characteristics of language
errors and measure the effectiveness of potential interventions, we analyzed 684
reports of aviation errors (not just in maintenance) from the FAA/NASA Aviation
Safety Reporting System (ASRS) and the Accident/Incident Data System (AIDS),
 LANGUAGE ERROR IN AVIATION MAINTENANCE 27

to search for patterns of language errors. This analysis led to a communication

model for language errors and the need for more quantitative data on this potential
problem (Drury & Ma, 2003). We were also able to obtain survey data from an air-
frame manufacturer that gave reported levels of English ability and intervention
across many airlines worldwide. We then conducted focus groups to determine
language error patterns and how language errors were mitigated at MROs.

Models of Communication
Communication is defined as “a dynamic and irreversible process by which we en-
gage and interpret messages within a given situation or context, and it reveals the
dynamic nature of relationships and organizations” (Rifkind, 1996). An important
distinction made in communication theory is the temporal aspect: Communication
is either synchronous or asynchronous. In aviation maintenance, synchronous
communication is typically verbal, for example, via conversations or public ad-
dress announcements (e.g., on emergency procedures for evacuations or important
notifications related to plane parts; often in English, especially in bilingual coun-
tries), and asynchronous communication is typically written, (e.g., work documen-
tation or placards). In the context of aviation maintenance and inspection, commu-
nication (where the human factors movement began in the early 1990s) has been
the most frequent aspect studied (Taylor & Patankar, 2000).
The fundamental function of communication is to deliver a message from one
human being to another. In almost every aspect of aviation work, communication
also fulfills a secondary role as an enabler (or tool) that makes it possible to accom-
plish a piece of work (Kanki & Smith, 2001). Figure 1 presents a communication
model we (Drury & Ma, 2003) synthesized from our literature review (Johnson,
1972; McAuley, 1979; Wideman, 2002, etc.) to help guide in design and interpre-
tation of our studies.

FIGURE 1 The communication model synthesized from literature.

28 MA, DRURY, & MARIN

Based on basic communication theories, a communication process is com-

posed of the sender/receiver (e.g., people, manuals, computers, etc.), the mes-
sage (e.g., information, emotions, questions, etc.), the medium (e.g., speech,
text, sensory, etc.), filters or barriers, feedback, and so on (Griffith, 1999; Kanki
& Smith, 2001).
Fegyveresi (1997) summarized many variables that influence communication,
such as workload, fatigue, personality traits, gender bias, standard phraseology,
experience level, vocal cues, and so on. Language and cultural diversity can inten-
sify differences and confusions in communication: For example, when sender or
receiver is not native or most masterful at the aviation maintenance language (or
both are not), the “encoding” and “decoding” processes can be potentially dis-
torted, and feedback might not be effectively and efficiently delivered, received,
and processed. However, research has found that a language barrier does not nec-
essarily result in unsafe aviation operations (e.g., in cockpit, Merritt & Ratwatte,
1997). From the model in Figure 1 we would expect that NNESs would have addi-
tional cognitive burdens, particularly for decoding as receivers of communication,
leading to a higher error potential. The communications channel in Figure 1 repre-
sents the document or other media used to convey the message, suggesting docu-
mentation design as an important intervention. Finally, the complexity of the no-
tion itself could be an error-sensitive factor, with complex ideas having greater
opportunities for error. To eliminate, or at least minimize, potential ambiguities
and other variances, people establish rules regarding which words, phrases, or
other elements will be used for communication, their meaning, and the way they
will be connected with one another. The aggregation of these rules is known as a
“protocol.” There are four types of protocol related to flight and aircraft safety
(Rifkind, 1996): verbal, written, graphical, and gestural protocols. According to
Rifkind (1996), the only verbal protocol that has been established throughout avia-
tion, including maintenance, is the use of English as the standard language. This
was done when the International Civil Aviation Organization (ICAO) was estab-
lished in 1944.

Original Equipment Manufacturer Survey of Airlines

We began quantification of the language error issues through access to data from a
survey of airline practices. From September 2002 to January 2003, an international
original equipment manufacturer (OEM) surveyed a large number of airlines (N =
113) throughout the world concerning their use of English and other languages in
flight operations and maintenance. The data were approximately evenly divided
between North America (n = 35), Europe (n = 16), Asia (n = 30), and the rest of
the world (n = 32). For current purposes we only analyzed the questions pertaining
to maintenance (Drury, Ma, & Marin, 2005).
 LANGUAGE ERROR IN AVIATION MAINTENANCE 29

Chi-square tests of each of three reported levels (i.e., any documents, mainte-
nance documents only, very little reading capability) showed a significant differ-
ence in English reading abilities between the two groups (Europe and North Amer-
ica vs. Asia and other), with chi-square > 7.0 for all levels (p < .001). The English
speaking ability of maintenance personnel also varied significantly among re-
gions. For the four levels (i.e., complex conversation, simple conversation, few
phrases, very little speaking capability), all the chi-square values exceeded 4.5 (p <
.005). Note again the contrast between (a) Europe and North America, where most
of the mechanics speak English, and (b) Asia and other, where there is less re-
ported speaking ability.
As written documentation is the primary control system for aircraft mainte-
nance, we expected that those airlines reporting low levels of English reading abil-
ity (n = 8 in Asia) would adopt some mitigating strategies in using the original doc-
uments (i.e., modification into European Association of Aerospace Manufacturers
[AECMA] Simplified English, translation into their native language). However,
for the maintenance manual, seven out of eight airlines kept the original OEM doc-
ument in English without any modification or translation, and only one airline
modified or rewrote it in English. For the structural maintenance manual, once
again the overwhelming majority (six out of eight) used the original English docu-
ments. For those airlines with a low level of English speaking ability, almost all
conducted onsite maintenance training and meetings in a language other than Eng-
lish (i.e., the native language); none of the airlines used English during casual talk-
ing. This might represent a mismatch to documentation used in the same task that
typically remained in English.

Focus Groups on Language Errors

Although the analyses of archival data from the OEM survey provided some in-
sight into language errors in maintenance, such data were not collected for that
purpose (cf. Drury, 1995). To collect valid information on the types of errors in-
volved, we conducted five focus groups, three at U.S.-based MROs and the other
two at U.K.-based MROs where NNESs form part of the labor pool (Drury & Ma,
2004).
The focus groups provided ample evidence that language errors exist, but also
that recovery mechanisms and mitigating factors are possible. Although documen-
tation was an important source of difficulty, there were other patterns in verbal
communication, including unexpected ones of regional accents of native English
speakers. The focus group study was also able to further document the time course
and propagation of errors, including error detection points and interventions.
Seven patterns of error scenarios in both verbal (synchronous) and written (asyn-
chronous) communication were summarized from the focus groups and are pre-
sented in the next section.
30 MA, DRURY, & MARIN

METHODOLOGY FOR LANGUAGE ERROR INCIDENCE

AND MITIGATION STUDY

From the accumulated evidence, it was clear that language errors do exist in both
U.S. and foreign MROs, and that the levels of English ability differed widely
across different world regions. However, there were no quantitative data on inci-
dence of language errors, actual (i.e., not just reported) levels of English ability, or
effectiveness of any of the various strategies used by MROs to reduce the inci-
dence or impact of language errors. These findings led to the two jointly conducted
main studies reported next, one to measure language error incidence and character-
istics, and the other (using the same sample of 941 participants) to measure the ef-
fectiveness of interventions.

Study 1: Language Error Incidence and Characteristics

Survey
A survey was administered to determine the relative prevalence of the following
seven scenarios derived from the focus groups:

Scenario 1: The AMT or inspector was not able to communicate verbally to the
level required for adequate performance.
Scenario 2: The AMT or inspector and the person to whom they were speaking
did not realize that the other had limited English ability.
Scenario 3: Native English speakers with different regional accents did not un-
derstand each others’ communications.
Scenario 4: The AMT or inspector did not understand a safety announcement
over the public address system.
Scenario 5: The AMT or inspector did not fully understand a safety placard.
Scenario 6: The AMT or inspector did not fully understand documentation in
English, for example a work card or a manual.
Scenario 7: The AMT or inspector did not fully understand a document trans-
lated from another language into their native language.

The first four were classified as synchronous communication, whereas the final
three were asynchronous. We would suspect that these seven differed in incidence
(e.g., Scenario 4 might not be that common) as well as in severity of outcomes. For
each of these seven scenarios the incidence questionnaire (available from the au-
thors upon request) asked if each had ever been encountered, and when the sce-
nario last occurred. The participants were asked to check the factors associated
with increased likelihood of the error occurring (9 likelihood factors), with miti-
gating each error (10 prevention factors), and with the discovery of each error (6
discovery factors).
 LANGUAGE ERROR IN AVIATION MAINTENANCE 31

In world regions where English is not the official language, participants were
offered a choice between English or translated versions (i.e., Chinese or Spanish)
of the questionnaire with instructions provided in either English or the native lan-
guages accordingly. In bilingual areas or countries (e.g., Hong Kong, Puerto
Rico), instructions were offered in English. In addition, researchers were always
present to clarify any misunderstanding related to the questionnaire (Drury et al.,
2005).

Study 2: Intervention Effectiveness Test

As noted earlier, written documentation is the basis for control in aviation mainte-
nance. It acts as a production control system for assigning tasks to AMTs, as a
quality control system that specifies when and how work is inspected, and as an
oversight control system that can be audited by the appropriate regulatory author-
ity. With a focus on asynchronous language error reduction—reducing potential
documentation errors—we adopted a methodology of comprehension tests of task
cards (e.g., Chervak, Drury, & Ouellette, 1996; Drury, Wenner, & Kritkausky,
1999) to quantify the effectiveness of language error interventions. Task cards
(also known as work cards) were chosen because they are universally used in avia-
tion maintenance as job aids, designed to control the work directly at the job site by
giving detailed work instructions and requiring signatures (sign-offs) for each step
completed.
We selected two task cards, one “easy” and one “difficult,” from four task cards
used in our previous research (Drury, Guy, Ma, & Wenner, in press), because task
difficulty was found to affect the effectiveness of one intervention strategy,
AECMA Simplified English. The selected easy and difficult task cards cover two
common aviation maintenance tasks, respectively: (a) change potable water tank
compressor inlet air filter cartridge and portable water engine bleed air filter ele-
ment (see Appendix for example); and (b) functionally check hydraulic system
reservoir pressurization check valve for proper operation. The complexity of these
task cards was evaluated by computational linguists at an airframe manufacturer
and University of Washington technical communications researchers using four
measures: word count, words per sentence, percentage passive voice, and the
Flesch–Kincaid reading score. Chervak and Drury (2003) found that the use of
Simplified English had a larger effect on more complex task cards. Both of the task
cards were then prepared in the Simplified English versions, which were critiqued
by the same experts and by the American Institute of Aeronautics and Astronautics
(AIAA) Simplified English Committee.
From the OEM survey, we learned that some organizations translate documents
into the native language of the employees. For other interventions, the focus
groups revealed that a more senior person (e.g., lead, foreman, and engineer) often
assumed a role of English coaching in operations. AMTs often use glossaries of
32 MA, DRURY, & MARIN

English and native language words pertaining to aviation maintenance. We wished

to examine these three managerial practices of interventions through experimen-
tation. After data were collected in the first world region, we introduced another
intervention of partial translation, which refers to translation of everything on the
document except technical terms, an intervention widely observed on the hangar
floor. Referring to the communication model in Figure 1, all of these potential in-
terventions can be perceived as different ways of specifically addressing language
issues and improving the communication medium or interface to reduce noise, dis-
tortion, and disruption.
A 10-item questionnaire (available from the authors on request) was used to test
comprehension of each task card. The comprehension was scored by the number of
correct responses, with time taken to complete the questionnaire as an additional
measure. The usability of each task card was evaluated using 15 rating scales
(available from the authors upon request).
In summary, a fully nested (between participants) 2 × 2 × 5 design was used
with factors as follows:

• Task card complexity: Simple or complex.

• Task card language: Simplified English or not simplified English.
• Language intervention: No intervention (English), English with glossary,
English with coach, full native language translation, or partial native lan-
guage translation.

Demographic data were collected on age, gender, job category, and years as an
AMT. We also collected language-related data to characterize the population of
AMTs and to provide potential covariates for analyses of intervention effective-
ness. These were years studying English and English reading grade level measured
by the Accuracy Level Test, which produces a value on the scale of reading grade
level, normed on U.S. public schools (Carver, 1987). It has been validated against
more detailed measures of reading level (see Chervak et al., 1996).
With all of the data collection English and translated packets prepared, we per-
formed two pilot experiments: one with 15 English-speaking maintenance person-
nel in the United Kingdom and the United States (Drury & Ma, 2003), and the
other with 40 native Chinese engineering students in the United States (Drury &
Ma, 2004).

Choice of Participants and Sites for Studies 1 and 2

It is important to have data collection in several regions of the world, especially be-
cause the OEM survey indicated that those in Asia and other countries had quite
different responses from those in North America and Europe. Previous studies
(Phillips, 2004; Seidenman & Spanovich, 2004) suggest that Asia and Latin Amer-
 LANGUAGE ERROR IN AVIATION MAINTENANCE 33

ica are the most frequent regions for contract maintenance. The major world lan-
guages are (in order) Chinese, Spanish, and English (Weber, 1997), so that our
choice of regions was made as Asia and Latin America, with one Spanish-speaking
country from Europe and a control sample from the United States.

Asia. Almost half of the top 10 Asian MROs are located in China. By choos-
ing MROs in mainland China, Hong Kong, and Taiwan, we were able to provide a
reasonably consistent language use, and also to include an area (Hong Kong) with
a tradition of English–Chinese bilingualism.

Latin America. Within Latin America, we chose Mexico as it has a large

concentration of full-service MRO sites, plus Colombia, and Argentina as areas
with much growth potential, and Puerto Rico as an officially bilingual area for
comparison with traditionally Spanish-only countries.

Europe. We chose Spain as we could compare the use of the same language,
Spanish, to our sample of Latin American countries.

United States. A control group from the United States was chosen to pro-
vide a baseline comparison for the other regions. The MRO with whom we worked
had a number of sites in the United States, of which we chose two in the Midwest
for convenience and for an expected low fraction of NNESs.

Data Collection Process for Studies 1 and 2

We collected data from 941 participants by visiting 19 MRO sites in the four world
regions already discussed. At each MRO site, an initial meeting with management
was used to explain verbally the objectives and conduct of the study, as a supple-
ment to our earlier written communications. At this meeting, we also discussed the
type of work at the site, the range of customers served, and the importance of lan-
guage issues and errors. Agreement was reached on the types of participants (e.g.,
AMTs, engineers, quality assurance personnel, managers). The company then
scheduled multiple participants to attend the studies in a secluded conference room
at approximately 75-min intervals. The participants were volunteers in the experi-
ment, but only after they had been assigned to attend by their managers.
Groups of participants were nominally of 6 people, but groups with between 2
and 10 participants were encountered. Using English first and then the local lan-
guage, each group of participants was welcomed, and the general objective of the
data collection (i.e., to understand language errors and how to reduce them) was
communicated. After giving informed consent and completing demographic ques-
tions, participants were given the Accuracy Level Test, starting the 10-min timed
test at the same time. The participants next started the task card comprehension test
34 MA, DRURY, & MARIN

(i.e., Study 2) at the same time. The participants were given one of the four task
cards and its associated comprehension 10-item questionnaires in rotation. They
were timed, but instructions emphasized accuracy. When this task had been com-
pleted, each participant was given the usability rating form. The participants were
informed that not all people in the room were getting the same task card, or the
same intervention condition. On a couple of occasions, a participant did not even
attempt the task in one of the first three intervention conditions because he did not
read English. In those few cases, the response was noted and the participant was
given the equivalent full translation condition.
The participants were then given the (untimed) seven-scenario Language Error
Incidence and Characteristics Survey (i.e., Study 1). Occasionally, slower partici-
pants were asked to take that questionnaire back to their workplace and return it on
completion.

INTEGRATION AND DISCUSSION OF RESULTS

We visited 19 MRO sites in four world regions. A general description of the char-
acteristics of each of the sites is presented in Table 1. All of the sites in mainland

TABLE 1
Background Information on the Maintenance, Repair, and Overhauls

Approximate No. Style of Using Task

Regions Area Site No. of Employees Card in Maintenance

Asia Mainland China 2-1 > 3,500 English-Chinese

Mainland China 2-2 1,500 English-Chinese
Hong Kong 2-3 600 English
Hong Kong 2-4 2,500 English
Mainland China 2-5 2,000 English-Chinese
Mainland China 2-6 800 English-Chinese
Taiwan 2-7 300 English
Taiwan 2-8 1,300 English
Taiwan 2-9 < 100 English
Latin America Mexico 3-1 900 English
Mexico 3-2 800 English
Mexico 3-3 1,200 English
Mexico 3-4 < 100 English
Puerto Rico 3-5 < 100 English
Colombia 3-6 1,300 English
Argentina 3-7 200 English
Europe Spain 4-1 3,800 English
North America United States 1-1 400 English
United States 1-2 700 English
 LANGUAGE ERROR IN AVIATION MAINTENANCE 35

China used a mixture of English and translated Chinese documentation in their

daily operations, and in the rest of the world only English documentation was used.

Demographic Comparisons
For Asia and Latin America, one-way analyses of variance (ANOVAs) were con-
ducted of each demographic, except for the categorical variable of gender that was
tested using chi-square. All comparisons gave significant differences by area in
Asia, and only gender was not significantly different across sites in Latin America,
as shown in Table 2. Our main concern, however, was for English reading levels.
Within each region the reading grade levels of NNESs were typically 4.5 to 5.5
for the samples tested. Higher levels were found where the countries or areas had a
history of bilingualism in English: Puerto Rico in Latin America (10.0) and Hong
Kong in Asia (6.6). In the United States and the United Kingdom for comparison,
reading grade levels were very high, about 14, as has been found in earlier studies
of AMTs (e.g., Drury et al., 1999). Grade levels 5 and 6 of English reflect an of-
ten-stated aim of documentation to be written for a “sixth-grade level,” although
such a recommendation was never meant to apply specifically to aviation mainte-
nance English.

Results From Study 1: Language Error Incidence

and Characteristics Survey
The seven scenarios in the incident survey were found to be well supported in all
regions (see Figure 2). There were differences in reporting these errors across the
countries, as shown in the detailed analysis of each region, but consistency across
countries or areas in each region was high.
A Friedman test of differences between scenario frequencies for the four re-
gions showed a highly significant difference between scenarios, S(6) = 18.9, p =
.004; that is, substantial agreement across regions. Three scenarios gave high fre-
quencies: Scenarios 1, 2, and 6 representing inadequate verbal abilities, mis-
perceived abilities, and misunderstanding English documents, respectively. Those
scenarios were the ones associated with direct communication surrounding the
work itself. All three of these had reported return frequencies between 4 and 10
times per year, and reflected imperfect written communication (work documents)
or imperfect verbal communication. Examples of scenarios collected from our fo-
cus groups confirmed these findings without adding any new scenarios.
Factors seen as influencing scenario incidence also had a large measure of
agreement across regions. For error likelihood factors, a Friedman test similar to
the one for scenario incidence was also highly significant, S(8) = 21.3, p = .006,
showing high agreement on the relative importance of these factors. There was a
consistent group of four highly rated factors:
36
TABLE 2
Demographics of the 19 Maintenance, Repair, and Overhaul Sites, With Mean Values and Test Results

North
Asia Latin America Europe America

Mainland Hong Puerto United

China Kong Taiwan Chi-Square, p Mexico Rico Colombia Argentina Chi-Square, p Spain States

No. tested 175 25 54 / 250 25 86 141 / 86 99

Percent female 25%* 4% 4% χ2(2) = 15.84** 4.0% 0 7.0% 2.9% χ2(3) = 4.2, 2.3% 1%
ns
Age 33.5* 42.9 40.5 F(2, 250) = 37.3* 35.6 34.6 31.8* F(3, 494) = 37.9 43.4
9.69** 9.69**
Years as aviation 8.6* 18.4* 13.6* F(2, 250) = 12.1 11.2 10.1 10.5 F(3, 468) = 13.0 17.4
maintenance 21.9** 1.44**
technician
Years learning 20.1* 35.6* 27.1* F(2, 243) = 3.5 9.5* 3.3 4.4 F(3, 363) = 8.0 N/A
English 79.9** 10.38**
Reading level 4.9* 6.6 5.8 F(2, 253) = 5.6 10.0* 4.9 4.8 F(3, 498) = 5.2 14.3
7.9** 27.63**

*Signifies a mean value different from the others at p < .05 on the post hoc Tukey test, or standardized residuals test for chi-square.
**p < .001.
 LANGUAGE ERROR IN AVIATION MAINTENANCE 37

FIGURE 2 Mean frequency with which each scenario was encountered by region.

• The AMT or inspector has inadequate written English ability.

• The AMT or inspector has inadequate verbal English ability.
• The task instructions are complex.
• Time pressure makes the AMT or inspector hurry.

The first two are connected to the individual performing the task; the third is a
function of the documentation, and the final factor is part of the social environment
of maintenance.
Prevention factors showed a similar pattern. Again, the Friedman test gave sig-
nificant factor differences across regions, S(9) = 22.5, p = .007. The five most fre-
quently cited factors that could prevent a language error were:

• The AMT or inspector is familiar with this particular job.

• The document follows good design practice.
• The document is translated into the native language of the AMT or inspector.
• The document uses terminology consistent with other documents.
• The AMT or inspector uses the aircraft as a communication device, for ex-
ample, to show the area to be inspected.

Error discovery factors were also consistent across regions, S(5) = 18.3, p =
.003, with just two emerging as highly reported:

• The AMT or inspector asked for assistance or clarification.

• The AMT or inspector appeared perplexed.
38 MA, DRURY, & MARIN

Referring to the communication model in Figure 1, both of the preceding factors

are indicative of feedback from the message recipient to the message sender. They
also occur very early in the process: Detection of language errors is typically re-
ported well before any maintenance or inspection errors have been committed, or the
aircraft is released for service. The typical picture arising across all of the measures
is that language errors of many types are possible, although only a few are frequent,
with a language error-prone activity having consistent characteristics:

• Complex task instructions.

• Poorly designed document, in English.
• Users with low ability in English and low familiarity with the task to be
performed.
• Time pressure to complete the task.

When listed in this way, language errors appear to have all of the usual human
factors antecedents of error, not just language error. All of these, apart from low
ability in English, can be found in classic human factors treatments, such as
Wickens and Hollands (2000), as well as those specifically directed at aviation or
aviation maintenance (e.g., Garland, Wise, & Hopkin, 1999; Maddox, 1998; Rea-
son & Hobbs, 2003). The implication is that if the “usual” error-shaping factors are
present, then the “usual” interventions should be effective, such as training (Tay-
lor, 1993), documentation design (Drury & Sarac, 1997), and organization design
(Reason, 1997; Taylor & Felten, 1993).

Results From Study 2: Intervention Effectiveness Test

As Table 3 shows, direct measurement of intervention effectiveness produced a
pattern of significant results largely consistent across interventions, regions, and
task cards; that is, interactions were largely absent, making interpretation simpler.
Two patterns are important for interventions’ effectiveness: the covariate, which
could have implications for selection and training, and the interventions them-
selves, which modify the task rather than the user.
For covariates, as expected, reading grade level and age were highly significant
covariates across all measures. Younger participants and those with better English
reading skills performed better, as has been seen in other studies of document com-
prehension (Chervak & Drury, 2003; Drury, Wenner, & Kritkausky, 2000). Such
results now extend to a nonnative English-speaking population. The significant
Reading Grade Level x Comprehension Score correlations show that increasing
mastery of English had a significant impact on comprehension and is a vindication
of the English language training programs invested in by many of the MROs we
visited.
 TABLE 3
Summary of Analysis of Variance Results for Intervention Performance

Accuracy/
Accuracy Time Time Loge(Time)

Asia Intervention F(3, 232) = 6.1, p = .001 F(3, 232) = 5.9, p = .001
Country F(2, 232) = 13.9, p < .001 F(2, 232) = 13.9, p < .001 F(2, 232) = 14.9, p < .001
Task card F(1, 232) = 6.2, p = .014 F(1, 232) = 7.1, p = .008
Simplified English
Reading level (covariate) F(1, 232) = 22.3, p < .001 F(1, 232) = 9.3, p = .003 F(1, 232) = 18.7, p < .001 F(1, 232) = 7.5, p = .007
Age (covariate) F(1, 232) = 17.4, p < .001 F(1, 232) = 11.7, p = .001 F(1, 232) = 17.1, p < .001 F(1, 232) = 9.7, p = .002
Latin Intervention
America Country F(3, 463) = 9.17, p < .001 F(3, 461) = 5.1, p = .002 F(3, 461) = 4.5, p = .004 F(3, 461) = 10.5, p < .001
Task card F(1, 461) = 5.2, p = .023 F(1, 461) = 6.0, p = .015
Task Card × Intervention F(4, 463) = 2.4, p = .021 F(5, 461) = 2.7, p = .032
Task Card × Simplified F(1, 461) = 5.5, p = .020 F(1, 461) = 4.7, p = .031 F(1, 461) = 5.6, p = .018
English
Reading level F(1, 463) = 258, p < .001 F(1, 461) = 106, p < .001 F(1, 461) = 121, p < .001 F(1, 461) = 367, p < .001
(covariate)
Age (covariate) F(1, 463) = 22.0, p < .001 F(1, 461) = 7.0, p < .001 F(1, 461) = 8.5, p = .004 F(1, 461) = 29.2, p < .001
Europe Intervention F(4, 72) = 2.3, p = .064
Reading level (covariate) F(1, 72) = 4.7, p = .033 F(1, 72) = 8.9, p = .004 F(1, 72) = 9.3, p = .003 F(1, 72) = 10.4, p = .002
Age (covariate) F(1, 72) = 3.3, p = .072 F(1, 72) = 3.5, p = .066 F(1, 72) = 3.8, p = .054

North Task card F(1, 84) = 14.2, p < .001 F(1, 84) = 14.8, p < .001 F(1, 84) = 7.6, p = .007
America MRO Site × Simplified F(1, 84) = 5.7, p = .020 F(1, 84) = 4.4, p = .039
English
Reading level (covariate) F(1, 84) = 30.3, p < .001 F(1, 84) = 8.5, p = .005 F(1, 84) = 9.1, p = .006 F(1, 84) = 25.3, p < .001
Age (covariate) F(1, 84) = 5.8, p = .018 F(1, 84) = 10.3, p = .002 F(1, 84) = 8.0, p = .006 F(1, 84) = 9.5, p = .003

39
Note. MRO = maintenance, repair, and overhaul.
40 MA, DRURY, & MARIN

Intervention effectiveness, measured by comprehension performance, was large-

ly unaffected by anything except some form of task card translation. Surprisingly,
Simplified English had no consistent effect, in contrast to our earlier findings that
Simplified English was most effective for NNESs (Chervak & Drury, 2003). That
finding was for NNESs in the United States, so perhaps Simplified English is less
useful when applied in a setting where the native language is something other than
English. This negative finding appeared for both Chinese and Spanish speakers.
Similarly, neither the interventions of a bilingual coach nor a glossary produced
any significant results, despite their widespread use as interventions at MRO sites.
We suspect that at least part of that was due to the fact that very few of the partici-
pants who were given these interventions actually used them during the compre-
hension test. Perhaps people were embarrassed in front of their peers, or did not
want to show “weakness” in front of a data collection team from the United States
with FAA funding. In hangar floor observations, AMTs did discuss their work
with bilingual supervisors and often produced well-worn English-native language
dictionaries. The fact remains that the only consistently significant intervention
was translation.
Figure 3 provides a direct visual comparison of the effects of translation, with
arrows showing changes between baseline condition and translation, in different
countries and areas. No statistical comparison was attempted: Our aim was not to
measure whether one country or area was “better” or “worse” than another but to

FIGURE 3 Changes in accuracy and time from baseline to translated conditions.

 LANGUAGE ERROR IN AVIATION MAINTENANCE 41

integrate the large mass of data across world regions. Figure 3 shows the accuracy
and elapsed time averages across the baseline and translated conditions, respec-
tively, averaged across both easy and difficult task cards. Both Simplified English
and non-Simplified English conditions were also averaged for the baseline data
points, as were full and partial translation conditions. The United States and Hong
Kong did not use translations.
Several points emerge from Figure 3. First, the accuracy of all countries in
North America, Asia, and Europe in the baseline condition was quite compara-
ble, all between about 70% and 80% accurate. (Note that our comprehension
test was quite difficult so that 100% would not be expected based on previous
results from the United States.) Second, in Latin America and Spain, accuracy
was brought up to this same high level by translation, even in Spain where the
accuracy was high anyway. At times this was accompanied by an increase in
performance speed, although at other times it was not. Third, in Asia, partici-
pants opted for constant (and high) accuracy, letting speed suffer when no
translation was provided. That is exactly the response the traveling public and
regulators would like to see. From the intervention effectiveness study, the
conclusion is that translation works as an error control strategy, bringing ac-
curacy performance to about the same level as in the United States. How-
ever, other considerations might be important in choosing translation as an
intervention.

Interpretation via Communications Model

In this study we have a large number of findings from several studies, all pertain-
ing to language error in aviation maintenance. The model in Figure 1 was used to
guide the study design, so it is natural to reexamine the model in light of the find-
ings. Questions of overall incidence of errors are matters for shaping policy, at the
level of the organization or the regulatory body, so that they are model independ-
ent. The fact that we showed that error scenarios exist and were encountered by
many maintenance personnel demonstrates that the topic is worthy of study.
The model does become useful when we examine the individual findings, as we
can use it to postulate the locus of effects and potentially guide interventions. Table 4
summarizes the findings as related to the various parts of the model in Figure 1. The
sender is responsible for the actual notion to be communicated and its encoding, thus
issues of inherent complexity of the notion, and whether it is translated into the native
language both pertain to the sender. The channel in our studies was primarily the task
card document, although it could be argued that translation is an attribute of the docu-
ment. We prefer to leave this aspect under the sender to emphasize management’s role
in error prevention. Many of the findings relate to the receiver of the communication;
that is, the person at the “sharp end” of the error causation tree (Reason & Hobbs,
42 MA, DRURY, & MARIN

TABLE 4
Summary of Findings in Relation to Communications Model

Main Locus Subsidiary Locus Study Finding

Sender Notion 1: Influencing factors Instruction complexity

2: Intervention study Task card complexity
Encode 1: Prevention factors Document translated
2: Intervention study Translated task card
1: Prevention factors Use of consistent terminology
Channel 1: Prevention factors Good document design
Receiver Decode 1: Scenario frequency AMT inadequate verbal abilities
1: Scenario frequency Misperceived abilities (two-way)
1: Influencing factors AMT inadequate written abilities
1: Influencing factors AMT inadequate verbal abilities
2: Intervention study Reading level covariate
Perceived notion 1: Scenario frequency AMT misunderstands document
1: Prevention factors AMT familiarity with job
Feedback 1: Prevention factors Aircraft as communications device
1: Discovery factors AMT asked for clarification
1: Discovery factors AMT appeared perplexed
Not in model 1: Influencing factors Time pressure on AMT

Note. AMT = aviation maintenance technician.

2003). Personal abilities figure strongly in decoding, and the final outcome (perceived
notion) shows where a misunderstanding is finally manifest. Note that this section
contains a two-way communications issue: the sender misperceiving the language
abilities of the receiver. This is partly an issue of poor feedback, but also serves to re-
emphasize how both sender and receiver need valid models of each other to communi-
cate effectively (Haig, Sutton, & Whittington, 2006). Finally, our model is not the ba-
sic transmission model (e.g., Shannon & Weaver, 1949) but incorporates feedback as
do almost all modern communication models. Our significant results for the feedback
locus involve the sender observing the receiver to assess communication effective-
ness, plus the use of the aircraft itself as a highly reliable two-way channel of commu-
nication between sender and receiver.
Overall, the quest for effective interventions finally rests with the management
environment within which the model is seen to operate, although this not specifi-
cally included in our model. Management can control the time pressure, or at least
its effect on how well the task is understood and performed. Note that for Asia, at
least, the intervention effectiveness study showed no speed–accuracy trade-off for
translation, implying that to some extent AMTs resist time pressures or expect to
be shielded from them.
We recognize the following two limitations in our studies: Study 1 relies on
self-reporting, and presumably, participants can only report those communication
 LANGUAGE ERROR IN AVIATION MAINTENANCE 43

errors that they later became aware of. It is possible that some communication er-
rors go undetected, and hence unreported, particularly if language ability is low.
As a result, the quantification of occurrences of seven scenarios of language errors
tends to be conservative. However, the tendency might not proportionally corre-
spond to the scenarios because some scenarios are more common than others, and
consequently error occurrence related to these familiar scenarios might be re-
ported more frequently. In Study 2, a group setting was used to efficiently collect
data from a large sample of participants at 19 MRO sites worldwide. As discussed
earlier, the group setting might have restricted the use of two of the interventions
(e.g., bilingual coaching and a glossary) during the comprehension test, despite
their widespread use as interventions at MRO sites. As a result, the effectiveness of
these two interventions might be underestimated because of the testing environ-
ment.
The contribution of this study has been to collect quantitative evidence to test
whether language barriers result in maintenance deficiencies, and further to pro-
vide quantitative evidence for how any potential problem can be managed. This is
a timely effort because according to a recent study of the MRO industry (Gardner,
2005), two thirds of MROs expect their revenues to increase over the next few
years, and almost all airlines surveyed plan to increase outsourcing or keep it con-
stant; the world regions investigated in this study, Latin America and Asia, are ex-
pected to continue attracting new work due to their attractive labor rates. Instead of
a simulated environment, future research could evaluate different interventions to
overcome language barriers using field study of task card comprehension and ex-
amination of actual execution of the maintenance tasks. Many of the visited MROs
said that they would adapt this task card comprehension methodology as a tool for
training and continuous improvement. More detailed research within MROs can
help to determine additional sources of errors and opportunities for improvement
by including a combination of factors such as identification of key points empha-
sized in quality assurance documents and auditing process, as well as coordinating
with their training programs and workers’ feedback.

CONCLUSIONS

Fortunately, language error management is not a new problem, only an increasing

one. Thus, practices have evolved at OEM suppliers, regulatory bodies, and main-
tenance organizations to address language errors. Our studies have collected such
interventions and tested them quantitatively to guide the aviation community.
Based on the data collected from 941 participants in four world regions—Asia,
Latin America, Europe, and North America—we conclude the following primary
findings:
44 MA, DRURY, & MARIN

• Of the seven scenarios tested, three were the most common, with reported
frequencies of 4 to 10 times per year. They involved synchronous (verbal)
and asynchronous (documentation) communication. Most of these language
errors were detected early in the process, typically when the AMT asked for
help or appeared perplexed.
• Language errors had many of the characteristics of human errors in general:
inadequate abilities of the AMT, low familiarity with the task, complex work
instructions, and time pressures for work completion.
• In a comprehension test of task cards, accuracy performance was generally
good, and was better in areas that were bilingual. None of the interventions
except translation proved effective. Glossaries and bilingual coaches were
rarely used by participants even when provided in the comprehension study.
• Translation of documents into English is an effective means of improving the
comprehension performance. The translation intervention is difficult and costly:
It can also be error prone unless done well. However, it did prove effective. In
Asia, the improvement was in speed only, but in other regions accuracy also im-
proved. Partial translation proved as effective as full translation where tested.
• Nonroutine repair forms, local contracts, and shift turnovers were typically
in the native language, whereas documents for audits, contracts with U.S.
companies, and maintenance manuals were typically in English. Training
and meetings were typically conducted in the native language.
• The social environment was found important in language error causation and
mitigation. Time pressure on AMTs and inspectors was reported as a major
cause of language errors. Regular testing of AMTs’ English ability and task
assignments recognizing the AMTs’ knowledge of English and familiarity
with the task are examples of social interventions.
• Better communications design, such as document design, standard protocols, job
aids, and job-related English training, would effectively reduce language errors.

ACKNOWLEDGMENTS

This work was performed under contract (No. 2002-G-025) by the Federal Avia-
tion Administration; contract monitor, William K. Krebs. Part of this study was
presented at the 2003–5 Human Factors and Ergonomics Society Annual Meeting,
and is in the Proceedings.

REFERENCES

Aircraft Repair and Maintenance Advisory Committee. (2001). FAR Part 145 (Report to Secretary of
Transportation). Washington, DC: Federal Aviation Administration. Electronic Code of Federal
Regulations (e-CFR). Retrieved February 7, 2009, from http://ecfr.gpoaccess.gov/cgi/t/test/text-
idx?c=ecfr&tpl=/ecfrbrowse/Title14/14ctr145_main_02.tpl
 LANGUAGE ERROR IN AVIATION MAINTENANCE 45

Carver, R. P. (1987). Technical manual for the Accuracy Level Test. Kansas City, MO: REVTAC
Publications.
Chervak, S. C., & Drury, C. G. (2003). Effects of job instruction on maintenance task performance. Oc-
cupational Ergonomics, 3, 121–132.
Chervak, S., Drury, C. G., & Ouellette, J. L. (1996). Simplified English for aircraft workcards. In Pro-
ceedings of the Human Factors and Ergonomics Society 39th Annual Meeting (pp. 303–307). Santa
Monica, CA: HFES.
Drury, C. G. (1995). The use of archival data. In J. R. Wilson & E. N. Corlette (Eds.), Evaluation of hu-
man work: A practical ergonomics methodology (2nd ed., pp. 101–112). Philadelphia: Taylor &
Francis.
Drury, C. G., Guy, K., Ma, J., & Wenner, C. (in press). Outsourcing aviation maintenance: Human fac-
tors implications. International Journal of Aviation Psychology.
Drury, C. G., & Ma, J. (2003, October). Do language barriers result in aviation maintenance errors? Paper
presented at the 47th annual meeting of the Human Factors and Ergonomics Society, Denver, CO.
Drury, C. G., & Ma, J. (2004). Language errors in aviation maintenance: Year 1 interim report (Re-
search Grant No. 2002-G-025). Washington, DC: Federal Aviation Administration.
Drury, C. G., Ma, J., & Marin, C. K. (2005). Language error in aviation maintenance: Final report
(Research Grant No. 2002-G-025). Washington, DC: Federal Aviation Administration.
Drury, C. G., & Sarac, A. (1997). Documentation design aids development (Human Factors in Aviation
Maintenance—Phase Seven, Progress Report, Tech. Rep. No. DOT/FAA/AM-97/xx). Springfield,
VA: National Technical Information Service.
Drury, C. G., Wenner, C., & Kritkausky, K. (1999). Development of process to improve work documen-
tation of repair stations (FAA/Office of Aviation Medicine Tech. Rep. No. AAM-240). Springfield,
VA: National Technical Information Service.
Drury, C. G., Wenner, C., & Kritkausky, K. (2000). Information design issues in repair stations. In R. S.
Jensen (Ed.) Proceedings of Tenth International Symposium on Aviation Psychology (pp. 291–300).
Columbus, OH: Ohio State University.
Federal Aviation Administration. (2003). Review of air carrier’s use of aircraft repair stations (FAA
IG Rep. No. AV-2003-047). Washington, DC: Author.
Federal Aviation Administration. (2005a). Air carriers’ use of noncertificated repair facilities (Tech.
Report No. AV2006031). Washington, DC: FAA Office of the Inspector General.
Federal Aviation Administration (2005b). Air carriers’ use of noncertificated repair facilities (Report
No. AV2006031). Washington, DC: FAA Office of the Inspector General.
Fegyveresi, A. (1997). Vocal cues and pilot/ATC communication. International Symposium on Avia-
tion Psychology, 1, 81–84.
Gardner, M. (2005). Outsourcing: A look at some of the industry practices. Aircraft Maintenance Tech-
nology, August 8. Retrieved July 23, 2006, from http://www.amtonline.com/publication/article.jsp?
pubId=1&id=1713
Garland, D. J., Wise, J. A., & Hopkin, V. D. (Eds.). (1999). Handbook of aviation human factors.
Mahwah, NJ, Lawrence Erlbaum Associates, Inc.
Griffith, E. (1999). A first look at communication theory (4th ed.). Boston: McGraw-Hill.
Haig, K. M., Sutton, S., & Whittington, J. (2006). SBAR: A shared mental model for improving com-
munication between clinicians. , 32, 167–175.
Johnson, D. W. (1972). Reaching out. Englewood Cliffs, NJ: Prentice Hall.
Kanki, B. G., & Smith, G. M. (2001). Training aviation communication skills. In E. Salas, C. A.
Bowers, & E. Edens (Eds.), Improving teamwork in organizations: Applications of resource man-
agement training (pp. 95–127). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Maddox, M. (Ed.). (1998). The human factors guide for aviation maintenance 3.0. Washington, DC:
Federal Aviation Administration/Office of Aviation Medicine. Retrieved July 17, 2002, from
http://hfskyway.faa.gov
46 MA, DRURY, & MARIN

McAuley, J. G. (1979). People to people: Essentials of personal and public communication. Belmont,
CA: Wadsworth.
Merritt, A., & Ratwatte, S. (1997). Who are you calling a safety threat? A debate on safety in mono-
versus multi-cultural cockpits. In R. Jensen (Ed.), International Symposium on Aviation Psychology,
(pp. 661–666). Columbus, OH: The Ohio State University.
Phillips, E. H. (2004). MRO mired in red ink. Aviation Week & Space Technology, 160(3), 62–63.
Reason, J. (1997). Approaches to controlling maintenance error. In Proceedings of the FAA/AAM 11th
Meeting on Human Factors Issues in Aviation Maintenance and Inspection. San Diego, CA. Wash-
ington, DC: Federal Aviation Administration/Office of Aviation Medicine.
Reason, J., & Hobbs, A. (2003). Managing maintenance error: A practical guide. Burlington, VT:
Ashgate.
Rifkind, L. (1996). Communication human factors guide for aviation maintenance. Washington, DC:
Federal Aviation Administration/Office of Aviation Medicine. Retrieved July 17, 2002, from:
Seidenman, P., & Spanovich, J. (2004). European and Asian MRO rebounding and expanding. Avia-
tion Week & Space Technology, 161(10), S1–S8.
Shannon, C. E., & Weaver, W. (1949). A mathematical model of communication. Urbana: University of
Illinois Press.
Sparaco, P. (2002). French foresee broader, multinational MRO network. Aviation Week and Space
Technology, 156(14), 72–73.
Taylor, J. C. (1993). The effects of crew resource management (CRM) training in maintenance: An
early demonstration of training effects on attitudes and performance Human Factors in Aviation
Maintenance—Phase Two Progress Report, DOT/FAA/AM-93/5). Alexandria, VA: National Tech-
nical Information Service.
Taylor, J. C., & Felten, D. F. (1993). Performance by design. Upper Saddle River, NJ: Prentice Hall.
Taylor, J. C., & Patankar, M. (2000). The role of communication in the reduction of human error. In
Proceedings of the 14th International Symposium on Human Factors in Aircraft Maintenance and
Inspection, Vancouver, BC, Canada. Washington, DC: Federal Aviation Administration.
Weber, G. (1997). Top languages: The world’s 10 most influential languages. Language Today, 2.
Wickens, C. D., & J. G. Hollands. (2000). Engineering psychology and human performance. Upper
Saddle River, NJ: Prentice Hall.
Wideman, R. M. (2002). Communication models. Retrieved July 17, 2002, from http://www.maxwideman.
com/isacons4iac1432/

Manuscript first received: September 2006

APPENDIX

47
Copyright of International Journal of Aviation Psychology is the property of Lawrence Erlbaum Associates and
its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for individual use.