DOT/FAA/AM-06/18

Office of Aerospace Medicine Washington, DC 20591

Human Error and Commercial Aviation Accidents: A Comprehensive, Fine-Grained Analysis Using HFACS
Scott Shappell
Clemson University Clemson, SC 29634

Cristy Detwiler
Kali Holcomb Carla Hackworth

Federal Aviation Administration

Civil Aerospace Medical Institute

Oklahoma City, OK 73125 Albert Boquet Embry-Riddle Aeronautical University Daytona Beach, FL 32114
Douglas Wiegmann

Mayo Clinic Rochester, MN 55905

July 2006

Final Report

NOTICE
This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest of information exchange. The United States Government assumes no liability for the contents thereof. ___________ This publication and all Office of Aerospace Medicine technical reports are available in full-text from the Civil Aerospace Medical Institutes publications Web site:
www.faa.gov/library/reports/medical/oamtechreports/index.cfm

Technical Report Documentation Page

1. Report No.

DOT/FAA/AM-06/18
4. Title and Subtitle

2. Government Accession No.

3. Recipient's Catalog No.

Human Error and Commercial Aviation Accidents: A Comprehensive, Fine-Grained Analysis Using HFACS
7. Author(s)

5. Report Date

July 2006
6. Performing Organization Code 8. Performing Organization Report No.

Shappell SA, Detwiler CA, Holcomb KA, Hackworth CA, Boquet AJ, 4 Wiegmann DA
9. Performing Organization Name and Address
1

Clemson University, Clemson, South Carolina 29634 2 FAA Civil Aerospace Medical Institute, Oklahoma City, OK 73125 3 Embry-Riddle Aeronautical University, Daytona Beach, FL 32114 4 Mayo Clinic; Rochester, MN 55905
12. Sponsoring Agency name and Address

10. Work Unit No. (TRAIS)

11. Contract or Grant No.

Office of Aerospace Medicine Federal Aviation Administration 800 Independence Ave., S. W. Washington, D.C. 20591
15. Supplemental Notes 16. Abstract

13. Type of Report and Period Covered

14. Sponsoring Agency Code

Work was accomplished under approved subtask AM-B-05-HRR-521. The Human Factors Analysis and Classification System (HFACS) is a theoretically based tool for investigating and analyzing human error associated with accidents and incidents. Previous research has shown that HFACS can be reliably used to identify general trends in the human factors associated with military and general aviation accidents. The aim of this study was to extend previous examinations of aviation accidents to include specific aircrew, environmental, supervisory, and organizational factors associated with 14 CFR Part 121 (Air Carrier) and 14 CFR Part 135 (Commuter) accidents using HFACS. The majority of causal factors were attributed to the aircrew and the environment, with decidedly fewer associated with supervisory and organizational causes. Comparisons were made between HFACS categories and traditional situational variables such as weather, lighting, and geographic region. Recommendations were made based on the HFACS findings presented.

17. Key Words

Human Error, HFACS, Aviation Accidents, Commercial Aviation

19. Security Classif. (of this report) 20. Security Classif. (of this page)

18. Distribution Statement

Document is available to the public through the Defense Technical Information Center, Ft. Belvior, VA 22060; and the National Technical Information Service, Springfield, VA 22161
21. No. of Pages

Unclassified

Unclassified

22

22. Price

Form DOT F 1700.7 (8-72)

Reproduction of completed page authorized

Human Error and CommErCial aviation aCCidEnts: a ComprEHEnsivE, FinE-GrainEd analysis usinG HFaCs
Flying is not inherently dangerous, but to an even greater extent than the sea, it is terribly unforgiving .
Captain A. G. Lumplugh, British Aviation Insurance Group

INTRODUCTION
Since Silas Christofferson first carried passengers on his hydroplane between San Francisco and Oakland harbors in 1913, engineers and psychologists have endeavored to improve the safety of passenger and cargo flight. What began as an industry fraught with adversity and at times tragedy has emerged as arguably one of the safest modes of transportation today. Indeed, no one can question the tremendous strides that have been made since those first passenger flights nearly a century ago. However, while commercial1 aviation accident rates have reached unprecedented levels of safety, little, if any, improvement has been realized over the last decade for either the air carrier or commuter/air taxi industry (Figure 1). Indeed, some have even suggested that the current accident rate is as good as it gets or is it? The challenge for the Federal Aviation Administration (FAA) and other civil aviation safety organizations is to improve an already very safe industry. The question is where to start when most of the low hanging fruit (e.g., improved powerplant and airframe technology, advanced avionics, and the introduction of automation) have been picked. Although percentages vary, most would agree that somewhere between 60-80% of aviation accidents are due, at least in part, to human error (Shappell & Wiegmann, 1996). That being said, it may be surprising that with few exceptions (e.g., Billings & Reynard, 1984; Gaur, 2005; Li, Baker, Grabowski, & Rebok, 2001; Shappell & Wiegmann, 2003a, 2003b; Wiegmann & Shappell, 2003) most studies to date have focused on situational factors or pilot demographics, rather than the underlying human error causes of accidents. While no one disagrees that factors like weather, lighting (i.e., day versus night), and terrain contribute to accidents, pilots have little, if any, control over them. Likewise, little can be done to affect ones gender, age, occupation,

or even flight experience, as flight hours alone are not the sole determinant of a safe pilot. Judging from current accident rates, situational and pilot demographic data alone have provided little in the way of preventing accidents, apart from identifying target populations for the dissemination of safety information. This is not to say that these variables are unimportant, nor would anyone argue that they do not influence aviation safety. However, given the multi-factorial nature of accidents (Baker, 1995), it may make more sense to examine these variables within the context of what we know about human error and accident causation. Perhaps then we might be able to affect human error and reduce aviation accidents beyond current levels. The problem is that, unlike situational and demographic variables that are tangible and well-defined (e.g., instrument meteorological conditions and visual meteorological conditions), human error is much more complex, making it difficult to apply any sort of taxonomy that is both easily understood and universally accepted. However, that may have changed with the development of the Human Factors Analysis and Classification System (HFACS) in the mid-1990s. In fact, since the U.S. Navy/Marine Corps fielded the original version in 1997, HFACS has been used to reliably investigate and classify human error in a variety of high-risk settings including civilian aviation 1. 135 2005; Shappell & by year Figure (Gaur,& 121 Accident RateWiegmann, 2003a, 2003b, 2004; Wiegmann & Shappell, 2001a, 2003).
4 .5 4
Accidents per 100,000 Flight Hours

Commuter Air Carrier

3 .5 3 2 .5 2 1 .5 1 0 .5 0 85 86 87 88 89 90 91 92 93 94 95 96 97 9 8 9 9 '0 0 '0 1 '0 2 '0 3 '0 4

Year

Figure 1. Air carrier and commuter/air taxi accident rates since 1985 (Source: NTSB).
The FAA distinguishes between two types of commercial operations: those occurring under 14 CFR Part 121 Air Carrier Operations and those occurring under CFR Part 135 - Commuter/air taxi operations.
1

ORGANIZATIONAL INFLUENCES Resource Management Organizational Climate UNSAFE SUPERVISION Operational Process

Inadequate Supervision

Planned Inappropriate Operations

Failed to Correct Problem

Supervisory Violations

PRECONDITIONS FOR UNSAFE ACTS

Environmental Factors

Condition of Operators Adverse Mental States Adverse Physiological States UNSAFE ACTS Errors Violations Perceptual Errors Physical/ Mental Limitations

Personnel Factors

Physical Environment

Technological Environment

Crew Resource Management

Personal Readiness

Decision Errors

Skill-Based Errors

Routine

Exceptional

Figure 2. The HFACS framework.

HFACS
It is generally accepted that (Wiegmann & Shappell, 2001a, 2003) aviation accidents are typically the result of a chain of events that often culminate with the unsafe acts of operators (aircrew). The aviation industry is not alone in this belief, as the safety community has embraced a sequential theory of accident investigation since Heinrich first published his axioms of industrial safety in 1931 (Heinrich, Peterson, & Roos, 1931). However, it was not until Reason published his Swiss cheese model of human error in 1990 that the aviation community truly began to examine human error in a systematic manner. Drawing upon Reasons (1990) concept of latent and active failures, HFACS describes human error at each of four levels: 1) the unsafe acts of operators (e.g., aircrew, maintainers, air traffic controllers), 2) preconditions for unsafe acts, 3) unsafe supervision (i.e., middle-management), and 4) organizational influences (Figure 2). A brief description of each causal category is provided to familiarize the reader. Unsafe Acts of Operators The unsafe acts of operators (aircrew) can be loosely classified into one of two categories: errors and violations (Reason, 1990). While both are common within
2

most settings, they differ markedly when the rules and regulations of an organization are considered. That is, while errors represent authorized behavior that fails to meet the desired outcome, violations refer to the willful disregard of the rules and regulations. It is within these two overarching categories that HFACS describes three types of errors (decision, skill-based, and perceptual) and two types of violations (routine and exceptional). Errors Decision errors. One of the more common error forms, decision errors, represents conscious, goal-intended behavior that proceeds as designed, yet the plan proves inadequate or inappropriate for the situation. Often referred to as honest mistakes, these errors typically manifest as poorly executed procedures, improper choices, or simply the misinterpretation and/or misuse of relevant information. Skill-based errors. In contrast to decision errors, the second error form, skill-based errors, occurs with little or no conscious thought. Indeed, just as decision errors can be thought of as thinking errors, skill-based errors can be thought of as doing errors. For instance, little thought goes into turning ones steering wheel or shifting gears in an automobile. Likewise, basic flight skills such as stick and rudder movements and visual scanning refer more to how one does something rather than where one is going or why. The difficulty with these highly
2

A complete description of all 19 HFACS causal categories is available elsewhere (see Wiegmann & Shappell, 2003).

practiced and seemingly automatic behaviors is that they are particularly susceptible to attention and/or memory failures. As a result, skill-based errors frequently appear - the breakdown in visual scan patterns, inadvertent activation/deactivation of switches, forgotten intentions, and omitted items in checklists. Even the manner (or skill) with which one flies an aircraft (aggressive, tentative, or controlled) can affect safety. Perceptual errors. While decision and skill-based errors have dominated most accident databases and have, therefore, been included in most error frameworks, the third and final error form, perceptual errors, has received comparatively less attention. No less important, these perceiving errors arise when sensory input is degraded, or unusual as is often the case when flying at night, in the weather, or in other visually impoverished environments. Faced with acting on imperfect or incomplete information, aircrew run the risk of misjudging distances, altitude, and decent rates, as well as responding incorrectly to a variety of visual/vestibular illusions. Violations Routine violations. Although there are many ways to distinguish between types of violations, two distinct forms have been identified based on their etiology. The first, routine violations tend to be habitual by nature and are often enabled by a system of supervision and management that tolerates such departures from the rules (Reason, 1990). Often referred to as bending the rules, the classic example is that of the individual who drives his/her automobile consistently 5-10 mph faster than allowed by law. While clearly against the law, the behavior is, in effect, sanctioned by local authorities (police) who often will not enforce the law until speeds in excess of 10 mph over the posted limit are observed. Exceptional violations. These types of violations, on the other hand, are isolated departures from authority, neither typical of the individual nor condoned by management. For example, while authorities might condone driving 65 in a 55 mph zone, driving 105 mph in a 55 mph zone would almost certainly result in a speeding ticket. It is important to note that, while most exceptional violations are appalling, they are not considered exceptional because of their extreme nature. Rather, they are regarded as exceptional because they are neither typical of the individual nor condoned by authority. Preconditions for Unsafe Acts Simply focusing on unsafe acts, however, is like focusing on a patients symptoms without understanding the underlying disease state that caused it. As such, investigators must dig deeper into the preconditions for unsafe acts. Within HFACS, three major subdivisions
3

are described: 1) condition of the operator, 2) personnel factors, and 3) environmental factors. Condition of the Operator Adverse mental states. Being prepared mentally is critical in nearly every endeavor; perhaps it is even more so in aviation. With this in mind, the first of three categories, adverse mental states, was created to account for those mental conditions that adversely affect performance and contribute to unsafe acts. Principal among these are the loss of situational awareness, mental fatigue, circadian dysrhythmia, and pernicious attitudes such as overconfidence, complacency, and misplaced motivation. Adverse physiological states. Equally important, however, are those adverse physiological states that preclude the safe conduct of flight. Particularly important to aviation are conditions such as spatial disorientation, visual illusions, hypoxia, illness, intoxication, and a whole host of pharmacological and medical abnormalities known to affect performance. It is important to understand that conditions like spatial disorientation are physiological states that cannot be turned on or off they just exist. As a result, these adverse physiological states often lead to the commission of unsafe acts like perceptual errors. For instance, it is not uncommon in aviation for a pilot to become spatially disoriented (adverse physiological state) and subsequently misjudge the aircrafts pitch or attitude (perceptual error), resulting in a loss of control and/or collision with the terrain. Physical and/or mental limitations. The third and final category of substandard conditions, physical/mental limitations, includes those instances when necessary sensory information is either unavailable, or if available, individuals simply do not have the aptitude, skill, or time to safely deal with it. In aviation, the former often includes not seeing other aircraft or obstacles due to the size and/or contrast of the object in the visual field. Likewise, there are instances when an individual simply may not possess the necessary aptitude, physical ability, or proficiency to operate safely. After all, just as not everyone can play linebacker for their favorite professional football team or be a concert pianist, not everyone has the aptitude or physical attributes necessary to fly aircraft. Personnel Factors Often times, things that we do to ourselves will lead to undesirable conditions and unsafe acts as described above. Referred to as personnel factors, these preconditions have been divided into two general categories: crew resource management and personal readiness. Crew resource management (CRM). It is not hard to imagine that when all members of the crew are not acting in a coordinated manner, confusion (adverse mental

state) and poor decisions in the cockpit can ensue. Crew resource mismanagement, as it is referred to here, includes the failures of both inter- and intra-cockpit communication, as well as communication with Air Traffic Control (ATC )and other ground personnel. This category also includes those instances when crewmembers do not work together as a team, or when individuals directly responsible for the conduct of operations fail to coordinate activities before, during, and after a flight. Personal readiness. Individuals must, by necessity, ensure that they are adequately prepared for flight. Consequently, the category of personal readiness was created to account for those instances when rules such as disregarding crew rest requirements, violating alcohol restrictions, or selfmedicating, are not adhered to. However, even behaviors that do not necessarily violate existing rules or regulations (e.g., running ten miles before piloting an aircraft or not observing good dietary practices) may reduce the operating capabilities of the individual and are, therefore, captured here as well. Environmental Factors Although not human per se, environmental factors can also contribute to the substandard conditions of operators and hence to unsafe acts. Very broadly, these environmental factors can be captured within two general categories: the physical environment and the technological environment. Physical environment. The impact that the physical environment can have on aircrew has long been known and much has been documented in the literature on this topic (e.g., Nicogossian, Huntoon, & Pool, 1994; Reinhart, 1996). The term physical environment refers to both the operational environment (e.g., weather, altitude, terrain), as well as the ambient environment, such as heat, vibration, lighting, and toxins in the cockpit. For example, flying into adverse weather reduces visual cues, which can lead to spatial disorientation and perceptual errors. Other aspects of the physical environment such as heat can cause dehydration, which reduces a pilots alertness level, producing a subsequent slowing of decision-making processes or even the inability to control the aircraft. Likewise, a loss of pressurization at high altitudes or maneuvering at high altitudes without supplemental oxygen in unpressurized aircraft can result in hypoxia, which leads to delirium, confusion, and a host of unsafe acts. Technological environment. Pilots that often find themselves in a technological environment that can also have a tremendous impact on their performance. Within the context of HFACS, the term technological environment encompasses a variety of issues, including the design of equipment and controls, display/interface characteristics,
4

checklist design, and automation, to name a few. Indeed, one of the classic design problems first discovered in aviation was the similarity between the controls used to raise and lower the flaps and those used to raise and lower the landing gear. Such similarities often caused confusion among pilots, resulting in the frequent raising of the landing gear while still on the ground. Likewise, automation designed to improve human performance can have unforeseen consequences. For example, highly reliable automation has been shown to induce adverse mental states such as overconfidence and complacency, resulting in pilots following the instructions of the automation even when common sense suggests otherwise. In contrast, unreliable automation can often result in a lack of confidence and disuse of automation even though aided performance is safer than unaided performance (Wickens & Hollands, 2000). Unsafe Supervision Clearly, aircrews are responsible for their actions and, as such, must be held accountable. However, in some instances, they are the unwitting inheritors of latent failures attributable to those who supervise them. To account for these latent failures, the overarching category of unsafe supervision was created within which four categories (inadequate supervision, planned inappropriate operations, failed to correct known problems, and supervisory violations) are included. Inadequate supervision. This category refers to failures within the supervisory chain of command as a direct result of some supervisory action or inaction. At a minimum, supervisors must provide the opportunity for individuals to succeed. It is expected, therefore, that individuals will receive adequate training, professional guidance, oversight, and operational leadership, and that all will be managed appropriately. When this is not the case, aircrew can become isolated, thereby increasing the risks associated with day-to-day operations. Planned inappropriate operations. The risk associated with supervisory failures come in many forms. Occasionally, for example, the operational tempo and/or schedule are planned such that individuals are put at unacceptable risk and, ultimately, performance is adversely affected. As such, the category of planned inappropriate operations was created to account for all aspects of improper or inappropriate crew scheduling and operational planning, which may focus on such issues as crew pairing, crew rest, and managing the risk associated with specific flights. Failed to correct known problems. The remaining two categories of unsafe supervision, the failure to correct known problems and supervisory violations, are similar, yet considered separately within HFACS. The failure to correct known problems refers to those instances when

deficiencies among individuals, equipment, training, or other related safety areas are known to the supervisor, yet are allowed to continue uncorrected. For example, the failure to consistently correct or discipline inappropriate behavior certainly fosters an unsafe atmosphere but is not considered a violation if no specific rules or regulations were broken. Supervisory violations. This category is reserved for those instances when supervisors willfully disregard existing rules and regulations. For instance, permitting aircrew to operate an aircraft without current qualifications or license is a flagrant violation that may set the stage for the tragic sequence of events that may follow. Organizational Influences Where decisions and practices by front-line supervisors and middle-management can adversely impact aircrew performance, fallible decisions of upper-level management may directly affect supervisors and the personnel they manage. Unfortunately, these organizational influences often go unnoticed or unreported by even the best-intentioned accident investigators. The HFACS framework describes three latent organizational failures: 1) resource management, 2) organizational climate, and 3) operational processes. Resource management. This category refers to the management, allocation, and maintenance of organizational resources, including human resource management (selection, training, staffing), monetary safety budgets, and equipment design (ergonomic specifications). In general, corporate decisions about how such resources should be managed center around two distinct objectives the goal of safety and the goal of on-time, cost-effective operations. In times of prosperity, both objectives can be easily balanced and satisfied. However, there may also be times of fiscal austerity that demand some give and take between the two. Unfortunately, history tells us that safety is often the loser in such battles, as safety and training are often the first to be cut in organizations experiencing financial difficulties. Organizational climate. The concept of an organizations culture has been described in many ways; however, here it refers to a broad class of organizational variables that influence worker performance. One telltale sign of an organizations climate is its structure, as reflected in the chain-of-command, delegation of authority and responsibility, communication channels, and formal accountability for actions. Just like in the cockpit, communication and coordination are vital within an organization. However, an organizations policies and culture are also good indicators of its climate. Consequently, when policies are ill-defined, adversarial, or conflicting, or when they are supplanted by unofficial
5

rules and values, confusion abounds, and safety suffers within an organization. Operational process. Finally, operational process refers to formal processes (operational tempo, time pressures, production quotas, incentive systems, schedules, etc.), procedures (performance standards, objectives, documentation, instructions about procedures, etc.), and oversight within the organization (organizational self-study, risk management, and the establishment and use of safety programs). Poor upper-level management and decisions concerning each of these organizational factors can also have a negative, albeit indirect, effect on operator performance and system safety.

PURPOSE
The goal of the present study was twofold: 1) to extend our previous HFACS analyses beyond military and general aviation (GA) to include a comprehensive analysis of commercial aviation; and 2) to combine the power of a theoretically derived human error framework (i.e., HFACS) with traditional situational and demographic data from the accident records. In accomplishing both objectives, the present study will begin to quantify the role human error plays in the genesis of commercial aviation accidents.

METHOD
Data Commercial aviation accident data (i.e., 14 CFR Part 121 air carrier; 14 CFR Part 135 commuter) from calendar years 1990-2002 were obtained from databases maintained by the National Transportation Safety Board (NTSB) and the FAAs National Aviation Safety Data Analysis Center (NASDAC). The NTSB reports two levels of investigation: factual and final. The factual investigation is a preliminary report that only includes basic descriptive information associated with the accident (location, time-of-day, weather conditions, etc.) but lists no causal factors. The final report contains all the information in the factual report as well as the causal factors associated with the accident. Consequently, only final reports were included in this study. Also eliminated from consideration were accidents that were classified as having undetermined causes and those that were attributed to sabotage, suicide, or criminal activity (e.g., stolen aircraft). The data were culled further to include only those accidents that involved aircrew or supervisory error. Of the remaining 1,020 accidents, 181 involved air carrier aircraft and 839 involved commuter aircraft.

Table 1. Frequency of Accidents Associated with an Aircrew or Supervisory Human Error.

Year 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 Total Average Aircrew/Supervisory Error Only Air Carrier Commuter Combined 9 81 90 10 71 81 9 67 76 14 67 81 11 74 85 13 59 72 14 71 85 22 68 90 14 62 76 15 62 77 20 62 82 18 52 70 12 43 55 181 839 1020 13.92 64.54 78.46 Total Accidents 134 121 103 99 113 105 123 130 121 120 135 120 92 1516 116.6 Percentage 67% 67% 74% 82% 75% 69% 69% 69% 63% 64% 61% 58% 60% 68%

Note: Percentages represent the percent of commercial (both air carrier and commuter) aviation accidents associated with aircrew/supervisory error. For example, 90 of 134 commercial aviation accidents (67%) were associated with aircrew and/or supervisory error.

A summary of the remaining air carrier and commuter accidents involving aircrew or supervisory error is presented in Table 1. Sixty-eight percent of all commercial aviation accidents included in this study involved some form of aircrew or supervisory error. Causal Factor Analysis Using HFACS Six pilots were recruited from the Oklahoma City area as subject-matter experts (SMEs). All were certified flight instructors with a minimum of 1,000 flight hours at the time they were recruited. Each pilot was provided roughly 16 hours of instruction on the HFACS framework, which included didactic lecture and practice (with feedback) using the HFACS framework with NTSB/NASDAC accident reports. After training, the pilot-raters were randomly assigned accidents so at least two separate pilot-raters independently analyzed each accident. Using narrative and tabular data obtained from both the NTSB and the FAA NASDAC, the pilot-raters were instructed to classify each aircrew or supervisory causal factor identified by the NTSB using the HFACS framework. Note that only those causal and contributory factors identified by the NTSB were classified. That is, the pilot-raters were instructed not to introduce additional casual factors that were not identified by the original investigation. After the pilot-raters made their initial classifications of the NTSB causal factors (i.e., skill-based error, decision-error, etc.), the two independent ratings were compared. Where disagreements existed, the corresponding pilot-raters were instructed to reconcile their differences, and the consensus classification was included in the database for further analysis. Overall, pilot-raters agreed
6

on the classification of causal factors within the HFACS framework more than 85% of the time, an excellent level of agreement considering that this was essentially a classification/decision-making task. Human Factors Quality Assurance The data used in this study were drawn from NTSB/ NASDAC investigation reports that are often highly technical in nature, requiring a fundamental understanding of specific terms, flight conditions, and the overall domain of aviation to be effectively classified and coded. As aviation SMEs, the pilot-raters were able to clearly understand all aspects of the accident report. Consequently, they were considered the appropriate personnel for conducting the overall HFACS analysis of the commercial accident reports. Pilots, however, are not SMEs in the domain of psychology or human factors and may not fully understand the theoretical underpinnings associated with the various error types within the HFACS framework. As a result, pilots might classify human error data somewhat differently than SMEs in human factors. On the other hand, pilots in this study were trained on HFACS, which provided some level of expertise when assessing human error. In fact, an earlier study addressed this issue by comparing the coded database of a commercial pilot rater to that of a psychologist and found the data to be reliable (Wiegmann & Shappell, 2001a). Nonetheless, to ensure that the pilot raters grasped the psychological aspects underlying human error and HFACS, four additional SMEs (all co-authors of this manuscript) with expertise in human factors/aviation psychology examined each HFACS classification that the pilot SMEs had assigned to a given human cause factor.

Table 2. Frequency and percentage of accidents associated with each HFACS causal category by type of operation.
HFACS Category Organizational Influences Resource Management Organizational Climate Operational Process Unsafe Supervision Inadequate Supervision Planned Inappropriate Operations Failed to Correct Known Problems Supervisory Violations Preconditions of Unsafe Acts Environmental Conditions Technological Environment Physical Environment Conditions of the Operator Adverse Mental States Adverse Physiological States Physical/Mental Limitations Personnel Factors Crew Resource Management Personal Readiness Unsafe Acts of the Operator Skill-based Errors Decision Errors Perceptual Errors Violations Air Carrier N (%) 4 (2.2) 0 (0.0) 21 (11.6) 15 (8.3) 3 (1.7) 0 (0.0) 0 (0.0) Commuter N (%) 0 (0.0) 4 (0.5) 29 (3.5) 21 (2.5) 5 (0.6) 0 (0.0) 2 (0.2) Total N (%) 4 (0.4) 4 (0.4) 50 (4.9) 36 (3.5) 8 (0.8) 0 (0.0) 2 (0.2)

11 (6.1) 67 (37.0) 6 (3.3) 6 (3.3) 6 (3.3) 34 (18.8) 0 (0.0) 77 (42.5) 71 (39.2) 10 (5.5) 31 (17.1)

4 (0.5) 525 (62.6) 60 (7.2) 18 (2.1) 39 (4.6) 75 (8.9) 3 (0.4) 499 (59.5) 303 (36.1) 56 (6.7) 205 (24.4)

15 (1.5) 592 (58.0) 66 (6.5) 24 (2.4) 45 (4.4) 109 (10.7) 3 (0.3) 576 (56.5) 374 (36.7) 66 (6.5) 236 (23.1)

Note: Numbers in the table are frequencies and percentages (parentheses) of accidents that involved at least one instance of an HFACS category. For example 77 of the 181 air carrier accidents (77/181 or 42.5%) were associated with at least one skill-based error. Because accidents are generally associated with more than one causal factor, the percentages in the table do not add up to 100%.

To aid in the process, descriptive statistics were used to identify outliers in the data, after which the corresponding NTSB/NASDAC report was obtained. The reports were then independently reviewed by a minimum of two human factors SMEs for agreement with the previous codes. After the human factors SMEs came to a consensus, the codes were either changed in the database or left as the pilot SMEs originally coded them. In the end, less than 5% of all causal factors were modified during the human factors quality assurance process.

RESULTS
Overall A summary of the HFACS analyses of commercial aviation accidents can be found in Table 2. What is apparent from the data is that the majority of human causal factors identified in the database involved aircrew and their environment (i.e., unsafe acts of operators and preconditions for unsafe acts) rather than supervisory or organizational factors. Nevertheless, when organizational influences were observed, they typically involved operational processes
7

such as inadequate or non-existent procedures, directives, standards, and/or requirements, or in the case of commuter operations, inadequate surveillance of operations. Unsafe supervision, on the other hand, typically involved inadequate supervision, in general, or the failure to provide adequate training. As anticipated, a large number of environmental conditions were identified within the commercial aviation database, particularly those associated with aspects of the physical environment like weather and lighting. However, they were not uniformly distributed across air carrier and commuter operations, as considerably more issues associated with the physical environment were observed during commuter (63%) than air carrier operations (37%). In contrast, the accident record revealed surprisingly few problems associated with the technological environment. Preconditions associated with aircrew were also frequently observed within the accident record. For instance, crew resource management failures were identified in nearly one out of every five air carrier accidents examined. Even more interesting, the nature of the CRM failure differed

between the two commercial operations. That is, while over 60% of the CRM failures associated with air carrier accidents involved inflight CRM failures (inflight crew coordination, communication, monitoring of activities, etc.), over 80% of the CRM failures observed during commuter operations involved preflight activities (such as planning and briefing). Although arguably not as common, the condition of the operator was cited as a causal factor in several of the accidents examined. For instance, adverse mental states (e.g., diverted attention, pressure, etc.) were identified in just over 7% of the commuter accidents, followed by physical/mental limitations (lack of experience) and adverse physiological states (spatial disorientation, visual illusions, etc.). As seen in other aviation operations (Shappell & Wiegmann, 1995, 1997, 2003a, 2003b, 2004; Wiegmann & Shappell, 1997, 2001a, 2001b, 2003) the majority of commercial aviation accident causal factors were found at the unsafe act level. Indeed, just over half of the accidents were associated with at least one skill-based error, followed by decision errors (36.7%) and violations of the rules and regulations (23.1%). Perceptual errors were much less common, accounting for roughly 7% of the accidents in the database. Because of the differences between air carrier and commuter operations (i.e., airframes, crew composition, size of the organization, etc.) it was anticipated that there would be differences in the pattern of human error observed - particularly where the unsafe acts of aircrew were concerned. However, a comparison of the unsafe acts committed during these operations (Figure 3) yielded very little disparity. In fact, the only significant difference involved skill-based errors, which were nearly twice as likely to have occurred during accidents involving commuter than air carrier aircraft (2 = 17.368, p<.001; odds ratio = 1.982). On the surface, it did appear that slightly more violations were committed during accidents involving commuter than air carrier operations; however, the difference was not statistically significant.4 Likewise, the small differences observed for decision and perceptual errors did not reach statistical significance. Similar to other civil aviation accident data we have reported (Shappell & Wiegmann, 2003a, 2003b, 2004; Wiegmann & Shappell, 2003), there was little variation

in the distribution of unsafe acts committed annually by aircrew flying either air carrier or commuter operations (Figure 4). When accidents occurred in either type of commercial operation, they were typically associated with more skill-based errors, followed by decision errors, violations, and perceptual errors, respectively. This was true even though the air carrier data had to be averaged over 3-4 year blocks due to the small number of accidents in the database (Figure 4A). Moreover, with the exception of violations, which has shown a slight increase since the 1993-1995 time frame, the annualized data were relatively flat, suggesting that there has been little impact on any specific type of human error over the last 13 years. 14 CFR Part 135 - Commuter Operations Because of the relatively small number of air carrier accidents in the database related to aircrew/supervisory error, additional fine-grained analyses of those data were not possible. However, the same was not true for commuter operations. Therefore, a series of more detailed analyses were conducted using these data. Visual Conditions Given the relatively large percentage of accidents associated with physical conditions, in particular those associated with prevailing weather conditions and lighting, it seemed reasonable to begin with these two environmental causal factors. As can be seen in Figure 5A, just over 70% of the accidents occurred during visual meteorological conditions (VMC). Likewise, roughly 70% of the accidents occurred in broad daylight (Figure 5B). To capitalize on the threat posed by both environmental causal factors, the two were combined to create a new variable that captured the visual conditions at the time of the accident. Specifically, two levels of visual conditions were created: 1) clear visual conditions, which included accidents that occurred during VMC and daylight conditions, and 2) impoverished visual conditions, which included accidents occurring during instrument meteorological conditions (IMC) or at twilight/night. Unlike the results seen with weather and lighting conditions alone, when they were combined, the percentage of accidents occurring in clear visual conditions were only marginally higher than those occurring in visually impoverished conditions (Figure 5C). It would appear that, while weather and lighting conditions are important factors in aviation, their impact is potentially magnified when a pilots ability to see outside the aircraft is taken into consideration.

The overarching category of violations was used rather than the subordinate categories of routine and exceptional violations because differentiating between the two, post hoc, is complicated by the fact that most investigations do not provide the detail necessary to make a reliable distinction between the two types of violations. 4 Given that Chi square analyses are strongly influenced by sample size, a conservative p-value of p<.001 was adopted to reduce the likelihood that spurious significant results would be obtained.

100 90 80 70 60 50 40 30 20 10 0

Percentage of Accidents

Percentage of Accidents

14 CFR Part 121 14 CFR Part 135

A.

100 90 80 70 60 50 40 30 20 10 0 1990-1992 1993-1995 1996-1998

Skill-based Errors Decision Errors Violations Perceptual Errors

Skill-based Error

Decision Error

Perceptual Error

Violation

Figure 3. HFACS unsafe acts by type of operation.

Year

1999-2002

B.

100

Percentage of Accidents

90 80 70 60 50 40 30 20 10 0 90 91 92 93 94 95 96 97 98 99 '00 '01 '02

Skill-based Errors

Decision Errors Violations Perceptual Errors

Percentages do not add up to 100%

Year

Figure 4. Percentage of unsafe acts committed by aircrew during air carrier (Panel A) and commuter (Panel B) operations (by year).

A.
Percentage of Accidents

100 90 80 70 60 50 40 30 20 10 0

B.
Percentage of Accidents
VMC IMC

100 90 80 70 60 50 40 30 20 10 0

Day

Twilight

Night

C.
Percentage of Accidents

100 90 80 70 60 50 40 30 20 10 0

D.
Percentage of Accidents

Clear

Impoverished

100 90 80 70 60 50 40 30 20 10 0

Clear Impoverished

Skill-based Error

Decision Error

Perceptual Error

Violation

Figure 5. Percentage of commuter accidents by weather conditions (Panel A), lighting conditions (Panel B), visual conditions (Panel C) and visual conditions by unsafe acts (Panel D).

Table 3. Commuter Unsafe Acts Fine-Grained Analysis by Clear vs. Impoverished.

SKILL BASED ERRORS
CLEAR Subject Compensation for Wind Conditions Airspeed Visual Lookout N (%) 42 (10.8) 38 (9.7) 32 (8.2) Subject Aircraft Control Airspeed Clearance IMPOVERISHED N (%) 28 (10.6) 27 (10.2) 21 (7.9)

DECISION ERRORS
CLEAR Subject Unsuitable Terrain Selection In-Flight Planning/Decision Pre-flight Planning/Decision CLEAR N (%) 43 (21.5) 38 (19.0) 21 (10.5) IMPOVERISHED Subject In-Flight Planning/Decision Flight into Known Adverse Weather Pre-flight Planning/Decision IMPOVERISHED Subject Intentional VFR Flight into IMC Procedures/Directives Flight into Known Adverse Weather N (%) 37 (24.3) 11 (7.2) 9 (5.9)

Violations
Subject Procedures/Directives Checklist Refueling N (%) 15 (23.8) 9 (14.3) 6 (9.5) N (%) 53 (30.1) 39 (22.2) 10 (5.7)

Note: Percentages in the table reflect the percentage of a given causal factor within the HFACS category (e.g., compensation for wind conditions accounted for 42 of 390 [10.8%] skill-based errors occurring in clear conditions).

Naturally, one would expect the pattern of human error to be different during accidents in clear versus visually impoverished conditions. Indeed, when visual conditions were compared across the unsafe acts of aircrew, an interesting pattern of human error emerged. While skill-based errors were the most common error form observed during accidents in clear and impoverished conditions (Figure 5D), violations were five times more likely to be attributed to accidents in visually impoverished conditions (2 = 92.332, p<.001; odds ratio = 5.077). Upon closer examination (Table 3), intentional flight into IMC while operating under visual flight rules (i.e., VFR flight into IMC) accounted for nearly a third of the violations observed during impoverished visual conditions. In addition, the failure to adhere to procedures/ directives (violation), poor inflight planning/decision making (decision error), the loss of control in-flight (skillbased errors), and the failure to maintain sufficient airspeed (skill-based error) all were commonly cited as causes during accidents in visually impoverished conditions. The failure to adhere to procedures/directives (violation) was also frequently seen among accidents in clear conditions, as was poor in-flight planning/decision-making (decision error). However, unlike impoverished visual conditions, commuter accidents occurring in the clear were often associated with the selection of unsuitable terrain (decision error) and the inability to compensate for winds (skill-based error).

Injury Severity Previous investigations of GA accidents have shown distinct differences in the pattern of human error associated with fatal and non-fatal aviation accidents (Shappell & Wiegmann, 2003a, 2003b; Wiegmann & Shappell, 2003). A similar examination of commuter accidents revealed that roughly 30% of all commuter accidents resulted in at least one fatality (Figure 6A). As with the findings regarding visual conditions, skill-based errors were associated with the majority of fatal and non-fatal accidents followed by decision errors, violations, and perceptual errors (Figure 6B). Of note however, violations were more than three times as likely to be associated with fatal accidents (2 = 48.239, p<.001; odds ratio = 3.145). Upon closer examination, it appears that causal factors such as intentional VFR flight into IMC (violation), poor in-flight planning/decision making (decision error), and control of the aircraft and airspeed (skill-based error) were the most frequently cited aircrew errors associated with fatal accidents (Table 4). In contrast, non-fatal accidents appear to be more closely associated with the failure to compensate for winds (skill-based error), loss of directional control on the ground (skill-based error), selection of unsuitable terrain (decision error), poor inflight planning/decision-making (decision error), and the failure to follow procedures/directives (violation). Given the similarity in the pattern of human errors associated with visual conditions and injury severity (fatal
10

Percentage of Accidents

A.

100 90 80 70 60 50 40 30 20 10 0

Non-Fatal

Fatal

B.
Percentage of Accidents

100 90 80 70 60 50 40 30 20 10 0

Non-Fatal Fatal

Skill-based Error

Decision Error

Perceptual Error

Violation

Figure 6. Percentage of fatal and non-fatal commuter accidents (Panel A) and percentage of fatal and non-fatal accidents by the unsafe acts (Panel B).

Table 4. Commuter Unsafe Acts Fine-Grained Analysis by Non-Fatal vs. Fatal.

SKILL BASED ERRORS
NON - FATAL Subject Compensation for Wind Conditions Directional Control Visual Lookout N (%) 44 (9.6) 44 (9.6) 35 (7.6) Subject Airspeed Aircraft Control Proper Altitude FATAL N (%) 35 (17.9) 23 (11.7) 16 (8.2)

DECISION ERRORS
NON - FATAL Subject Unsuitable Terrain Selection In-Flight Planning/Decision Planning/Decision NON - FATAL N (%) 46 (19.3) 35 (14.7) 22 (9.2) FATAL Subject In-Flight Planning/Decision Flight into Known Adverse Weather Planning/Decision FATAL Subject Intentional VFR Flight into IMC Procedures/Directives Aircraft Weight and Balance N (%) 40 (35.1) 9 (7.9) 8 (7.0)

VIOLATIONS
N (%) 23 (19.5) 20 (16.9) 12 (10.2)

Subject Procedures/Directives Intentional VFR Flight into IMC Checklist

N (%) 37 (30.6) 28 (23.1) 9 (7.4)

Note: Percentages in the table reflect the percentage of a given causal factor within the HFACS category (e.g., compensation for wind conditions accounted for 44 of 459 [9.6%] skill-based errors occurring in non-fatal commuter accidents).

11

tal

Regional Comparison Our previous investigation of GA accidents (Detwiler, Hackworth, Holcomb, Boquet, Pfleiderer, Wiegmann, & Shappell, 2006) suggested that differences in the pattern of human error associated with commuter accidents in Alaska versus the rest of the U.S. might exist. However, unlike GA, our regional investigation of commuter aviation accidents revealed no significant differences between Alaska and the rest of the U.S. with regard to the various Figure 13. 135 Unsafe Acts Under Impoverished Conditions categories of unsafe acts. Even the fine-grained analysis Resulting in Figure 7. Injury severity by visual conditions. a Fatality of unsafe acts revealed similar patterns for commuter accidents occurring in Alaska and the rest of the U.S. (Table 6). For instance, the failure to maintain adequate 100 altitude/clearance was the most frequently cited skill-based 90 error in Alaska and the rest of the U.S. 80 The only notable difference involved the type of vio70 lations and decision errors committed in Alaska versus 60 the rest of the U.S. Specifically, while the most common 50 violation occurring in the rest of the U.S. involved the 40 failure to adhere to procedures and directives; intentional 30 VFR flight into IMC was more common in Alaska. It was 20 10 also noteworthy that the decision to take off or land on 0 unsuitable terrain was observed more often in Alaska. Skill-based Decision Perceptual Violation
100 90 80 70 60 50 40 30 20 10 0 FAR

Percentage of Accidents

Error

Error

Error

Figure 8. Percentage of unsafe acts committed by commuter aircrews during impoverished visual conditions that resulted in fatalities.

DISCUSSION
In the present study, we examined a variety of human and environmental factors associated with more than 1000 commercial aviation accidents over a 13-year time frame. Given the sheer number of causal factors associated with these accidents, one might believe that there are literally thousands of ways to crash an aircraft. The results of this study, however, demonstrate that accidents appearing to be unique at first glance can be organized based upon underlying situational, demographic, and cognitive mechanisms of accident causation. In this way, previously unidentified trends in the accident record can be exposed. Overall Generally speaking, nearly 70% of the commercial aviation accidents occurring between 1990 and 2002 were associated with some manner of aircrew or supervisory error. However, the percentage varied slightly when air carrier (45%) and commuter (75%) aviation accidents were considered separately. This finding is consistent with results reported elsewhere (Li et al., 2001). However, while other studies typically focused on situational and demographic data, this study employed a human error framework (HFACS) to reveal the specific types of human error associated with commercial aviation accidents.

vs. non-fatal), it made sense to examine the combination of the two variables. As illustrated in Figure 7, the largest percentage of fatal commuter accidents occurred in visually impoverished conditions. In contrast, when the accident occurred in clear visual conditions, a much smaller percentage resulted in fatalities. Indeed, commuter accidents were over four times more likely to result in fatalities if they occurred in visually impoverished conditions (2 = 83.978, p<.001; odds ratio = 4.256). Perhaps more important, skill-based errors were still the most frequently cited human error during fatal accidents in impoverished visual conditions (Figure 8). However, the differences observed in previous analyses between skill-based errors, decision errors, and violations were much less obvious. Still, fully one-half of the fatal accidents occurring in visually impoverished conditions involved at least one violation often intentional VFR flight into IMC (Table 5). Not surprising, given the environmental conditions at the time, poor in-flight planning (decision error) was also commonly cited among this subset of the data.
12

Table 5. Fine-Grained Analysis for Fatal Accidents Associated with Commuter Operations in Impoverished Conditions.
SKILL BASED ERRORS
FATAL and IMPOVERISHED Subject Airspeed Proper Altitude Aircraft Control N (%) 19 (16.0) 16 (13.4) 15 (12.6)

DECISION ERRORS
FATAL and IMPOVERISHED Subject In-Flight Planning/Decision Flight into Known Adverse Weather Unintentional VFR Flight into IMC FATAL and IMPOVERISHED N (%) 26 (32.5) 7 (8.8) 5 (6.3)

VIOLATIONS

Subject Intentional VFR Flight into IMC Procedures/Directives IFR Procedure

N (%) 33 (31.7) 13 (12.5) 11 (10.6)

Note: Percentages in the table reflect the percentage of a given causal factor within the HFACS category (e.g., airspeed accounted for 19 of 119 [16%] skill-based errors occurring in impoverished conditions where a fatality occurred).

Table 6. Commuter Unsafe Acts Fine-Grained Analysis for Alaska versus the Rest of the U.S.
SKILL BASED ERRORS
Alaska Subject Altitude/Clearance Compensation for Winds N (%) 52 (20.4) 29 (11.4) Rest of the U.S. Subject Altitude/Clearance Aircraft Control N (%) 66 (16.5) 36 (9.0)

DECISION ERRORS
Alaska Subject Unsuitable Terrain Selection In-Flight Planning/Decision Alaska N (%) 39 (32.0) 18 (14.8) Rest of the U.S. Subject In-Flight Planning/Decision Pre-flight Planning/Decision Making Rest of the U.S. Subject Procedures/Directives Intentional VFR Flight into IMC N (%) 57 (24.8) 21 (9.1)

VIOLATIONS
N (%) 38 (42.2) 14 (15.6)

Subject Intentional VFR Flight into IMC Procedures/Directives

N (%) 40 (26.8) 19 (12.8)

Note: Percentages in the table reflect the percentage of a given cause factor within the HFACS causal category (e.g., compensation for wind conditions accounted for 29 of 255 [11.4%] skill-based errors occurring in Alaskan commuter accidents).

13

Organizational Influences and Unsafe Supervision Consistent with previous work (Wiegmann & Shappell, 2001a), comparatively few commercial aviation accidents were associated with organizational and/or supervisory causal factors - particularly within the commuter aviation industry. In spite of this, a relatively large proportion of accidents involved issues related to operational processes. Causal factors associated with the remaining HFACS organizational causal categories, resource management and organizational climate, were rarely observed in the data. A closer inspection revealed that the particular type of operational process cited appeared to be dependent on the type of operation involved. Namely, air carrier accidents were typically associated with the manner in which procedures or directives were communicated assuming they existed at all. In contrast, commuter accidents were more often associated with a lack of organizational oversight. Exactly why this difference might exist requires a more in-depth investigation than what was performed here. However, the data do provide some insight into the types of organizational influences that have impacted commercial aviation safety. Like organizational influences, causal factors attributed to middle-management centered on a single causal category (i.e., inadequate supervision) rather than the full range of unsafe supervision described within the HFACS framework. That being said, nearly 1 in 10 air carrier accidents were associated with some manner of inadequate supervision. However, unlike organizational factors, large differences were not observed between air carrier and commuter operations. Instead, when supervisors were identified as causal in the chain of events leading to an accident, issues such as the lack of general supervision/ oversight or the failure to provide adequate training were usually identified. Nevertheless, a larger question looms over the commercial accident data. Namely, Does the current accident data reflect the scope of the organizational/supervisory problem within commercial aviation, or is it possible that issues associated with middle- and upper-level management are under-reported? Consider, for example, a recently published report in which 48 accidents across the spectrum of civil aviation in India were examined using HFACS (Gaur, 2005). Of these, nearly half (21/48) involved aircraft operations similar to those reported here. Although it was not possible to separate their summary findings by type of operation, it is interesting to note that Gaur reported a large percentage of accidents were attributed, at least in part, to organizational influences (52%) and unsafe supervision (25%). Presumably, most of these were associated with Indian commercial aviation, since GA operations are often not associated with the upper tiers
14

of HFACS (Wiegmann et. al., 2005). To the extent that management of U.S. air carriers can be compared with foreign-flagged air carriers, at least this research suggests that current accident investigations may not capture all the organizational influences associated with commercial aviation accidents. At a minimum then, a review of how investigators are trained on organizational and supervisory influences of accident causation may be in order. It might also prove beneficial to incorporate the use of a human error framework that includes supervisory and organizational components. Preconditions for Unsafe Acts (Aircrew) With a couple of notable exceptions, causal categories within the preconditions for unsafe acts were also lightly populated. One of those exceptions was the large proportion of accidents (particularly among commuter aviation) influenced by prevailing weather conditions and reduced visibility. This was not particularly surprising since studies like the one conducted by Baker, Lamb, Li, and Dodd (1993) reported similar results in their examination of commuter accidents between 1983 and 1988. However, what makes this particular finding noteworthy is that the problem appears to have persisted even though the FAA and its industry partners have gone to great lengths over the last several years to improve pilot skills and weather decision-making. Likewise, a sizeable effort has been invested in crew resource management training, particularly within the air carrier industry. However, in the two decades since its implementation, the debate continues over whether or not these pioneering efforts have been effective (Salas, Burke, Bowers, & Wilson, 2001). After all, the findings here and elsewhere (U.S. General Accounting Office, 1997; Wiegmann & Shappell, 2001a) suggest that failures of CRM still contribute to a large proportion of commercial aviation accidents. Even so, there may be reason for guarded optimism. While on average nearly one in five air carrier accidents examined here were due, at least in part, to a CRM failure, the percentage dropped dramatically to just one out of 55 accidents in 2002, and that one involved an air carrier. Whether this was a statistical blip on the screen or a sustained improvement in the area remains to be seen. While previous efforts suggested that factors associated with the physical environment and crew resource management would be identified among the commercial data, it was surprising that other areas, in particular the condition of the operator (aircrew), were not identified in the accident record more often. The exception involved commuter aviation accidents, where a number of adverse mental states (64 out of 839 accidents, or 7.2%) and physical/mental limitations (43 out of 839, or 4.6%) were observed.

In some ways, the fact that many commuter aviation operations are single-piloted may explain why adverse mental states played a more prominent role among these accidents. For instance, without a second set of eyes in the cockpit any distraction would likely be exacerbated and detract the pilot from the task at hand flying the aircraft. Likewise, the aviation literature is ripe with examples where pressure, either self-induced or from management, has led pilots to accept risks beyond their abilities. At least one study suggests that this has been an issue with commuter aviation in Alaska (Conway, Mode, Berman, Martin, & Hill, 2005). Understandably then, diverted attention and pressure (whether self-induced or from management) were occasionally cited in the commuter accident record. Because of this, it seems that some manner of risk management training and/or simply reinforcing the basic tenets of aviation (i.e., aviate, navigate, and communicate in that order) should be a component of any intervention strategy employed by the commuter aviation industry. Perhaps more disconcerting than issues of attention and psychological pressure were the large number of commuter aviation accidents associated with the pilots lack of experience something rarely seen among the air carrier accidents examined. Whether this represents a lack of flight hours or merely inexperience with a particular operational setting or aircraft remains to be determined. Still, flight hours alone may not be sufficient to overcome the lack of experience observed here. After all, flying straight and level in VMC will not prepare a pilot for the complexities of instrument flight or the dangers of flying in other potentially hazardous environments. Unsafe Acts of Operators (Aircrew) As with our previous efforts involving civil and military aviation (Wiegmann & Shappell, 1997, 1999, 2001a, 2001b), skill-based errors were the most prevalent form of aircrew error among the commercial aviation accidents examined. Particularly widespread were technique errors associated with handling or controlling the aircraft. More important, when the commercial data reported here were combined with our previous investigations of GA accidents (Wiegmann et al., 2005; Detwiler et. al., 2006) an interesting finding emerged. It appears that the percentage of skill-based errors associated with accidents increases systematically as one moves from air carrier (43%) to commuter (60%) to GA (73%) operations. At first glance, this would appear to suggest that pilot skill and proficiency are best among the air carrier industry and become progressively more suspect within commuter and GA. Recall that skill-based errors, by definition, occur during the execution of routine events (Reason, 1990; Rasmussen, 1982). Furthermore, once a particular skill
15

is developed, it must be maintained through repetition and experience. Thus, most people would agree that GA pilots fly less and participate in fewer recurrent training sessions than their commercial counterparts. It stands to reason that their proficiency would be less than their commercial counterparts and may explain why skill-based errors are more prevalent among GA accidents. However, the same cannot be said for commuter pilots that in some cases receive more flight time than air carrier pilots and may participate in the same level of recurrent training. Instead, the data seem to suggest that commuter aircrews fall somewhere in between air carrier and GA pilots with regard to proficiency something that, if true, may necessitate additional regulations and currency requirements beyond what exists within the industry. On the other hand, the data may simply reflect wellknown differences in the sophistication of the aircraft being flown (e.g., was the aircraft instrument certified, was it outfitted with conventional instrumentation or technically advanced avionics). Or it could reflect operational requirements placed on the aircrews (e.g., flying very structured, well-planned operations versus comparatively less structured flights). Perhaps there is something still that has yet to be considered. Regardless, skill-based errors were not the only error form identified within the commercial aviation database. Decision errors, violations, and, to a lesser extent, perceptual errors were found in a large proportion of the accidents examined. For example, decision errors were observed in roughly four out of every ten commercial aviation accidents, while violations and perceptual errors were observed in 23% and 7% of the accidents, respectively. Some have even argued that decision errors and violations are of the same ilk (i.e., both involve decisions by aircrew that go awry) and should actually be combined in the HFACS framework. If this were true, the combined causal category of decision error/violation would be roughly equivalent to that seen with skill-based errors. While on the surface, combining decision errors and violations may make sense, given that both involve conscious decisions, the motivation behind them, as well as the intervention strategies that have proven effective in the past, argue against it. As discussed earlier, violations represent the willful disregard for the rules and regulations and are often driven by intrinsic motivation, overconfidence, and other hazardous attitudes. In contrast, decision errors are often the result of a lack of knowledge and/or information, rather than ones attitude. Therefore, while scenario-based training, in-flight planning aids, and education may improve pilot decisionmaking, these approaches have been largely ineffective in stemming violations. Instead, enforcing current standards and increasing accountability in the cockpit may be the

only effective means to reduce violations of the rules a tactic that is often difficult to employ in civil aviation. As a result, the FAA and the commercial aviation industry may have to look to other avenues to reduce violations such as the use of flight simulators that can demonstrate the hazards associated with violating the rules (Knecht, Harris, & Shappell, 2003). Unlike skill-based errors, decision errors, and violations, perceptual errors contributed to the smallest percentage of commercial accidents, a percentage that was much less than that found in military research (Wiegmann & Shappell, 2003). However, given the non-tactical, non-aerobatic nature of commercial flight, this was not altogether unexpected. Whats more, a considerable effort has been brought to bear over the last several decades by the aerospace engineering and medicine communities to improve avionics, warning devices (ground collision avoidance systems), and awareness of perceptual errors due to visual and vestibular illusions. It would appear that those efforts have paid dividends. Still, it should also be noted that the differences observed between skill-based errors, decision errors, perceptual errors, and violations remained largely consistent across the 13 years of the study. The only possible exception was observed with air carrier accidents, where violations evidenced a small increase since 1993. Then again, some degree of caution should be taken in interpreting this particular finding, given that the air carrier data had to be collapsed into 3-4 year blocks due to the relatively small number of air carrier accidents occurring annually. What this implies is that interventions employed in the 1990s have had, at best, ubiquitous effects on the errors and violations committed by aircrew. Alternatively, it is quite possible that there has been no sustained impact of any particular intervention program. The latter should come as no surprise given that prior to this study, no comprehensive analysis of aircrew and supervisory error has been conducted using a theoretically derived framework of accident causation. 14 CFR Part 135 Commuter Operations One of the purposes of this study was to combine the power of traditional situational and demographic variables with a theoretically-based, human error framework to identify human error trends amid commercial aviation accidents. However, because of the sample size, only commuter aviation lent itself to this sort of analysis. Visual Conditions and Injury Severity With the development of sophisticated navigation instrumentation and other avionics, it is possible to fly safely in environments without any external visual cues. Yet, piloting an aircraft into visually impoverished
16

environments without the necessary instruments or training can, and often does, lead to disaster. One needs to look no further than the accident data reported here to see the magnitude of this hazard to commuter aviation. That is, nearly one-half of all commuter accidents occurred in a visually impoverished environment. Of those, an alarming 70% resulted in fatalities. In contrast, only about 30% of the accidents that occurred in broad daylight resulted in a fatality. Although interesting, this finding alone contributes little to our understanding of why aircraft crash in the weather or at night. However, when combined with HFACS, a distinguishable pattern of human error emerged. Indeed, while skill-based and decision errors were cited in a large proportion of these accidents, violations of the rules and regulations were five times more likely to occur during accidents in visually impoverished than in clear conditions. That is, intentional VFR flight into IMC, poor in-flight planning, and simply the failure to control the aircraft all were commonly associated with fatal accidents-particularly when they occurred in visually impoverished environments. Whats more alarming, many of these causal factors have been identified to some extent in the past (e.g., Baker, Lamb, Li, & Dodd, 1993). So why is this still a problem and, more importantly, how could a professional pilot make such a decision to fly into hazardous weather? At least one study (Burian, Orasanu, & Hitt, 2000) suggests that pilots with less experience may not trust what their eyes are telling them and so proceed on blindly (p. 25). Referred to as plan continuation errors, Wiegmann, Goh, and OHare (2002) suggest that under certain conditions these errors are more often attributable to poor situation assessment than to motivational judgment, per se. In other words, sometimes experienced pilots simply misjudge the situation and make an honest mistake. Regardless, proper planning, both in the air and on the ground, is a critical component of flight safety. The solution may be to improve the quality of weather-related information to the pilot so that sound go/no-go decisions can be made. However, it is one thing to misjudge weather information or make a bad decision, it is quite another to willfully fly into IMC without proper training or equipment. Such an act begs the question, Why would someone take such an exceptional risk? One possibility is social pressure. Indeed there are several examples of pilots being pressured by passengers or other aircrew to continue to their destination despite cues that they should do otherwise (Holbrook, Orasanu, & McCoy, 2003). In fact, at least for GA, the presence of passengers on board seems to influence the likelihood that an accident would be associated with VFR flight into IMC (Goh & Wiegmann, 2002).

Still, social pressures cannot fully explain why a pilot would elect to fly VFR into IMC particularly during cargo or repositioning flights where no passengers are on board. Alternatively, OHare and his colleagues (Batt & OHare, 2005; OHare & Owen, 1999; OHare & Smitheram, 1995) have offered an explanation structured around how pilots frame the situation of continuing or discontinuing flight into adverse weather. They found that pilots who framed diverting from a flight plan as a loss (e.g., loss of time, economic loss, or expense of effort) tend to continue flight into adverse weather; whereas those who frame a rerouting decision as a gain (e.g., in personal safety) tend to divert more. Indeed, gains and losses take on more meaning as pilots get closer to their destination. Another possibility is that commuter pilots, on average, may not have the requisite experience to decide when a particular situation is beyond their ability. While this argument may hold for GA, where pilot experience varies from the novice to pilots with thousands of flight hours, commuter pilots typically have more experience than their GA counterparts well before their first paying passenger boards the aircraft. However, experience is a double-edged sword as others (e.g., Thomson, Onkal, Avcioglu, & Goodwin, 2004) have suggested that, as pilots gain experience through more flight hours, risk taking may also increase due to overconfidence and successful exposure to risky events. Put simply, experts may be more likely to take risks than novices. Regional Differences In many ways, Alaska is the one of the worlds most demanding aviation environments, offering virtually every situation a pilot or operator might be confronted with. In a sense, there are very few situations experienced by pilots in the lower 48 states that have not been experienced by those in Alaska. Perhaps this is why few differences were observed in the pattern of human error associated with Alaska and the rest of the U.S. However, one area where differences did exist was the violation of the rules and regulations; to be specific, VFR flight into IMC. Precisely why commuter pilots would be more prone to fly into adverse weather in Alaska than the rest of the U.S. is unknown, but at least one study (Conway, et. al., 2005) has shown that aircrews of highrisk operators in Alaska (those with a higher fatal crash rate than would be expected given the number of pilots they employed) differed from other operators in both experience and working conditions. On average pilots of high risk operators worked one hour more per day and 10 hours more per week than control pilots. They were also more likely to fly into unknown weather conditions.
17

Although their study did not identify any specific reason these pilots were more prone to take risks, it did suggest that factors such as pilot fatigue and experience, financial pressures on operators, and inadequate weather information, particularly in combination, may provide some clues. Another area where regional differences existed involved takeoff and landing from unsuitable terrain. Although rarely associated with fatalities, these accidents are no less important given the staggering cost to recover an aircraft stuck on a sandbar or some other remote area. Unlike VFR flight into IMC, these accidents are much easier to understand because there are simply not many concrete runways and taxiways in Alaska. Instead, Alaskan commuter pilots may have to resort to frozen ice, sandbars, and other natural runways for support. Not surprising, what appears suitable from the air turns out to be unsuitable for aircraft when landed upon. The obvious solution is to provide more suitable runways; pour more concrete, if you will. However, given the remoteness and harsh conditions of some of these areas, providing traditional runways would not be practical. Alternatively, some sort of training and awareness of what constitutes a suitable landing area, combined with the creation of more traditional runways, where possible, may be the only viable solution. In light of the unique nature of the Alaskan environment, the FAA and Alaskan aviation community have joined efforts to employ a variety of safety programs aimed at reducing accidents associated with commuter operations. With programs like the FAAs Circle of Safety and Capstone, and non-profit aviation safety organizations like the Medallion Foundation, it is hoped that improvements in Alaskan aviation safety will be realized.

CONCLUSIONS
We are often told that sometimes the best studies ask more questions than they answer. If that sage wisdom is indeed true, then perhaps the present study was worthwhile. Regardless of ones opinion of accident data and the current aviation accident investigation process, these data represent our best understanding of the underlying human error component of commercial aviation accidents. Even more, the results presented here represent the marriage of traditional demographic and human error analyses of commercial aviation (air carrier and commuter). While some of the findings may come as no surprise, they do provide data, where often only opinion existed. Whats more, they provide a foundation for the development, implementation, and quantifiable assessment of putative intervention and mitigation strategies.

REFERENCES
Baker, S. (1995). Putting human error into perspective. Aviation, Space, and Environmental Medicine, 66, 521. Baker, S., Lamb, M., Li, G., & Dodd, R. (1993). Human factors in crashes of commuter airplanes. Aviation, Space, and Environmental Medicine, 64, 63-8. Batt, R. & OHare (2005). Pilot behaviors in the face of adverse weather: A new look at an old problem. Aviation, Space and Environmental Medicine, 76, 552-9. Billings, C. & Reynard, W. (1984). Human factors in aircraft incidents: Results of a 7-year study. Aviation, Space, and Environmental Medicine, 55, 960-5. Burian, B., Orasanu, J., & Hitt, J. (2000). Weatherrelated decision errors: differences across flight types. Proceedings of the XIVth Triennial Congress of the International Ergonomics Association/44th Annual Meeting of the Human Factors and Ergonomics Society, pp. 22-4. Conway, G., Mode, N., Berman, M., Martin, S., & Hill, A. (2005). Flight safety in Alaska: Comparing attitudes and practices of high- and low-risk air carriers. Aviation, Space, and Environmental Medicine, 76, 52-7. Detwiler, C., Hackworth, C., Holcomb, K., Boquet, A., Pfleiderer, E., Wiegmann, D., & Shappell, S. (2006). Beneath the tip of the iceberg: A human factors analysis of general aviation accidents in Alaska verses the rest of the United States. Federal Aviation Administration, Office of Aerospace Medicine Technical Report No. DOT/FAA/AM-06/7. Office of Aerospace Medicine: Washington, DC. Gaur, D. (2005). Human factors analysis and classification system applied to civil aircraft accidents in India. Aviation, Space, and Environmental Medicine, 76, 501-5. Goh, J. & Wiegmann, D. (2002). Human error analysis of accidents involving visual flight rules flight into adverse weather. Aviation, Space, and Environmental Medicine, 78(8), 817-22. Heinrich, H., Peterson, D., & Roos, N. (1931). Industrial accident prevention: A safety management approach (1st ed.). New York, NY: McGraw-Hill.

Holbrook, J., Orasanu, J., & McCoy, C. (2003). Weatherrelated decision making by aviators in Alaska. Proceedings of the 12th International Symposium on Aviation Psychology, 576-81. Knecht, W., Harris, H., & Shappell, S. (2003). Effects of visibility, cloud ceiling and financial incentive on general aviation voluntary takeoff into adverse weather. Proceedings of the 12th International Symposium on Aviation Psychology, 669-73. Li, G., Baker, S., Grabowski., J., & Rebok, G. (2001). Factors associated with pilot error in aviation crashes. Aviation, Space, and Environmental Medicine, 72, 52-8. Nicogossian, A.E., Huntoon, C.L., & Pool, S.L. (1994). Space physiology and medicine (3rd ed.). Baltimore, MD: Williams and Wilkins. OHare, D. & Owen, D. (1999). Continued VFR into IMC: An empirical investigation of the possible causes; Final report on preliminary study. Unpublished manuscript. University of Otago, Dunedin, New Zealand. OHare, D. & Smitheram, T. (1995). Pressing on into deteriorating conditions: An application of behavioral decision theory to pilot decision making. International Journal of Aviation Psychology, 5, 351-70. Rasmussen, J. (1982). Human errors: A taxonomy for describing human malfunction in industrial installations. Journal of Occupational Accidents, 4, 311-33. Reason, J. (1990). Human error. New York, NY: Cambridge University Press. Reinhart, R.O. (1996). Basic flight physiology (2nd ed.). New York, NY: McGraw-Hill. Salas, E., Burke, C.S., Bowers, C.A., & Wilson, K.A. (2001). Team training in the skies: Does crew resource management training work? Human Factors, 4, 641-74. Shappell, S. & Wiegmann, D. (1995). Controlled flight into terrain: The utility of an information processing approach to mishap causal factors. Proceedings of the Eighth International Symposium on Aviation Psychology, The Ohio State University, 1300-6. Shappell, S. & Wiegmann, D. (1996). U.S. naval aviation mishaps 1977-92: Differences between single- and dual-piloted aircraft. Aviation, Space, and Environmental Medicine, 67(1), 65-9.
18

Shappell, S. & Wiegmann, D. (1997). Why would an experienced aviator fly a perfectly good aircraft into the ground? Proceedings of the Ninth International Symposium on Aviation Psychology, The Ohio State University, 26-32. Shappell, S. & Wiegmann, D. (2003a). A human error analysis of general aviation controlled flight into terrain (CFIT) accidents occurring between 19901998. Federal Aviation Administration, Office of Aerospace Medicine Technical Report No. DOT/ FAA/AM-03/4. Office of Aerospace Medicine: Washington, DC. Shappell, S. & Wiegmann, D. (2003b). Reshaping the way we look at general aviation accidents using the human factors analysis and classification system. Proceedings of the Twelfth International Symposium on Aviation Psychology, The Ohio State University). Shappell, S. & Wiegmann, D. (2004). HFACS analysis of military and civilian aviation accidents: A North American comparison. Proceedings of the Annual Meeting of the International Society of Air Safety Investigators, Gold Coast, Australia. Thomson, M., Onkal, D., Avcioglu, A., & Goodwin, P. (2004). Aviation risk perception: A comparison between experts and novices. Risk Analysis, 6, 1585-95. United States General Accounting Office (1997). Human Factors: FAAs guidance and oversight of pilot crew resource management training can be improved. (GAO/RCED-98-7). Washington, DC: Author. Wickens, C.D. & Hollands, J.G. (2000). Engineering psychology and human performance (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

Wiegmann, D., Goh, J., & OHare, D. (2002). The role of situation assessment and flight experience in pilots decisions to continue visual flight rules flight into adverse weather. Human Factors, 44(2), 189-97. Wiegmann, D. & Shappell, S. (1997). Human factors analysis of post-accident data: Applying theoretical taxonomies of human error. The International Journal of Aviation Psychology, 7(1), 67-81. Wiegmann, D. & Shappell, S. (1999). Human error and crew resource management failures in naval aviation mishaps: A review of U.S. Naval Safety Center data, 1990-96. Aviation, Space, and Environmental Medicine, 70(12), 1147-51. Wiegmann, D. & Shappell, S. (2001a). Human error analysis of commercial aviation accidents: Application of the Human Factors Analysis and Classification System (HFACS). Aviation, Space and Environmental Medicine, 72, 1006-16. Wiegmann, D. & Shappell, S. (2001b). Human error perspectives in aviation. International Journal of Aviation Psychology, 11(4), 341-57. Wiegmann, D. & Shappell, S. (2003). A human error approach to aviation accident analysis: The human factors analysis and classification system. Burlington, VT: Ashgate. Wiegmann, D., Shappell, S., Boquet, A., Detwiler, C., Holcomb, K., & Faaborg, T. (2005). Human error and general aviation accidents: A comprehensive, fine-grained analysis using HFACS. Federal Aviation Administration, Office of Aerospace Medicine Technical Report No. DOT/FAA/AM-05/24. Office of Aerospace Medicine: Washington, DC.

19