Predictive Aircraft Maintenance:
Detecting Imminent Part Failure with Cox Regression and Advanced Ensemble Learning Methods
2Lt Abraham Eaton Advised By: Dr. Allison Chang Sponsored By: MIT Lincoln Laboratory,
2Lt Joshua Couse Dr. Chelsea Curran Group 43,
Ms. Cindy Engholm Air Traffic Control Systems
Mr. Yan Glina
Problem Statement
Problem Description: The United States Air Force spends $50B+ yearly in aircraft operations and Project Timeline:
maintenance. Maintaining complex and aging aircraft requires large maintenance crews and around the Jan — April May—June July—Aug
clock operations. Over the past several years, aircraft mission capable rates have been declining while
maintenance costs have increased. How can the Air Force use machine learning to predict when com-
Preliminary Modeling Model Building
ponents will require maintenance in order to keep planes flying and our nation safe? Problem Framing
Feature Engineering Parameter Tuning
Project Goal: Develop an end to end machine learning pipeline that utilizes historical maintenance Data Exploration
Feature Selection Presenting Results
records to predict near term component failure.
Data
Data Modeling Objectives
• Predict rarely occurring failures of
Air Force Aircraft Maintenance Log Data Cleaning Pipeline Final Dataset specific aircraft components
• Prediction must be early enough to allow for
• 167k Observations maintenance crew and supply adjustments
• Details a specific part removal, replacement, • Filtered data to include accurate tail numbers
installation or inspection, and date completed • Condensed records into daily maintenance sum- • Focus on components with largest
Indicates type of malfunction, and type of mainte-
•
mary by aircraft tail unscheduled maintenance requirements
nance (scheduled v. unscheduled) • Completed time-series for missing dates
• 3.5 Million records spanning 12 years Generated additional features
• Anonymous Tail Numbers
•
• Maximize failure detection rate while
• Total breakages, total man hours, total un-
• 800k Blank Values limiting false positive rate to tolerable
scheduled maintenance jobs, etc measure
• No Dependent Variable • Created indicator for part breakage
Modeling
1. Decide Which Component Failure to Predict 2. Build Variety of Models and Evaluate Performance
Top Performing Baseline Model
• Our team tested a variety of Model AUC
machine learning methods to
determine which approaches
performed the best on our pre- CART 0.53
diction task.
• After finding mediocre results Logistic
using classification methods we 0.62
Regression
decided to reframe our problem
as time-to-event prediction task
Random
0.66
• Using a Cox regression model Forest
allowed us to take advantage of
the time-varying covariates in
our survival analysis Boosting 0.68
Wheel and Tire failures occur frequently and require the
Prediction Variable: Binary indicator of Wheel & Tire failure within 5 days
most total man-hours out of all components
3. Develop Survival Model 3. Final Building Model and Performance
Survival Analysis
Final Dataset Predictions
Median Time-to-Failure:
90 Days
• Utilizing the survival model’s predictions as an
additional feature to a gradient boosting model
improved performance
• Tuning hyperparameter such as depth, learning rate, L2 True Positive False Positive
AUC
normalization constant, and training iterations ensures Rate Rate
best out of sample model performance 0.88 0.57 0.09
• The survival curve above demonstrates how the wheels
Impact and Conclusion
& tires of an aircraft deteriorate over time
Developed effective data management pipeline to
Savings:
•
• A Survival model outputs a survival probability for each transform millions of records into a series of concise and
day a part is observed providing a statistic of component
informative daily maintenance summaries
health
• Our model was able to preemptively identify 2,531 part
~$27M
• The median time-to-failure metric represents the point in
time when half of the components have failed and is a
valuable metric for aircraft maintainers
failures before they occurred, showing potential to save
approximately $27M Mission Hours:
• Identified an opportunity to add 1,501 available flight 1.5K+ Available Flight Hours
hours
