FDA
Perspectives on
Human Factors
in Device Development
Molly Follette Story, PhD
FDA /CDRH / ODE
Understanding Regulatory Requirements
for Human Factors Usability Testing
RAPS Webinar – June 7, 2012
Overview
• What are human factors & usability?
• Relevant regulations and standards
• FDA’s human factors guidance
• Final words
2 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
What are Human Factors? Usability?
• Human factors: “…the application of knowledge
about human capabilities (physical, sensory,
emotional, and intellectual) and limitations to the
design and development of tools, devices,
systems, environments, and organizations….”
(ANSI/AAMI HE75:2009, Introduction)
• Usability: “Characteristic of the USER INTERFACE
that establishes EFFECTIVENESS, EFFICIENCY, ease
of USER learning and USER satisfaction” (ISO/IEC
62366:2007, Definition 3.17)
3 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Why is FDA Concerned about HF?
• Device safety
• Device effectiveness
4 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Ergonomics and “Hedonomics”
Usability
Effectiveness
Safety
Source: Hancock, Pepe & Murphy (2005), Ergonomics in Design, 13 (1), 8‐14
5 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Device‐User Interface
Information
Information Processing Control
Perception Actions
Human
I N T E R F A C E I N T E R F A C E
Machine
Output Input
Processing
6 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Device Use Errors
Transdermal Patch Products
• Original user instructions: where to apply patch
Source: Carlos M. Mena-Grillasca, FDA / CDER / OMEPRM
7
7 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Device Use Errors
Transdermal Patch Products
• Revised user instructions: where to apply patch
8
8 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Use‐Related Risks: Infusion Pumps
Hazard Corresponding
Use‐Related Cause(s) (samples)
(samples) Risk(s) to Health
Infusion stopped •Underdose The user forgets to resume the pump
prematurely •Delay of therapy after suspending it
User is unaware of battery capacity
The user fails to •Overdose Background noise or nuisance alarms
detect or •Underdose cause user to fail to detect/ignore them
understand pump •Delay of therapy User muffles pump’s speaker/audio,
notifications •Incorrect therapy either intentionally or unintentionally
Wrong •Incorrect therapy User selects and sets up pump with
medication or •Delay of therapy incorrect medication or concentration
concentration is Medication is correct but user selects
delivered incorrect concentration or delivery rate
9 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
FDA Regulations Relevant to HF
Quality System regulation:
21 CFR 820.30, Design Controls
– The need for human factors is implied:
c) Design input – includes “needs of the user and patient”
f) Design verification – performance criteria met
g) Design validation – “… devices conform to defined user
needs and intended uses and shall include testing of
production units under actual or simulated use
conditions. Design validation shall include software
validation and risk analysis….” [incl. use‐related risks]
10 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
820.30(c) Design Input
• User and Patient Needs
– Relative to device safety
• Users can operate the device without injury or negative clinical
consequences to either the user or the patient
– Relative to device effectiveness
• Users can operate the device successfully for the intended uses
• Potential use errors and failures have been
eliminated or limited to the extent possible
through appropriate application of human factors
methods
11 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
820.30(f) & (g) Design Verif. & Valid.
• Design Verification:
– Did I make the product right?
• Design Validation:
– Did I make the right product?
12 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors Standards
AAMI/ANSI HE75:2009
• General considerations and
principles
– Managing the risk of use error
– Usability testing
• Design elements
– Controls
– Software
• Integrated solutions
– Mobile medical devices
– Home health care
13 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors Standards
ISO/IEC 62366:2007
Medical devices – Application
of usability engineering to
medical devices
• Usability engineering process
• Accompanying document
• Training
– Note: Currently undergoing
revision
14 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors Standards
ANSI/AAMI/ISO
14971:2007
Medical devices – Application
of risk management to
medical devices
• Risk management
• Risk analysis
• Risk evaluation
• Evaluation of overall residual
risk acceptability
15 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
FDA HF Guidance
Medical Device Use‐Safety:
Incorporating Human
Factors Engineering into
Risk Management
[Link]
MedicalDevices/
DeviceRegulationand
Guidance/
GuidanceDocuments/
[Link]
– NOTE: Issued in 2000
16 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Draft Guidance
[Link]
MedicalDevices/
DeviceRegulationand
Guidance/
GuidanceDocuments/
[Link]
– NOTE: Issued in 2011 –
not yet in effect but
reflects FDA‐CDRH’s
current thinking and
approach to human
factors.
17 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
2011 Draft Human Factors Guidance
• Regulations, Guidance, Standards
• Considerations: Device Users, Use
Environments and User Interfaces
• Preliminary Analyses and Hands‐On Studies
• Risk Control and Design Iteration
• Human Factors Validation Studies
18 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Define intended use, users,
environment 1
Human factors
engineering Identify use related hazards 2
process for Estimate & prioritize use
3
medical error risk
devices Implement risk controls 4
Yes
No Validate safety of use 5
Risk New risks
Acceptable? Introduced? 6
yes no
Document process 7
Source: ANSI/AAMI
Monitor unanticipated risks
HE75:2009 in post market
19 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors of Device Use
Define intended use,
users, environment 1
20 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Users
• Professional or non‐professional
– Job title and responsibilities
• Knowledge and experience levels
• Age and functional capabilities
– Physical, sensory/perceptual, cognitive/intellectual
• Mental and emotional condition
21 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Use Environment
• Clinical environment
– Hospital, clinic, etc.
• Transitional care environment
– Rehabilitation, assisted living, long‐term care, etc.
• Home environment
– House, mobile home, townhouse, apartment, etc.
• Community setting
– Office, school, retail, outdoors, etc.
• Mobile environment
– Car, plane, train, bus, ambulance, medevac, etc.
22 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
User Interface
• Tasks
– Device set‐up: installation, assembly, calibration, etc.
– Device use: various aspects
– Device cleaning, maintenance, disposal, etc.
• Interactions
– Input
• Connections, knobs/dials, switches, buttons, touch screens, etc.
– Output
• Visual: component status, displays, lights, etc.
• Auditory: motors/fans, clicks, alerts/alarms, beeps, voice, etc.
• Tactile: resistance, vibration, temperature, etc.
23 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Identify use related hazards 2
Preliminary Studies Estimate & prioritize
use error risk 3
• Identify and analyze: intended users, use
environments, expected use scenarios
• Develop initial product concepts and prototypes
• Identify and explore potential device use‐
related hazards and risks and their potential
clinical consequences
• Explore different design alternatives and
identify the trade‐offs between them
24 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Preliminary HF Analyses
• Study use of current devices
– Contextual inquiry
– Focus groups and interviews
• Study use of device under development
– Function and task analyses, failure mode and effects
analyses, etc.
– Heuristic analyses
– Expert reviews
25 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Preliminary Hands‐On Studies
• Identify issues not findable through application
of analytical techniques, e.g.:
– The use environment has negative effects
– The demands associated with use of the device exceed
the user’s capabilities
– Aspects of device use are inconsistent with the user’s
expectations or intuition
– The device is used in unexpected ways
– The device is used in inappropriate but foreseeable
ways, for which adequate controls were not applied
26 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Preliminary Hands‐On Studies
• Done while the device is under development
– Testing can involve simple product mock‐ups or early
prototypes, or even partial designs (e.g., components)
operating in simulated‐use modes
– Test participants should be representative of the
intended user population(s)
– At early stages, use‐related problems can be addressed
more easily and less expensively
– Conduct studies iteratively to optimize the device
design and ensure the human factors/usability
validation testing results will be successful
27 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Preliminary Hands‐On Studies
• Conduct separate studies on labeling and
training (recommended):
– Assess the clarity and effectiveness of all labeling (e.g.,
instructions for use, other documentation, packaging)
– Determine the format and complexity of training that
will be necessary (if any)
28 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Risk Control Implement risk controls 4
• Develop risk mitigation strategies as needed:
– Modify the interface design, user instructions, and/or
training to address the problems found
• Re‐test to assess whether mitigation strategies:
– Effectively reduced the known risks and
– Did not introduce any new risks
• Residual risk can be acceptable if it is:
– Reasonably limited, difficult to eliminate or further
reduce, and outweighed by the device’s benefits
29 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors Validation Validate safety of use 5
• Demonstrates and provides evidence that a
medical device, as designed, can be used safely
and effectively:
– By people who are representative of the intended users
– Under expected use conditions
– For essential and critical (high‐risk) tasks
30 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Test Populations
• All intended user populations – e.g.:
– Healthcare professionals, pharmacists,
engineers/technicians, home health care aides,
family caregivers, patients
– Pediatric and geriatric populations need careful
consideration
• Not company employees
• Representative of U.S. population
31 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Device Testing Conditions
• Use finalized device design and labeling
• Present within typical context of use
• Incorporate expected use conditions
– E.g., lighting, sound, and activity (distraction) levels
• Allow realistic device‐user interactions
– Provide participants with device as they would receive it
(e.g., in its original packaging, with all documentation)
– Allow participants to use the device as they normally
would (e.g., without interference from test facilitator)
32 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Selection of Tasks Tested
• Tasks tested in validation do not necessarily
include everything in the instructions for use
– Include essential tasks – i.e., tasks necessary for
successful use of the device for its intended purpose
– Include safety‐critical tasks – i.e., tasks on which users
could make errors or could fail to complete, which
would have negative clinical impact
• All warnings and most caution statements in the device
labeling imply safety‐critical tasks
• All tasks requiring responses to alarms are safety‐critical
33 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Selection of Tasks Tested
• Include in the human factors validation study
protocol and report a rationale for the tasks you
choose to include in your testing
– Base task selection on results of preliminary analyses
• E.g., task analyses and identification of use‐related risks
– Incorporate findings of preliminary hands‐on studies
• E.g., tasks found to be error‐prone or difficult for users
– Define criteria for task success prior to the test
34 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Instructions and Training
• Participant interaction with instructions should
approximate reality
– Labeling used in device validation should be final versions
– Availability of labeling should represent realistic situation
• Provision of training should approximate reality
– Training used in device validation study should be
comparable to the training that actual users will receive
– Consider integrating period of time for learning gained
through training to “decay” before conducting device
validation study, depending on actual conditions of use
35 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors/Usability Data
• Objective (performance) data:
– Facilitator observes and notes all use errors, failures and
difficulties, including details about performance, e.g.:
• Task success or failure, use error, close call, reference to
instructions for use, need for assistance, evidence of difficulty or
confusion, unsolicited comments
– Facilitator should not interfere with participant’s
independent and natural use of device
– Measuring speed of task completion: only appropriate if
it is clinically meaningful
– Number of task attempts allowed: varies by type of
device and context of use
36 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors/Usability Data
• Subjective (narrative comment) data:
– Discuss user performance after use, particularly
regarding perceived reasons for any essential and
critical task errors, failures and difficulties
– Solicit participant feedback on design of device,
labeling and training
• Unacceptable methods and data:
– Use of “think aloud” technique: not natural
– Forced‐choice questionnaires: introduce bias
– User opinions on rating scales: can introduce bias,
usually does not demonstrate safety or effectiveness
37 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Validation Data Analysis Risk
Acceptable?
New risks
Introduced? 6
• Analyze all use errors and failures
– Determine root cause and potential clinical
consequences
– Determine need to modify device, labeling, or training
– Identify true residual risks
• Use errors/failures are not of equal importance
– Some errors might be frequent but inconsequential
– Some errors might be rare but reveal a hazardous
design deficiency that was not previously recognized
38 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Human Factors Report Document process 7
1. Intended device users, uses, environments
2. Description and images of device user interface
3. Summary of known use problems
4. Use tasks: descriptions, risk priorities, success
criteria
5. Summary of preliminary evaluations
6. Validation study protocol and results
7. Conclusion
39 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Status of Draft Guidance Document
• Document issued June 22, 2011
• Comment period closed Sept. 19, 2011
– About 600 comments were submitted
• We are reviewing the comments
– Some comments are minor and easy to address
– Some comments are significant and require ODE to
establish new policies
• We hope to finalize the document soon…
40 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
FDA Expectations for HF Data
• Conduct a comprehensive risk assessment
• Identify and mitigate risks, including all use‐
related risks
• Conduct human factors/usability validation
testing on any strategies implemented to
mitigate significant use‐related risks
• Document everything in the Design History File
41 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Advice: Consult FDA Early
• Discuss product development plans with FDA
before your design is finalized (and changes
would be difficult)
– Through a pre‐submission (e.g. “Pre‐IDE”) mechanism
– Interact with experts in CDRH (and/or CDER/CBER) who
have advice to share based on experience
• FDA will review human factors/usability
validation study protocols on request
– Before implementation is recommended!
42 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
CDRH “Pre‐IDE” Meetings
• Can be used to interact with FDA regarding any
future submission or application
– Can request a face‐to‐face meeting or teleconference or
simple written response
– Quicker turnaround: FDA often responds within 60 days
– Can be used to request review of human factors
validation study protocols
• We try to turn these around within 2‐3 weeks
• Guidance on IDE Policies and Procedures:
– [Link]
GuidanceDocuments/[Link]
43 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
FDA/CDRH Human Factors Program
[Link]
humanfactors
• Premarket Info
– Design & Documentation
• Postmarket Info
– Surveillance & Reporting
• Info for Consumers,
Patients, Caregivers
• General Human Factors
Info and Resources
44 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
New HFES‐AAMI Web Site
[Link]
[Link]
[Link]
• Resources
• Consultant Directory
• Organizations
• Events
45 Human Factors Regulations & Standards FDA’s HF Guidance Final Words
Questions
Molly Follette Story: [Link]@[Link]
FDA/HF web site: www/[Link]/humanfactors
46
