Crosswalk Between BSA Framework to Build Trust in

AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

GOVERNANCE

Policies and
Governance Framework

Objectives: Integrate AI risk Sec. 1.2.4: Organizational Integration and Management of Risk: AI
Processes management into broader risk risk management should be integrated and incorporated into broader
management functions. enterprise risk management strategies and processes. Treating AI risks
along with other critical risks, such as cybersecurity and privacy, will yield
a more integrated outcome and organizational efficiencies.

Processes: Establish processes Govern 1: Policies, processes, procedures, and practices across the
for identifying risks, assessing organization related to the mapping, measuring, and managing of AI
the materiality of those risks, risks are in place, transparent, and implemented effectively.
and mitigating risks at each
stage of the AI lifecycle. Govern 4.3: Organizational practices are in place to enable AI testing,
identification of incidents, and information sharing.

Govern 6.2: Contingency processes are in place to handle failures or

incidents in third-party data or AI systems deemed to be high-risk.

Measure 3: Mechanisms for tracking identified AI risks over time are in

place.

Manage 3: AI risks and benefits from third-party entities are managed.

Evaluation Mechanisms: Govern 1.5: Ongoing monitoring and periodic review of the risk
Establish mechanisms, such as management process and its outcomes are planned and organizational
metrics and benchmarks, that roles and responsibilities clearly defined, including determining the
the organization will use to frequency of periodic review.
evaluate whether policies and
procedures are being carried
out as specified.

Periodic Review: Organizations Govern 1.5: Ongoing monitoring and periodic review of the risk
should periodically review and management process and its outcomes are planned and organizational
update their AI governance roles and responsibilities clearly defined, including determining the
framework so it remains frequency of periodic review.
fit-for-purpose and capable
of addressing the evolving
landscape of risk.

Executive Oversight: Govern 2.3: Executive leadership of the organization takes responsibility
Governance framework should for decisions about risks associated with AI system development and
be backed by executive deployment.
oversight, including
(1) approval of governance
policies, (2) active role
in overseeing product
development lifecycle, and (3)
accountability for go/no-go
decisions for high-risk systems.

www.bsa.org 1
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

GOVERNANCE (continued)

Personnel,
Governance Framework

Independence: Personnel Govern 5.1: Organizational policies and practices are in place to collect,
Roles, and should be structured in consider, prioritize, and integrate feedback from those external to the
Responsibilities a manner that facilitates team that developed or deployed the AI system regarding the potential
separate layers of independent individual and societal impacts related to AI risks.
review. For example, risk
management responsibilities Measure 1.3: Internal experts who did not serve as front-line developers
may be split between a for the system and/or independent assessors are involved in regular
product development team, assessments and updates. Domain experts, users, AI actors external
a compliance team, and a to the team that developed or deployed the AI system, and affected
governance team. communities are consulted in support of assessments as necessary per
organizational risk tolerance.

Competence, Resourcing, and Govern 2.2: The organization’s personnel and partners receive AI
Influence: Provide adequate risk management training to enable them to perform their duties
training and resources for and responsibilities consistent with related policies, procedures, and
personnel to fulfill their agreements.
governance functions and
ensure that personnel are
empowered to address and/or
escalate risks.

Diversity: Establish team Govern 3.1: Decision-making related to mapping, measuring, and
with diverse perspectives and managing AI risks throughout the lifecycle is informed by a diverse team
lived experiences, and include (e.g., diversity of demographics, disciplines, experience, expertise, and
traditionally underrepresented backgrounds).
perspectives throughout the
lifecycle of the AI design and Map 1.2: Interdisciplinary AI actors, competencies, skills, and capacities
development process. Where for establishing context reflect demographic diversity and broad domain
diversity is lacking on internal and user experience expertise, and their participation is documented.
team, consult with external Opportunities for interdisciplinary collaboration are prioritized.
stakeholders as appropriate.
Measure 1.3: Internal experts who did not serve as front-line developers
for the system and/or independent assessors are involved in regular
assessments and updates. Domain experts, users, AI actors external
to the team that developed or deployed the AI system, and affected
communities are consulted in support of assessments as necessary per
organizational risk tolerance.

Measure 4.1: Measurement approaches for identifying AI risks are

connected to deployment context(s) and informed through consultation
with domain experts and other end users. Approaches are documented.

Measure 4.2: Measurement results regarding AI system trustworthiness

in deployment context(s) and across the AI lifecycle are informed
by input from domain experts and relevant AI actors to validate
whether the system is performing consistently as intended. Results are
documented.

Measure 4.3: Measurable performance improvements or declines based

on consultations with relevant AI actors, including affected communities,
and field data about context-relevant risks and trustworthiness
characteristics are identified and documented.

2 www.bsa.org
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

PROJECT CONCEPTION

Identify and
Impact Assessment

Document the intent and Map 1.1: Intended purposes, potentially beneficial uses, context-
Document purpose of the system. specific laws, norms and expectations, and prospective settings in
Objectives and which the AI system will be deployed are understood and documented.
Assumptions Considerations include: the specific set or types of users along with
their expectations; potential positive and negative impacts of system
uses to individuals, communities, organizations, society, and the planet;
assumptions and related limitations about AI system purposes, uses, and
risks across the development or product AI lifecycle; and related TEVV
and system metrics.

Map 3: AI capabilities, targeted usage, goals, and expected benefits

and costs compared with appropriate benchmarks are understood.

Clearly define the model’s Map 1.1: Intended purposes, potentially beneficial uses, context-
intended effects. specific laws, norms and expectations, and prospective settings in
which the AI system will be deployed are understood and documented.
Considerations include: the specific set or types of users along with
their expectations; potential positive and negative impacts of system
uses to individuals, communities, organizations, society, and the planet;
assumptions and related limitations about AI system purposes, uses, and
risks across the development or product AI lifecycle; and related TEVV
and system metrics.

Map 2.1: The specific tasks and methods used to implement the tasks
that the AI system will support are defined (e.g., classifiers, generative
models, recommenders).

Map 3.3: Targeted application scope is specified and documented

based on the system’s capability, established context, and AI system
categorization.

Clearly define intended use Map 1.1: Intended purposes, potentially beneficial uses, context-
cases and context in which the specific laws, norms and expectations, and prospective settings in
system will be deployed. which the AI system will be deployed are understood and documented.
Considerations include: the specific set or types of users along with
their expectations; potential positive and negative impacts of system
uses to individuals, communities, organizations, society, and the planet;
assumptions and related limitations about AI system purposes, uses, and
risks across the development or product AI lifecycle; and related TEVV
and system metrics.

Map 1.4: The business value or context of business use has been clearly
defined or—in the case of assessing existing AI systems—re-evaluated.

Select and Identify “fairness” metrics that Measure 1: Appropriate methods and metrics are identified and
Document Metrics will be used as a baseline for applied.
for Evaluating assessing bias in the AI system.
Fairness

Document Identify stakeholder groups Map 5: Impacts to individuals, groups, communities, organizations, and
Stakeholder that may be impacted by the society are characterized.
Impacts system.

www.bsa.org 3
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

PROJECT CONCEPTION (continued)

Document
Impact Assessment

For each stakeholder group, Map 1.1: Intended purposes, potentially beneficial uses, context-
Stakeholder document the potential specific laws, norms and expectations, and prospective settings in
Impacts benefits and potential adverse which the AI system will be deployed are understood and documented.
(continued) impacts, considering both the Considerations include: the specific set or types of users along with
intended uses and reasonably their expectations; potential positive and negative impacts of system
foreseeable misuses of the uses to individuals, communities, organizations, society, and the planet;
system. assumptions and related limitations about AI system purposes, uses, and
risks across the development or product AI lifecycle; and related TEVV
and system metrics.

Map 5.1: Likelihood and magnitude of each identified impact (both

potentially beneficial and harmful) based on expected use, past uses of
AI systems in similar contexts, public incident reports, feedback from
those external to the team that developed or deployed the AI system,
or other data are identified and documented.

Map 5.3: Practices and personnel for supporting regular engagement

with relevant AI actors and integrating feedback about positive,
negative, and unanticipated impacts are in place and documented.

Assess whether the nature of Measure 2: AI systems are evaluated for trustworthy characteristics.
the system makes it prone to
potential bias-related harms Measure 2.3: AI system performance or assurance criteria are measured
based on user demographics. qualitatively or quantitatively and demonstrated for conditions similar to
deployment setting(s). Measures are documented.

Measure 2.4: The functionality and behavior of the AI system and its
components—as identified in the MAP function—are monitored when in
production.

Measure 2.5: The AI system to be deployed is demonstrated to be valid

and reliable. Limitations of the generalizability beyond the conditions
under which the technology was developed are documented.

Measure 2.8: Risks associated with transparency and accountability—as

identified in the MAP function—are examined and documented.

Measure 2.9: The AI model is explained, validated, and documented,

and AI system output is interpreted within its context—as identified in
the MAP function—to inform responsible use and governance.

Measure 2.11: Fairness and bias—as identified in the MAP function—

are evaluated and results are documented.

Document Risk If risk of bias is present, Manage 1: AI risks based on assessments and other analytical output
Mitigations document efforts to mitigate from the MAP and MEASURE functions are prioritized, responded to,
risks. and managed.

Manage 1.3: Responses to the AI risks deemed high priority, as

identified by the MAP function, are developed, planned, and
documented. Risk response options can include mitigating, transferring,
avoiding, or accepting.

4 www.bsa.org
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

PROJECT CONCEPTION (continued)

Document Risk
Impact Assessment

Document how identified risks Measure 1: Appropriate methods and metrics are identified and
Mitigations and potential harms of each applied.
(continued) risk will be measured and how
the effectiveness of mitigation Measure 1.1: Approaches and metrics for measurement of AI risks
strategies will be evaluated. enumerated during the MAP function are selected for implementation
starting with the most significant AI risks. The risks or trustworthiness
characteristics that will not—or cannot—be measured are properly
documented.

Measure 1.2: Appropriateness of AI metrics and effectiveness of

existing controls are regularly assessed and updated, including reports
of errors and potential impacts on affected communities.

Measure 4: Feedback about efficacy of measurement is gathered and

assessed.

If risk of bias is present, Manage 1: AI risks based on assessments and other analytical output
document efforts to mitigate from the MAP and MEASURE functions are prioritized, responded to,
risk. and managed.

Manage 1.3: Responses to the AI risks deemed high priority, as

identified by the MAP function, are developed, planned, and
documented. Risk response options can include mitigating, transferring,
avoiding, or accepting.

If risks are unmitigated, Manage 1.4: Negative residual risks (defined as the sum of all
document why the risk was unmitigated risks) to both downstream acquirers of AI systems and end
deemed acceptable. users are documented.

Independence
Risk Mitigation Best Practices

Seek feedback from a diverse Govern 3.1: Decision-making related to mapping, measuring, and
and Diversity set of stakeholders to inform managing AI risks throughout the lifecycle is informed by a diverse team
the impact assessment. (e.g., diversity of demographics, disciplines, experience, expertise, and
backgrounds).
Govern 5.1: Organizational policies and practices are in place to collect,
consider, prioritize, and integrate feedback from those external to the
team that developed or deployed the AI system regarding the potential
individual and societal impacts related to AI risks.
Map 1.2: Interdisciplinary AI actors, competencies, skills, and capacities
for establishing context reflect demographic diversity and broad domain
and user experience expertise, and their participation is documented.
Opportunities for interdisciplinary collaboration are prioritized.
Measure 1.3: Internal experts who did not serve as front-line developers
for the system and/or independent assessors are involved in regular
assessments and updates. Domain experts, users, AI actors external
to the team that developed or deployed the AI system, and affected
communities are consulted in support of assessments as necessary per
organizational risk tolerance.
Measure 4.1: Measurement approaches for identifying AI risks are
connected to deployment context(s) and informed through consultation
with domain experts and other end users. Approaches are documented.
Measure 4.2: Measurement results regarding AI system trustworthiness
in deployment context(s) and across the AI lifecycle are informed
by input from domain experts and relevant AI actors to validate
whether the system is performing consistently as intended. Results are
documented.
Measure 4.3: Measurable performance improvements or declines based
on consultations with relevant AI actors, including affected communities,
and field data about context-relevant risks and trustworthiness
characteristics are identified and documented.

www.bsa.org 5
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

PROJECT CONCEPTION (continued)

Transparent
Risk Mitigation Best Practices

Share impact assessment Govern 4.2: Organizational teams document the risks and potential
Documentation documentation with personnel impacts of the AI technology they design, develop, deploy, evaluate,
working on later stages of the and use, and they communicate about the impacts more broadly.
AI pipeline so that risks and
potential unintended impacts Measure 1.3: Internal experts who did not serve as front-line developers
can be monitored throughout for the system and/or independent assessors are involved in regular
the development process. assessments and updates. Domain experts, users, AI actors external
to the team that developed or deployed the AI system, and affected
communities are consulted in support of assessments as necessary per
organizational risk tolerance.

Accountability Ensure that senior leadership Govern 2.3: Executive leadership of the organization takes responsibility
and Governance has been adequately briefed on for decisions about risks associated with AI system development and
potential high-risk AI systems. deployment.

DATA ACQUISITION

Maintain Govern 4.2: Organizational teams document the risks and potential
Impact Assessment

Maintain sufficient records

Records of Data to enable “recreation” of impacts of the AI technology they design, develop, deploy, evaluate,
Provenance the data used to train the AI and use, and they communicate about the impacts more broadly.
model, verify that its results
are reproducible, and monitor Manage 3.2: Pre-trained models which are used for development are
for material updates to data monitored as part of AI system regular monitoring and maintenance.
sources.

Examine Data for Scrutinize data for historical Map 2.3: Scientific integrity and TEVV considerations are identified
Potential Biases biases. and documented, including those related to experimental design, data
collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

Evaluate “representativeness” Map 2.3: Scientific integrity and TEVV considerations are identified
of the data. and documented, including those related to experimental design, data
collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

Scrutinize data labeling Map 2.3: Scientific integrity and TEVV considerations are identified
methodology. and documented, including those related to experimental design, data
collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

Document Risk Document whether and Map 2.3: Scientific integrity and TEVV considerations are identified
Mitigations how data was augmented, and documented, including those related to experimental design, data
manipulated, or re-balanced to collection and selection (e.g., availability, representativeness, suitability),
mitigate bias. system trustworthiness, and construct validation.

Independence
Risk Mitigation Best Practices

To facilitate robust interrogation Govern 3.1: Decision-making related to mapping, measuring, and
and Diversity of the datasets, data review managing AI risks throughout the lifecycle is informed by a diverse team
teams should include personnel (e.g., diversity of demographics, disciplines, experience, expertise, and
that are diverse in terms of their backgrounds).
subject matter expertise and
lived experiences. Map 1.2: Interdisciplinary AI actors, competencies, skills, and capacities
for establishing context reflect demographic diversity and broad domain
and user experience expertise, and their participation is documented.
Opportunities for interdisciplinary collaboration are prioritized.

Measure 1.3: Internal experts who did not serve as front-line developers
for the system and/or independent assessors are involved in regular
assessments and updates. Domain experts, users, AI actors external
to the team that developed or deployed the AI system, and affected
communities are consulted in support of assessments as necessary per
organizational risk tolerance.

6 www.bsa.org
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

DATA ACQUISITION (continued)

Re-Balancing Map 2.3: Scientific integrity and TEVV considerations are identified
Risk Mitigation Best Practices

Consider re-balancing with

Unrepresentative additional data. and documented, including those related to experimental design, data
Data collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

Measure 2.11: Fairness and bias—as identified in the MAP function—

are evaluated and results are documented.

Consider re-balancing with Map 2.3: Scientific integrity and TEVV considerations are identified
synthetic data. and documented, including those related to experimental design, data
collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

Measure 2.11: Fairness and bias—as identified in the MAP function—

are evaluated and results are documented.

Data Labeling Establish objective and scalable Map 2.3: Scientific integrity and TEVV considerations are identified
labeling guidelines. and documented, including those related to experimental design, data
collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

Accountability Integrate data labeling Map 2.3: Scientific integrity and TEVV considerations are identified
and Governance processes into a comprehensive and documented, including those related to experimental design, data
data strategy. collection and selection (e.g., availability, representativeness, suitability),
system trustworthiness, and construct validation.

DATA PREPARATION AND MODEL DEFINITION

Document
Impact Assessment

Document rationale for choices Govern 1: Policies, processes, procedures, and practices across the
Feature Selection made during the feature organization related to the mapping, measuring, and managing of AI
and Engineering selection and engineering risks are in place, transparent, and implemented effectively.
Processes processes and evaluate their
impact on model performance.

Document potential correlation Measure 2.11: Fairness and bias—as identified in the MAP function—
between selected features are evaluated and results are documented.
and sensitive demographic
attributes.

Document Model Document rationale for the Govern 1.4: The risk management process and its outcomes are
Selection Process selected modeling approach. established through transparent policies, procedures, and other controls
based on organizational risk priorities.

Map 2.1: The specific tasks and methods used to implement the tasks
that the AI system will support are defined (e.g., classifiers, generative
models, recommenders).

Identify, document, and justify Govern 1.4: The risk management process and its outcomes are
assumptions in the selected established through transparent policies, procedures, and other controls
approach and potential based on organizational risk priorities.
resulting limitations.
Map 2.2: Information about the AI system’s knowledge limits and how
system output may be utilized and overseen by humans is documented.
Documentation provides sufficient information to assist relevant AI
actors when making decisions and taking subsequent actions.

www.bsa.org 7
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

DATA PREPARATION AND MODEL DEFINITION (continued)

Feature Selection
Risk Mitigation Best Practices

Examine for biased proxy Measure 2.11: Fairness and bias—as identified in the MAP function—
features. are evaluated and results are documented.

Scrutinize features that correlate Measure 2.11: Fairness and bias—as identified in the MAP function—
to sensitive attributes. are evaluated and results are documented.

Independence Seek feedback from diverse Govern 3.1: Decision-making related to mapping, measuring, and
and Diversity stakeholders with domain- managing AI risks throughout the lifecycle is informed by a diverse team
specific expertise. (e.g., diversity of demographics, disciplines, experience, expertise, and
backgrounds).

Map 1.2: Interdisciplinary AI actors, competencies, skills, and capacities

for establishing context reflect demographic diversity and broad domain
and user experience expertise, and their participation is documented.
Opportunities for interdisciplinary collaboration are prioritized.

Measure 1.3: Internal experts who did not serve as front-line developers
for the system and/or independent assessors are involved in regular
assessments and updates. Domain experts, users, AI actors external
to the team that developed or deployed the AI system, and affected
communities are consulted in support of assessments as necessary per
organizational risk tolerance.

Measure 4.1: Measurement approaches for identifying AI risks are

connected to deployment context(s) and informed through consultation
with domain experts and other end users. Approaches are documented.

Measure 4.2: Measurement results regarding AI system trustworthiness

in deployment context(s) and across the AI lifecycle are informed
by input from domain experts and relevant AI actors to validate
whether the system is performing consistently as intended. Results are
documented.

Measure 4.3: Measurable performance improvements or declines based

on consultations with relevant AI actors, including affected communities,
and field data about context-relevant risks and trustworthiness
characteristics are identified and documented.

Model Selection Avoid inscrutable models in No equivalent category

circumstances where both the
risk and potential impact of bias
are high.

8 www.bsa.org
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

VALIDATING, TESTING, AND REVISING THE MODEL

Document
Impact Assessment

Document how the system (and Measure 1: Appropriate methods and metrics are identified and
Validation individual components) will be applied.
Processes validated to evaluate whether
it is performing consistent Measure 2.3: AI system performance or assurance criteria are measured
with the design objectives and qualitatively or quantitatively and demonstrated for conditions similar to
intended deployment scenarios. deployment setting(s). Measures are documented.

Measure 2.4: The functionality and behavior of the AI system and its
components—as identified in the MAP function—are monitored when in
production.

Measure 2.5: The AI system to be deployed is demonstrated to be valid

and reliable. Limitations of the generalizability beyond the conditions
under which the technology was developed are documented.

Measure 2.9: The AI model is explained, validated, and documented,

and AI system output is interpreted within its context—as identified in
the MAP function—to inform responsible use and governance.

Measure 2.13: Effectiveness of the employed TEVV metrics and

processes in the MEASURE function are evaluated and documented.

Document re-validation Manage 2.2: Mechanisms are in place and applied to sustain the value
processes of deployed AI systems.

Manage 2.3: Procedures are followed to respond to and recover from a

previously unknown risk when it is identified.

Document Testing Test the system for bias by Measure 2.11: Fairness and bias—as identified in the MAP function—
Processes evaluating and documenting are evaluated and results are documented.
model performance.

Document how testing was Measure 2.1: Test sets, metrics, and details about the tools used during
performed, which fairness TEVV are documented.
metrics were evaluated, and
why those measures were Measure 2.11: Fairness and bias—as identified in the MAP function—
selected. are evaluated and results are documented.

Document model interventions. Manage 1.3: Responses to the AI risks deemed high priority, as
identified by the MAP function, are developed, planned, and
documented. Risk response options can include mitigating, transferring,
avoiding, or accepting.

Manage 2.3: Procedures are followed to respond to and recover from a

previously unknown risk when it is identified.

Manage 2.4: Mechanisms are in place and applied, and responsibilities

are assigned and understood, to supersede, disengage, or deactivate
AI systems that demonstrate performance or outcomes inconsistent with
intended use.

www.bsa.org 9
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

VALIDATING, TESTING, AND REVISING THE MODEL (continued)

Model
Risk Mitigation Best Practices

Evaluate potential model Manage 1.3: Responses to the AI risks deemed high priority, as
Interventions refinements to address bias identified by the MAP function, are developed, planned, and
surfaced during testing. documented. Risk response options can include mitigating, transferring,
avoiding, or accepting.

Manage 2.3: Procedures are followed to respond to and recover from a

previously unknown risk when it is identified.

Manage 2.4: Mechanisms are in place and applied, and responsibilities

are assigned and understood, to supersede, disengage, or deactivate
AI systems that demonstrate performance or outcomes inconsistent with
intended use.

Independence Validation and testing Measure 1.3: Internal experts who did not serve as front-line developers
and Diversity documentation should be for the system and/or independent assessors are involved in regular
reviewed by personnel who assessments and updates. Domain experts, users, AI actors external
were not involved in the to the team that developed or deployed the AI system, and affected
system’s development. communities are consulted in support of assessments as necessary per
organizational risk tolerance.

PREPARING FOR DEPLOYMENT AND USE

Document Lines
Impact Assessment

Define and document who is Govern 1.5: Ongoing monitoring and periodic review of the risk
of Responsibility responsible for the system’s management process and its outcomes are planned and organizational
outputs and the outcomes they roles and responsibilities clearly defined, including determining the
may lead to, including details frequency of periodic review.
about how a system’s decisions
can be reviewed if necessary. Govern 2.1: Roles and responsibilities and lines of communication
related to mapping, measuring, and managing AI risks are documented
and are clear to individuals and teams throughout the organization.

Establish management plans Manage 2.3: Procedures are followed to respond to and recover from a
for responding to potential previously unknown risk when it is identified.
incidents or reports of system
errors. Manage 2.4: Mechanisms are in place and applied, and responsibilities
are assigned and understood, to supersede, disengage, or deactivate
AI systems that demonstrate performance or outcomes inconsistent with
intended use.

Manage 4: Risk treatments, including response and recovery, and

communication plans for the identified and measured AI risks are
documented and monitored regularly.

Manage 4.1: Post-deployment AI system monitoring plans are

implemented, including mechanisms for capturing and evaluating
input from users and other relevant AI actors, appeal and override,
decommissioning, incident response, recovery, and change
management.

Document Document what processes Manage 4.1: Post-deployment AI system monitoring plans are
Processes for and metrics will be used to implemented, including mechanisms for capturing and evaluating
Monitoring Data evaluate whether production input from users and other relevant AI actors, appeal and override,
data (i.e., input data the system decommissioning, incident response, recovery, and change
encounters during deployment) management.
differs materially from training
data. Measure 4.2: Measurement results regarding AI system trustworthiness
in deployment context(s) and across the AI lifecycle are informed
by input from domain experts and relevant AI actors to validate
whether the system is performing consistently as intended. Results are
documented.

10 www.bsa.org
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

PREPARING FOR DEPLOYMENT AND USE (continued)

Document
Impact Assessment

For static models, document Manage 3.2: Pre-trained models which are used for development are
Processes for how performance levels monitored as part of AI system regular monitoring and maintenance.
Monitoring Model and classes of error will be
Performance monitored over time and Manage 4.1: Post-deployment AI system monitoring plans are
benchmarks that will trigger implemented, including mechanisms for capturing and evaluating
review. input from users and other relevant AI actors, appeal and override,
decommissioning, incident response, recovery, and change
management.

For models that are intended Manage 4.1: Post-deployment AI system monitoring plans are
to evolve over time, implemented, including mechanisms for capturing and evaluating
document how changes will input from users and other relevant AI actors, appeal and override,
be inventoried; if, when, decommissioning, incident response, recovery, and change
and how versions will be management.
captured and managed; and
how performance levels will
be monitored (e.g., cadence
of scheduled reviews,
performance indicators that
may trigger out-of-cycle review).

Document Audit Document the cadence at which Govern 1.5: Ongoing monitoring and periodic review of the risk
and End-of-Life impact assessment evaluations management process and its outcomes are planned and organizational
Processes will be audited to evaluate roles and responsibilities clearly defined, including determining the
whether risk mitigation controls frequency of periodic review.
remain fit-for-purpose.

Document expected timeline Govern 1.7: Processes and procedures are in place for
that system support will be decommissioning and phasing out AI systems safely and in a
provided and processes for manner that does not increase risks or decrease the organization’s
decommissioning system trustworthiness.
in event that it falls below
reasonable performance Manage 4.1: Post-deployment AI system monitoring plans are
thresholds. implemented, including mechanisms for capturing and evaluating
input from users and other relevant AI actors, appeal and override,
decommissioning, incident response, recovery, and change
management.

Monitoring for Measure 1.2: Appropriateness of AI metrics and effectiveness of

Risk Mitigation Best Practices

Input data encountered

Drift and Model during deployment can be existing controls are regularly assessed and updated, including reports
Degradation evaluated against a statistical of errors and potential impacts on affected communities.
representation of the system’s
training data to evaluate Manage 2.2: Mechanisms are in place and applied to sustain the value
the potential for data drift of deployed AI systems.
(i.e., material differences
between the training data Manage 4.2: Measurable activities for continual improvements are
and deployment data that can integrated into AI system updates and include regular engagement with
degrade model performance). interested parties, including relevant AI actors.

Product Features Integrate product and user Manage 1.4: Negative residual risks (defined as the sum of all
and User Interface interface features to mitigate unmitigated risks) to both downstream acquirers of AI systems and
risk of foreseeable unintended end users are documented.
uses (e.g., interface that
enforces human-in-the-loop Manage 2.4: Mechanisms are in place and applied, and responsibilities
requirements, alerts to notify are assigned and understood, to supersede, disengage, or deactivate
when a system is being AI systems that demonstrate performance or outcomes inconsistent with
misused). intended use.

Manage 4.1: Post-deployment AI system monitoring plans are

implemented, including mechanisms for capturing and evaluating
input from users and other relevant AI actors, appeal and override,
decommissioning, incident response, recovery, and change
management.

www.bsa.org 11
 Crosswalk Between BSA Framework to Build Trust in AI and NIST AI Risk Management Framework

BSA FRAMEWORK NIST AI RISK MANAGEMENT FRAMEWORK

PREPARING FOR DEPLOYMENT AND USE (continued)

System
Risk Mitigation Best Practices

AI Developers should provide No equivalent category

Documentation sufficient documentation
regarding system capabilities,
specifications, limitations,
and intended uses to enable
AI Deployers to perform
independent impact assessment
concerning deployment risks.

Consider incorporating terms No equivalent category

into the End User License
Agreement that set forth
limitations designed to prevent
foreseeable misuses (e.g.,
contractual obligations to
ensure end user will comply
with acceptable use policy).

Sales and marketing materials No equivalent category

should be closely reviewed to
ensure that they are consistent
with the system’s actual
capabilities.

AI User Training AI Deployers should provide Govern 2.2: The organization’s personnel and partners receive AI
training for AI Users regarding risk management training to enable them to perform their duties
a system’s capabilities and and responsibilities consistent with related policies, procedures, and
limitations, and how outputs agreements.
should be evaluated and
integrated into a workflow.

Incident Response AI Deployers should maintain Govern 5.2: Mechanisms are established to enable the team
and Feedback a feedback mechanism to that developed or deployed AI systems to regularly incorporate
Mechanisms enable AI Users and Affected adjudicated feedback from relevant AI actors into system design and
Individuals (i.e., members of the implementation.
public who may interact with
the system) to report concerns Map 5.2: Practices and personnel for supporting regular engagement
about the operation of a with relevant AI actors and integrating feedback about positive,
system. negative, and unanticipated impacts are in place and documented.

Measure 3.3: Feedback processes for end users and impacted

communities to report problems and appeal system outcomes are
established and integrated into AI system evaluation metrics.

Measure 4: Feedback about efficacy of measurement is gathered and

assessed.

Manage 4: Risk treatments, including response and recovery, and

communication plans for the identified and measured AI risks are
documented and monitored regularly.

12 www.bsa.org