Responsible AI Procurement Approaches

This new white paper "Introduction to AI assurance" by the UK Department for Science, Innovation, and Technology from Feb 12, 2024, provides an EXCELLENT overview of assurance methods and international technical standards that can be utilized to create and implement ethical AI systems. The new guidance is based on the UK AI governance framework, laid out in the 2023 white paper "A pro-innovation approach to AI regulation". This white paper defined 5 universal principles applicable across various sectors to guide and shape the responsible development and utilization of AI technologies throughout the economy: - Safety, Security, and Robustness - Appropriate Transparency and Explainability - Fairness - Accountability and Governance - Contestability and Redress The 2023 white paper also introduced a suite of tools designed to aid organizations in understanding "how" these outcomes can be achieved in practice, emphasizing tools for trustworthy AI, including assurance mechanisms and global technical standards. See: https://lnkd.in/gydvi9Tt The new publication, "Introduction to AI assurance," is a deep dive into these assurance mechanisms and standards. AI assurance encompasses a spectrum of techniques for evaluating AI systems throughout their lifecycle. These range from qualitative assessments for evaluating potential risks and societal impacts to quantitative assessments for measuring performance and legal compliance. Key techniques include: - Risk Assessment: Identifies potential risks like bias, privacy, misuse of technology, and reputational damage. - Impact Assessment: Anticipates broader effects on the environment, human rights, and data protection. - Bias Audit: Examines data and outcomes for unfair biases. - Compliance Audit: Reviews adherence to policies, regulations, and legal requirements. - Conformity Assessment: Verifies if a system meets required standards, often through performance testing. - Formal Verification: Uses mathematical methods to confirm if a system satisfies specific criteria. The white paper also explains how organizations in the UK can ensure their AI systems are responsibly governed, risk-assessed, and compliant with regulations: 1.) For demonstrating good internal governance processes around AI, a conformity assessment against standards like ISO/IEC 42001 (AI Management System) is recommended. 2.) To understand the potential risks of AI systems being acquired, an algorithmic impact assessment by a accredited conformity assessment body is advised. This involves (self) assessment against a proprietary framework or responsible AI toolkit. 3.) Ensuring AI systems adhere to existing data protection regulations involves a compliance audit by a third-party assurance provider. This white paper also has exceptional infographics! Pls, check it out, and TY Victoria Beckman for posting and providing us with great updates as always!

"On Nov 6, the UK Department for Science, Innovation and Technology (DSIT) published a first draft version of its AI Management Essentials (AIME) self-assessment tool to support organizations in implementing responsible AI management practices. The consultation for AIME is open until Jan 29, 2025. Recognizing the challenge many businesses face in navigating the complex landscape of AI standards, DSIT created AIME to distill essential principles from key international frameworks, including ISO/IEC 42001, the NIST Risk Management Framework, and the EU AI Act. AIME provides a framework to: - Evaluate current practices by identifying areas that meet baseline expectations and pinpointing gaps. - Prioritize improvements by highlighting actions needed to align with widely accepted standards and principles. - Understand maturity levels by offering insights into how an organization's AI management systems compare to best practices. AIME's structure includes: - A self-assessment questionnaire - Sectional ratings to evaluate AI management health - Action points and improvement recommendations The tool is voluntary and doesn’t lead to certification. Rather, it builds a baseline for 3 areas of responsible AI governance - internal processes, risk management, and communication. It is intended for individuals familiar with organizational governance, such as CTOs or AI Ethics Officers. Example questions: 1) Internal Processes Do you maintain a complete record of all AI systems used and developed by your organization? Does your AI policy identify clear roles and responsibilities for AI management? 2) Fairness Do you have definitions of fairness for AI systems that impact individuals? Do you have mechanisms for detecting unfair outcomes? 3) Impact Assessment Do you have an impact assessment process to evaluate the effects of AI systems on individual rights, society and the environment? Do you communicate the potential impacts of your AI systems to users or customers? 4) Risk Management Do you conduct risk assessments for all AI systems used? Do you monitor your AI systems for errors and failures? Do you use risk assessment results to prioritize risk treatment actions? 5) Data Management Do you document the provenance and collection processes of data used for AI development? 6) Bias Mitigation Do you take steps to mitigate foreseeable harmful biases in AI training data? 7) Data Protection Do you implement security measures to protect data used or generated by AI systems? Do you routinely complete Data Protection Impact Assessments (DPIAs)? 8) Communication Do you have reporting mechanisms for employees and users to report AI system issues? Do you provide technical documentation to relevant stakeholders? This is a great initiative to consolidating responsible AI practices, and offering organizations a practical, globally interoperable tool to manage AI!" Very practical! Thanks to Katharina Koerner for summary, and for sharing!

The UK Department for Science, Innovation and Technology published the guide "Introduction to AI assurance," to provide an overview of assurance mechanisms and global technical standards for industry and #regulators to build and deploy responsible #AISystems. #Artificialintelligence assurance processes can help to build confidence  in #AI systems by measuring and evaluating reliable, standardized, and accessible evidence about their capabilities. It measures whether such systems will work as intended, hold limitations, or pose potential risks; as well as how those #risks are being mitigated to ensure that ethical considerations are built-in throughout the AI development #lifecycle. The guide outlines different AI assurance mechanisms, including: - Risk assessments - Algorithmic impact assessment - Bias and compliance audits - Conformity assessment - Formal verification It also provides some recommendations for organizations interested in developing their understanding of AI assurance: 1. Consider existing regulations relevant for AI systems (#privacylaws, employment laws, etc) 2. Develop necessary internal skills to understand AI assurance and anticipate future requirements. 3. Review internal governance and #riskmanagement practices and ensure effective decision-making at appropriate levels.  4. Keep abreast of sector-specific guidance on how to operationalize and implement proposed principles in each regulatory domain.  5. Consider engaging with global standards development organizations to ensure the development of robust and universally accepted standard protocols. https://lnkd.in/eiwRZRXz