IAPP AIGP, study guide domain iii

Domain iii: how to govern AI development

Which practice best ensures accountability in an AI project? = Defining and documenting clear roles and responsibilities for all AI stakeholders.

Domain III.a govern the designing and building of an AI model

  Plan → design → develop → readiness assessment

•  The AI system development life cycle

  •  1a) Plan

  •  design

  •  1b) Data collection and preprocessing

  •  1c) Build

  •  2) Verification and validation

  •  3) Deployment

  •  4) Operation and monitoring

•  Identify all laws that apply

•  Apply the policies, procedures, best practices, and ethical considerations to designing and building (below)

  
  

 1 Key steps in mapping, planning, scoping

• Defining the business context and use case of the AI system ,

•  perform cost-benefit analysis and trade-offs.

•  Why use AL? Objectives?

•  Define assumptions

•  Scope of AI project

•  Assess feasibility and plan accordingly

  •  Technical feasibility

  •  Data readiness

  •  Organizational readiness

  •  Risk management

•  Identify and classify internal/external risks

•  Construct a probability/severity harms matrix and a risk mitigation hierarchy.

•  Algorithmic impact assessment, using Privacy Impact Assessments as a starting point.

  •  Impact assessments should be planned to happen at different stages throughout the AI system’s lifecycle. The lifecycle includes the design, development, deployment and post-deployment phases so that risks can be identified and addressed immediately.

• create the governance structure

  •  determine whether an AI governance structure is already in place; then, if not, or if a new structure is needed, decide who the executive champion is and determine who will maintain the structure.

  •    Establish level of human oversight and involvement

•  Determine stakeholders, responsibilities, roles == most important for good governance!!

  •   Evaluate stakeholder salience.

  •  Include diversity of demographics, disciplines, experience, expertise and backgrounds.

  •   Perform positionality exercise.

  •   Determine level of engagement.

  •  Establish engagement methods.

  •   Identify AI actors during design, development, and deployment phases.

  •   Create communication plans for regulators and consumers that reflect compliance/disclosure obligations for transparency and explainability (UI copy, FAQs, online documentation, model or system cards).

•  Not yet model or architecture selection, so no feature

  
  

 2 key steps in the AI system design phase

• Implement a data strategy / data management that includes:

  •  Data collection,

  •  Data wrangling, cleansing, labeling.

  •  Apply Privacy-enhancing technologies (PETs) = are a collection of digital technologies and approaches that allow for the collection, processing, analysis, and sharing of information while safeguarding the privacy of personal data. They enable a relatively high level of utility from data while minimizing the need for extensive data collection and processing.

    •  Embody fundamental data protection principles by minimizing personal data use, maximizing information security, and/or empowering individuals.

    •  Anonymization = removing or permanently altering personal data elements to prevent the identification of individuals.

    •  Differential privacy = adds noise to datasets, so its impossible ot identify individuals from aggregated data

    •  Homomorphic encryption = computation on encrypted data. Reduces speed and scalability

    •  Federated learning = a type of machine learning that enables the training of machine learning models across multiple decentralized devices without transferring the raw data to a central server.

      •  Each device trains the model using its local data, and only aggregated updates get shared with the central server. This helps preserve data privacy as the raw data remains on the devices and is never shared.

    •  AI-generated synthetic data

      •  This approach uses advanced AI algorithms to generate synthetic data with the same statistical properties and correlations as real data but without including personally identifiable information

      •  Synthetic data is specifically used to increase the accuracy of models.

      •  Using agent-based modeling, this process generates possible new threats that mimic real data.

• select AI system architecture and model selection

  •   = choose the algorithm according to the desired level of accuracy and interpretability).

  •  Key stakeholder = AI dev or data scientists

•  Reducing bias in planning & design phase

  •  through stakeholder involvement, human oversight, and careful data collection.

    
    

 3 key steps in the AI system development phase

• Build the model

  • Perform feature selection and engineering 

    •  Feature engineering = extracting attributes and variables from raw data, or transforming raw data into something meanfinful and suitable for training and AI model (not data cleaning)

    •  Same feature must be used in training & testing

    •  Personal data can be used as features, but ideally not

    •  Raw data needs to be processed

    •  Too many features → over-fitting

  • Perform model training.

  • Perform model testing and validation. Best practices:

    •  Test data shoudl be representative of expected operational data

    •  Test data should be anonymized to the extent possible

    •  Testing shoudl be performed specific to the intended uses

•  How to identify and manage internal and external risks

  •  probability/severity harms matrix

  •  Risk mitigation hierarchy

  •  Stakeholder mapping

  •  Use case evaluation

  •  Benchmarking

  •  Pre-deployment pilots

  •  Testing

• Risk assessments ==

  •  Purpose == find  the areas of greatest risk in an AI system so risk mitigation resources can be allocated appropriately.

  •  Risk assessments can also be used to generate metrics for tracking and managing priorities throughout the model development life cycle.

•  In algorithm design, a greedy algorithm always takes the locally optimal choice at each step to quickly reach a solution. It does not consider the overall long-term outcome. This can work well for some problems (if it happens that local best choices lead to a global best solution), but not always.

  
  

Domain III.b govern data collection and use in training/testing

***  One of the most significant challenges when designing and developing  AI systems is ensuring the data used is appropriate for the intended purpose.

Managing unnecessary  data, especially data that may contain sensitive attributes, can increase risk.

•  Establish and follow the requirements for data governance, e.g.:

 Step 1 assess and document lawful rights to collect and use data,

•  then assess data quality (representativeness, accuracy, bias), quantity, integrity and fit-for-purpose.

•  Identify data gaps

 2. Establish and document data lineage and provenance.

•  Chart Data Lineage: Thoroughly trace the origin, movement, and transformation of data throughout the AI system. This ensures transparency and facilitates the identification of potential issues

• Document Data Provenance: Maintain detailed records of data sources, custody, and history to validate its authenticity and quality

 3. Plan and perform training and testing of the AI model (e.g., unit, integration, validation, performance, security, bias and interpretability).

•  Aka pre-deployment testing

  •  To ensure the model behaves as expected and aligns with organizational requirements

  •  Testing for unacceptable outcomes such as toxicity, discrimination, or hallucinations

 4. Identify and manage the issues and risks during training and testing of an the AI model.

 5. Document the training and testing process

• (e.g., to validate results, establish compliance and manage risks).

•  In the design phase of integrating data from different sources, identifying fits and gaps is crucial.

  •  This process involves understanding how well the data from the clinical research partner aligns with the healthcare network's existing data.

  •  ensures that the combined data set is coherent and can be effectively used for training the AI algorithm. This step helps in spotting any discrepancies, inconsistencies, or missing data that might affect the performance and accuracy of the AI model.

  •  addresses the integrity and compatibility of the data, which is foundational before applying any PETs, labeling, or evaluating the origin of the data.

 Techniques for data management and integrity

• Model disgorgement

  •  The technique used to remove the effects of improperly used data from an ML system. This process involves retraining or adjusting the model to eliminate any biases or inaccuracies introduced by the inappropriate data. It ensures that the model's outputs are not influenced by data that was not meant to be used or was used incorrectly.

•  Differential privacy

  •  is a technique used to ensure that the inclusion or exclusion of a single data point does not significantly affect the outcome of any analysis, providing a way to mathematically prove that no specific piece of training data has more than a negligible effect on the model or its output.

  •  achieved by introducing randomness into the data or the algorithms processing the data.

  •  In LLM training context: differential privacy helps in protecting individual data points while still enabling the model to learn effectively. By adding noise to the training process, differential privacy provides strong guarantees about the privacy of the training data.

•  know the technical aspects of data governance.

  •  How is data stored?

    •  Data lake = centralized storage system or RAW data. Scalable, cost-efficient. All data types, stored without predefined schema.

    •  Database = already structured collection of data (SQL, oracle) . optimized for transactional processing and real-time options.

  •  How to ensure data quality?

    •  Set, comm defined quality standards & practices = detailed standards

    •  Data stewardship, accountability

    •  Data profiling and cleansing

    •  Monitoring and reporting

    • Continuous improvement

    • 
      

•  Ensuring fairness when training an AI system == most depends on the data attributes and variability.

  •  Dataset = diverse, representative of the relevant population

  •  Fairness can be compromised if the data is biased or lacks variability, as the model may learn and perpetuate these biases.

 Diverse data attributes ensure that the model learns from a wide range of examples, reducing the risk of biased predictions.

Domain III.c govern release, monitoring, and maintenance

 1. Assess readiness and prepare for release into production (e.g., creating the model card and  satisfying conformity requirements).

• After completing model testing and validation, the most important step prior to deploying the model into production is to perform a readiness assessment.

•  This assessment ensures that the model is fully prepared for deployment, addressing any potential issues related to infrastructure, performance, security, and compliance. It verifies that the model meets all necessary criteria for a successful launch.

•   Other steps, such as defining a model-validation methodology, documenting maintenance teams and processes, and identifying known edge cases, are also important but come secondary to confirming overall readiness.

•   Model cards are short summary documents explaining the purpose of an AI model and provide information regarding a model’s development and performance, as well as additional details for model transparency purposes.

  •  = best practice method to identify and mitigate predictable risks associated with any secondary/unintended AI model use.

•  Readiness assessments in implementation

  •  Technical readiness

    •  Existing infrastructure (hardware, software, network) - meets requirements of AI system?

    •  Upgrades, modifications needed?

  •  Compliance

    •  Relevant regulations

    •  Ethical concerns + mitigations

  •  Stakeholder readiness

    •  Do users and those impacted understand?

    •  Give training, address concerns

 2. Conduct continuous monitoring of the AI model and establish a regular schedule for maintenance, updates and retraining.

•  During the first month of monitoring the model for bias, it is most important to continue disparity testing.

•  Disparity testing involves regularly evaluating the model's decisions, to identify and address any biases, ensuring that the model operates fairly across different demographic groups.

•  Regular disparity testing = critical practice for maintaining the fairness and reliability of AI models.

•  By continuously monitoring for and addressing disparities, organizations can ensure their AI systems remain compliant with ethical and legal standards, and mitigate any unintended biases that may arise in production.

 3. Conduct periodic activities to assess the AI model’s performance, reliability and safety (e.g., audits, red teaming, threat modeling and security testing).

•  Continue testing model accuracy in production environments. == test model inference

 4. Manage and document incidents, issues and risks.

 5. Collaborate with cross-functional stakeholders to understand why incidents arise from AI models

•  brittleness,

•  lack of robustness,

•  lack of quality data,

•  insufficient testing,

•  model drift

•  data drift

6. Make public disclosures to meet transparency obligations (e.g., technical documentation, instructions for use to deployers, and post-market monitoring plans).

Maintenance  = post-deployment

•  Operational aspects (more than defining new requirements or thresholds)

•  Updating proactively - on fresh data. To improve accuracy, prevent performance degradation, and adapt to changing patterns

•  Adaptations: Fine-tune the model's parameters, configurations, or even its architecture as needed to optimize performance and address shifts in operational requirements.

• Compliance and Ethics: Continuously monitor adherence to regulatory and ethical standards.

•  Implement updates or mitigations promptly if new risks or biases are identified.

•  Ensuring that all model components are subject a control framework.

•  Dedicating experts to continually monitor the model output.

•  C.Evaluating the need for an audit under certain standards.