Best Practices for AI Vendor Governance and Oversight
Expert-defined terms from the Professional Certificate in Artificial Intelligence Vendor Due Diligence Framework course at Greenwich School of Business and Finance. Free to read, free to share, paired with a globally recognised certification pathway.
**Accountability** #
The responsibility for the outcomes of AI systems, including any negative consequences. It is important to establish clear lines of accountability for AI systems to ensure that organizations can be held responsible for any harm caused by their AI systems.
**Algorithmic bias** #
Systematic prejudice or unfairness in the outcomes produced by AI algorithms. This can occur when the data used to train the algorithm is biased or when the algorithm itself is designed in a way that leads to biased outcomes.
**Artificial Intelligence (AI)** #
The simulation of human intelligence in machines that are programmed to think and learn like humans. AI can be divided into two main categories: narrow or weak AI, which is designed to perform a specific task, and general or strong AI, which is capable of performing any intellectual task that a human can.
**AI vendor governance** #
The process of managing and overseeing the use of AI systems by third-party vendors. This includes establishing clear guidelines for the use of AI, monitoring vendor compliance with those guidelines, and taking corrective action when necessary.
**AI vendor oversight** #
The process of supervising and monitoring the development, deployment, and use of AI systems by third-party vendors. This includes conducting regular audits and reviews of vendor AI systems, as well as providing guidance and support to vendors to ensure that their AI systems are aligned with organizational goals and values.
**Black box algorithms** #
AI algorithms that are difficult to understand or interpret, making it challenging to determine how they are making decisions. These algorithms can be problematic because it is difficult to identify and address any bias or unfairness in their outcomes.
**Bias** #
Prejudice or unfairness in the outcomes produced by AI systems. Bias can occur at various stages of the AI development and deployment process, including data collection, algorithm design, and model training.
**Challenges in AI vendor governance and oversight** #
There are several challenges that organizations may face when governing and overseeing the use of AI systems by third-party vendors. These include the lack of transparency and interpretability of AI algorithms, the potential for bias and unfairness in AI outcomes, and the need to balance the benefits of AI with the potential risks and harms.
**Clear guidelines** #
Detailed and specific rules or instructions that provide clear direction for the use of AI systems by third-party vendors. Clear guidelines can help to ensure that vendors are using AI in a responsible and ethical manner, and can help to prevent bias and unfairness in AI outcomes.
**Compliance monitoring** #
The process of regularly checking and verifying that third-party vendors are following the guidelines and rules established for the use of AI systems. Compliance monitoring can help to ensure that vendors are using AI in a responsible and ethical manner, and can help to prevent bias and unfairness in AI outcomes.
**Data bias** #
Systematic prejudice or unfairness in the data used to train AI algorithms. This can occur when the data is not representative of the population it is intended to serve, or when the data is collected or processed in a biased manner.
**Data governance** #
The process of managing and overseeing the use of data within an organization. Data governance includes establishing clear policies and procedures for data collection, storage, and use, as well as monitoring compliance with those policies and procedures.
**Ethical AI** #
AI systems that are designed and used in a way that is consistent with ethical principles and values. Ethical AI is important because it helps to ensure that AI is used in a responsible and trustworthy manner, and that it does not cause harm to individuals or society.
**Explainability** #
The ability to understand and interpret the decisions made by AI systems. Explainability is important because it allows humans to understand how AI systems are making decisions, and to identify and address any bias or unfairness in those decisions.
**Fairness** #
The absence of bias or unfairness in the outcomes produced by AI systems. Fairness is an important consideration in the design and use of AI because it helps to ensure that AI systems do not discriminate against certain groups of people or perpetuate existing social inequities.
**GDPR** #
The General Data Protection Regulation is a regulation in EU law on data protection and privacy in the European Union and the European Economic Area. It also addresses the transfer of personal data outside the EU and EEA areas.
**Guidance** #
Advice or recommendations provided to third-party vendors on the use of AI systems. Guidance can help to ensure that vendors are using AI in a responsible and ethical manner, and can help to prevent bias and unfairness in AI outcomes.
**Interpretability** #
The ability to understand and make sense of the internal workings of AI algorithms. Interpretability is important because it allows humans to understand how AI systems are making decisions, and to identify and address any bias or unfairness in those decisions.
**Model training** #
The process of using data to teach an AI algorithm how to make decisions or predictions. Model training is an important step in the development of AI systems, as it determines the accuracy and fairness of the algorithm's outcomes.
**Monitoring** #
The process of regularly checking and verifying that third-party vendors are following the guidelines and rules established for the use of AI systems. Monitoring can help to ensure that vendors are using AI in a responsible and ethical manner, and can help to prevent bias and unfairness in AI outcomes.
**Negative consequences** #
Harm or damage caused by the use of AI systems. Negative consequences can include physical harm, psychological harm, financial harm, or reputational harm.
**Narrow AI** #
AI systems that are designed to perform a specific task. Narrow AI is also known as weak AI.
**Oversight** #
Supervision and monitoring of the development, deployment, and use of AI systems by third-party vendors. Oversight includes conducting regular audits and reviews of vendor AI systems, as well as providing guidance and support to vendors to ensure that their AI systems are aligned with organizational goals and values.
**Responsible AI** #
AI systems that are designed and used in a way that is consistent with ethical principles and values, and that do not cause harm to individuals or society. Responsible AI is important because it helps to ensure that AI is used in a trustworthy and trustworthy manner.
**Risks and harms** #
Potential negative consequences or adverse effects that may result from the use of AI systems. Risks and harms can include physical harm, psychological harm, financial harm, or reputational harm.
**Supervision** #
The process of overseeing and monitoring the development, deployment, and use of AI systems by third-party vendors. Supervision includes conducting regular audits and reviews of vendor AI systems, as well as providing guidance and support to vendors to ensure that their AI systems are aligned with organizational goals and values.
**Third #
party vendors**: Companies or organizations that provide AI systems or services to other organizations. Third-party vendors are an important part of the AI ecosystem, as they often have the expertise and resources needed to develop and deploy advanced AI systems.
**Transparency** #
The ability to understand and see into the workings of AI algorithms. Transparency is important because it allows humans to understand how AI systems are making decisions, and to identify and address any bias or unfairness in those decisions.
**Unfairness** #
Prejudice or discrimination in the outcomes produced by AI systems. Unfairness can occur when the data used to train the algorithm is biased or when the algorithm itself is designed in a way that leads to biased outcomes.
**Vendor governance** #
The process of managing and overseeing the use of AI systems by third-party vendors. This includes establishing clear guidelines for the use of AI, monitoring vendor compliance with those guidelines, and taking corrective action when necessary.
**Vendor oversight** #
The process of supervising and monitoring the development, deployment, and use of AI systems by third-party vendors. This includes conducting regular audits and reviews of vendor AI systems, as well as providing guidance and support to vendors to ensure that their AI systems are aligned with organizational goals and values.
**Well #
structured and learner-friendly content**: Content that is organized and presented in a clear and logical manner, and that is easy for learners to understand and follow. Well-structured and learner-friendly content includes examples, practical applications, and challenges to help learners apply the concepts they have learned.