Glossary

Published

May 26, 2026

This glossary of terms provides short definitions for technical terms you find commonly used in our product documentation grouped by terms related to:

​ValidMind

ValidMind AI Risk Platform

These two features are intertwined and work in tandem to help streamline your risk management lifecycles.

ValidMind Library (library)
An open-source1 suite of documentation tools and test suites designed to document records (such as models), test records for weaknesses, and identify overfit areas. Enables automating the generation of documentation by uploading documentation and test results to the ValidMind Platform.

1 ​ValidMind GitHub: validmind-library

ValidMind Platform (platform)
A hosted multi-tenant architecture2 that includes the ​ValidMind cloud-based web interface, APIs, databases, documentation and validation engine, and various internal services.

​ValidMind core features

client library, Python client library
Enables the interaction of your development environment with the ValidMind Platform as part of the ValidMind Library.
content block
A modular document template component. Content blocks are used to populate text and test results in documentation, validation reports, ongoing monitoring reports, and custom document types.3
documentation automation
A core benefit of ​ValidMind that allows for the automatic creation of documentation using predefined templates and test suites.4
inventory
A feature of the ValidMind Platform where you can track, manage, and oversee the lifecycle of your records (such as models). Covers the full record lifecycle, including customizable approval workflows for different user roles, status and activity tracking, and periodic revalidation.
record
A tool tracked in the ValidMind Platform inventory,5 such as a model. Records include traditional statistical models, legacy systems, artificial intelligence/machine learning models, large language models (LLMs), agentic AI systems, and other documentable items that benefit from oversight, testing, and lifecycle management.

5 Refer to: model

document template
Lays out the structure of documents, segmented into various sections and sub-sections, and functions as a test suite to help automate your development, validation, monitoring, and other risk management processes. Document templates are available for default ​ValidMind document types6 as well as custom document types.

6 Default ​ValidMind document type templates:

7 Refer also to: documentation

documentation template7
A default ​ValidMind document template that serves as a standardized framework for developing and documenting records (such as models), including sections designated for record details, data descriptions, test results, and performance metrics. By outlining required documentation and recommended analyses, documentation templates ensure consistency and completeness across documentation and help guide developers through a systematic development process while promoting comparability and traceability of development outcomes.

​ValidMind documentation templates function as test suites by defining the structure of your documentation, specifying the tests that should be run, and how the results should be displayed.

validation report template8

8 Refer also to: validation reports

A default ​ValidMind document template that serves as a standardized framework for conducting and documenting validation, including sections designated for attaching test results, evidence, or artifacts (findings). By outlining required documentation, recommended analyses, and expected validation tests, validation report templates ensure consistency and completeness across validation reports and help guide validators through a systematic review process while promoting comparability and traceability of validation outcomes.
monitoring template, monitoring report template9

9 Refer also to: ongoing monitoring

A default ​ValidMind document template that serves as a standardized framework for ongoing monitoring, including sections designated for test results, performance metrics, and drift analyses. By outlining required monitoring checks and expected routine tests, monitoring templates ensure consistency and completeness across monitoring reports and help guide owners through a systematic monitoring process while promoting early detection of performance degradation.
test
A function contained in the library, designed to run a specific quantitative test on the dataset or record (such as a model). Test results are logged to the ValidMind Platform, where they are attached to documents.

Tests are the building blocks of ​ValidMind, used to evaluate and document records and datasets, and can be run individually or as part of a suite defined by your templates.

test suite
A collection of tests designed to run together to automate and generate documentation end-to-end for specific use cases.10

10 Learn more: test_suites

Artificial intelligence (AI) governance

AI

Refer to IBM’s series on artificial intelligence for more in-depth resources.

AI governance
The organizational framework for directing and overseeing how AI is designed, deployed, and used. It sets policy, accountability, and decision rights, covering ethics, compliance, risk appetite, lifecycle controls, and ongoing oversight across people, process, and technology.11

11 Refer also to: AI governance

AI system
A combination of software, algorithms, and data designed to perform tasks that typically require human intelligence. In AI governance, an AI system is the primary unit of management, distinct from individual records (such as models).12

12 Refer to:

AI use case
A specific application or deployment of AI technology to solve a business problem or achieve an objective. Use cases are often the unit of oversight in AI governance frameworks.
artificial intelligence (AI)
Artificial intelligence is a broad term used to classify machines that mimic human intelligence and human cognitive functions like problem-solving and learning.
deep-learning
A subset of machine learning that uses multi-layered neural networks (deep neural networks) to simulate the complex decision-making power of the human brain.
generative AI (GenAI)
Generative AI refers to deep-learning models that can generate high-quality text, images, and other content based on the data they were trained on.
human oversight
Controls and processes ensuring human involvement in AI-driven decisions. Required by regulations like the EU AI Act for high-risk AI systems to enable human intervention and override capabilities.13

13 Refer also to: EU AI Act

impact assessment
An evaluation of the potential risks, harms, and consequences associated with deploying an AI system. Impact assessments are a core artifact in AI governance programs.
large language model (LLM)
An advanced type of artificial intelligence model designed to understand, generate, and interact with human language at a sophisticated level, such as ChatGPT.14
machine learning (ML)
Machine learning is a subset of artificial intelligence that allows for optimization. It helps make predictions that minimize the errors that arise from merely guessing.
risk tier
A classification level assigned to an AI system based on its potential impact and risk. The EU AI Act defines tiers including prohibited, high-risk, limited-risk, and minimal-risk categories.15

15 Refer also to: EU AI Act

traditional statistical model
A mathematical framework used to analyze and make inferences from data. Traditional statistical models are foundational in statistics and serve to explain relationships, predict outcomes, and guide decision-making across various fields, such as economics, biology, engineering, and social sciences.
use case owner
The individual accountable for an AI use case within an organization. Responsible for decisions about AI deployment, compliance, and ongoing oversight.

AI governance

AI ethics
A set of principles and practices guiding the responsible design, development, and deployment of AI systems. Common tenets include fairness, transparency, accountability, privacy, and human well-being.
AI lifecycle
The end-to-end stages an AI system progresses through, including problem framing, data collection, model development, validation, deployment, monitoring, and retirement. Each stage carries distinct governance requirements.
AI risk
The potential for adverse outcomes — financial, reputational, ethical, regulatory, or societal — arising from the design, deployment, or use of AI systems. AI risk extends beyond traditional model risk to include concerns such as bias, opacity, misuse, and unintended consequences.
algorithmic accountability
The principle that organizations must take responsibility for the outcomes of the AI systems they deploy, including documenting decisions, monitoring performance, and providing mechanisms to identify and remediate harm.
bias, algorithmic bias
Systematic errors or unfair outcomes in AI system results that disproportionately affect specific groups. Sources include unrepresentative training data, flawed assumptions in system design, or feedback loops introduced during deployment. Detecting and mitigating bias is a core AI governance activity.
EU AI Act
A regulatory framework introduced by the European Union that classifies AI systems by risk tier16 — prohibited, high-risk, limited-risk, and minimal-risk — and imposes proportionate obligations such as risk management, data governance, transparency, human oversight, and conformity assessment.
explainability
The degree to which the internal mechanics or outputs of an AI system can be understood by humans. Explainability is a core requirement for high-risk AI systems and supports accountability, debugging, and regulatory review.
fairness
The principle that AI systems should produce equitable outcomes across individuals and groups. Fairness assessments are a routine part of bias evaluation and impact assessment within AI governance programs.
ISO/IEC 42001
An international management system standard for artificial intelligence published by the International Organization for Standardization. Provides requirements for establishing, implementing, maintaining, and continually improving an AI management system within an organization.
model card, system card
A standardized document that summarizes an AI system’s intended use, training data, performance characteristics, limitations, and ethical considerations. Model and system cards support transparency and informed deployment decisions.17

17 Refer also to: documentation

NIST AI Risk Management Framework (AI RMF)
A voluntary framework published by the U.S. National Institute of Standards and Technology to help organizations manage risks associated with AI. Organized around four core functions: govern, map, measure, and manage.
responsible AI
An umbrella approach to designing, building, and deploying AI systems in ways that are ethical, transparent, accountable, fair, and aligned with human values and societal expectations.
transparency
The disclosure of meaningful information about an AI system’s design, data, capabilities, limitations, and decision-making processes to relevant stakeholders. Transparency supports trust, accountability, and informed oversight.

Model risk management

Models

documentation, model documentation
A structured and detailed record pertaining to a record (such as a model), encompassing key components such as its underlying assumptions, methodologies, data sources, inputs, performance metrics, evaluations, limitations, and intended uses.

Within the realm of risk management, this documentation serves to ensure transparency, adherence to regulatory requirements, and a clear understanding of potential risks associated with the record’s application.

model
SR 26-218 (which supersedes SR 11-7) defines a model as a “complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates.” Simple arithmetic, deterministic rule-based processes, or software without statistical, economic, or financial theories underpinning their design or use are generally outside SR 26-2’s definition of a model.

Within ​ValidMind, a model is a type of record tracked in the inventory.19

19 Refer also to: record

model development
An iterative process in which many models are derived, tested, and built upon until a model fitting the desired criteria is achieved.
model inventory20

20 Refer also to: inventory

A systematic and organized record of all quantitative and qualitative models used within an organization. This inventory facilitates oversight, tracking, and assessment by listing each model’s purpose, characteristics, owners, validation status, and associated risks.
model lifecycle
Subset of stages defining the lifecycle of a model; encompasses all steps for operating, governing, and maintaining a model until it is decommissioned (model development, model validation, model approval, model implementation, retirement).
model risk
The potential for financial loss, incorrect decisions, or unintended consequences resulting from errors or inaccuracies in AI or machine learning models. Model risk typically arises from incorrect or inappropriate use of models, inaccurate assumptions, or limitations in data quality.

For example, consequences of unmitigated model risk can include adverse outcomes such as financial loss, damage to reputation, and regulatory penalties.

model risk management (MRM)
A structured approach to identifying, assessing, mitigating, and monitoring risks arising from the use of quantitative and qualitative models within an organization. Ensures that models are developed, validated, and used appropriately, with robust controls in place. Encompasses practices such as maintaining a model inventory, conducting periodic validations, and ensuring proper documentation.
model vendor, record vendor
A company that develops, documents, and sells models or records to financial institutions.
vendor model, vendor record
A model or record created by an external source, such as a vendor.

Model risk management

1st line of defense
Business unit(s) responsible for model development, initial validation, and implementation during the model lifecycle. As the 1st line of defense, model developers must document and test models to ensure that they are accurate, robust, and fit for purpose.
2nd line of defense
An independent oversight function that provides a governance framework for the model lifecycle. As the 2nd line of defense, model validators must independently validate and challenge models created by model developers to ensure that model risk management principles are followed.
3rd line of defense
Typically an internal audit function responsible for providing an independent and comprehensive review of the risk management processes and controls that the first two lines have implemented.
model developer, developer
Responsible for the design, implementation, and maintenance of models to ensure they are fit-for-purpose, accurate, and aligned with business requirements. As subject matter experts, they collaborate with model validators and other business units, ensuring the models are conceptually sound and robust.
model governance, governance
A framework of policies, procedures, and standards established to oversee the lifecycle of models within an organization. Ensures that models are developed, validated, implemented, and retired in a controlled and consistent manner, promoting accountability, transparency, and adherence to regulatory requirements.
model implementation, implementation
A collaborative effort among model developers and model owners. Model implementation includes a formalized implementation plan and associated procedures, a review of results, and a record of model change procedures.
model owner, owner
Responsible for coordinating model development, model implementation, ongoing model monitoring and maintaining the model’s administration, such as model documentation and model risk reporting.
model user, user
Those who rely on the model’s outputs to inform business decisions.
model validation, validation
A systematic process to evaluate and verify that a model is performing as intended, accurately represents the phenomena it is designed to capture, and is appropriate for its specified purpose. This assessment encompasses a review of the model’s conceptual soundness, data integrity, calibration, and performance outcomes, as well as testing against out-of-sample datasets.

Within model risk management, model validation ensures that potential risks associated with model errors, misuse, or misunderstanding are identified and mitigated.

model validator, validator
Responsible for conducting independent assessments of models to ensure their accuracy, reliability, and appropriateness for intended purposes. The role involves evaluating a model’s conceptual soundness, data integrity, calibration methods, and overall performance, typically using out-of-sample datasets.

Model validators identify potential risks and weaknesses, ensuring that models within an organization meet established standards and regulatory requirements, and provide recommendations to model developers for improvements or modifications.

three lines of defense
A structured approach to model risk management, consisting of three independent functions:
  • The first line consists of business units responsible for model development, validation, and implementation. They ensure that models are accurate, robust, and fit for purpose.
  • The second line is an independent model risk oversight function that provides a governance framework and guidance for model risk management.
  • The third line is the internal or external audit function, which assesses the robustness of model risk management practices and controls.
validation report
A formal document produced after a model validation process, outlining the artifacts, assessments, and recommendations related to a specific model’s performance, appropriateness, and limitations. Provides a comprehensive review of the model’s conceptual framework, data sources and integrity, calibration methods, and performance outcomes.

Within model risk management, the validation report is crucial for ensuring transparency, demonstrating regulatory compliance, and offering actionable insights for model refinement or adjustments.

Documentation

Each section of your development documentation should address critical aspects of the record’s lifecycle, from conceptualization and data preparation through development and ongoing management. This comprehensive documentation approach is essential for ensuring the record’s reliability, relevance, and compliance with business and regulatory standards.

For example, documentation for models typically include sections on:

conceptual soundness
Establishes the foundation of a selected record (such as a model), covering the overview, intended use and business use case, regulatory requirements, limitations, and the rationale behind selection. It emphasizes purpose, scope, and constraints, which are crucial for stakeholders to understand applicability and limitations.
data preparation
Details the data description, including dataset summary, data quality tests, descriptive statistics, correlations and interactions, and feature selection and engineering. It provides transparency into the data used for training, ensuring that the record (such as a model) is built on a solid and relevant dataset.
model development, development
Discusses the training, evaluation, explainability, interpretability, and diagnosis, including weak spots, overfit regions, and robustness. This section is vital for understanding how the record (such as a model) was developed, how it performs, and its areas of strength and weakness.
monitoring and governance
Focuses on the record (such as a model)’s ongoing monitoring plan, implementation, and governance plan. It outlines strategies for maintaining the performance over time and ensuring that it remains compliant with regulatory requirements and ethical standards.

Validation reports

A validation report is a comprehensive and structured review evaluating a record’s accuracy, performance, and suitability for its intended purpose. A report follows established validation guidelines to ensure consistency and adherence to internal and regulatory standards — encompassing the process of risk assessment, identifying areas of potential error or risk within the record’s components, supporting transparency, regulatory compliance, and informed decision-making by documenting the validator’s independent review and conclusions.

actions
Recommended steps or measures to address artifacts from validation or risk assessments.
artifacts (previously findings)
Observations or issues identified during validation, including any deviations from expected performance or standards. Artifacts are organized by type — default types provided by ​ValidMind include Validation Issue, Policy Exception, and Limitation. Custom artifact types can be created to track other categories relevant to your organization.
evidence
Material provided by the developer and reviewed by the validator, such as documentation, source code, datasets, monitoring reports or previous validation reports.
review
Entails a comprehensive evaluation process covering four key aspects of documentation to ensure thoroughness, compliance, and reliability:

  • Conceptual soundness — Examines the foundational elements of the record, including its overview, intended business use, regulatory requirements, and limitations. Ensures that the record’s purpose, scope, and constraints are well-defined and understood by stakeholders.

  • Data preparation — Assesses the quality and preparation of the data used for record training. Includes a detailed look at dataset summaries, data quality tests, descriptive statistics, correlations, interactions, and feature engineering. The aim is to verify that the record is built on a robust and relevant dataset.

  • Development — Focuses on the record’s development process, including training, evaluation, explainability, interpretability, and diagnosis. Highlights the record’s performance, identifies strengths and weaknesses, and ensures that any potential issues such as overfitting are addressed. Evaluates the assumptions made and examines the qualitative information and judgments to ensure they are conducted appropriately and systematically.

  • Monitoring and governance — Evaluates the ongoing strategies for monitoring the record’s performance and ensuring compliance with regulatory and ethical standards. Involves checking the implementation of the monitoring plan and governance strategies to maintain the record’s efficacy over time. Covers reporting outputs to ensure transparency and accuracy in the record’s documented results.

These elements collectively ensure that the documentation is thorough, transparent, and meets all necessary standards and regulatory requirements.

model risk assessment, risk assessment
The process of identifying and evaluating risks associated with the use and potential errors in a record (such as a model).
model risk areas, risk areas
Specific components or aspects of a record (such as a model) where risk might be present, such as data inputs, algorithms, or implementation.
validation guidelines
Established standards or procedures for conducting thorough and consistent validations, usually aligned with principles within specific records (such as models) or AI risk frameworks.

Ongoing monitoring

backtesting
Comparing a record’s predictions against actual outcomes to verify its predictive power and reliability.
compliance and regulatory adherence
Ensuring that the record (model) continues to meet evolving regulatory requirements and standards.
model drift, drift
Changes in data patterns, input distributions, or record (such as a model) behavior that may indicate a degradation in performance over time.
model performance, performance
The measure of a record’s accuracy, stability, and robustness in achieving its intended outcomes, which is regularly evaluated through monitoring after deployment to ensure ongoing reliability.
ongoing monitoring, ongoing monitoring report, ongoing monitoring plan, monitoring plan
A comprehensive and structured periodic report assessing the record’s performance and compliance over time, ensuring it remains valid under changing conditions. Monitoring includes key elements such as data sources, inputs, performance metrics, and periodic evaluations, ensuring transparency and visibility of the record’s performance in the production environment.
recalibrating models, recalibrating
The process of adjusting a record (such as a model) to account for detected drift or changes in the underlying data or environment.
reporting and governance
The documentation of monitoring artifacts and communication to stakeholders to support decision-making and maintain transparency.

Attestations

attestation
A formal process where attestation participants certify key record (model) information at a specific time. Attestation is part of your audit trail and confirms that governance, documentation, and control requirements are met.
attestation instance
The invocation of the attestation process on the ValidMind Platform. Created when the attestation is triggered by the schedule you set up, it includes a snapshot with record activity and artifacts, questionnaire responses and review status, forming a full record of the review and approval process.
attestation participant
A user who participates in the attestation workflow as a submitter, reviewer, or approver. Submitters are assigned from inventory record stakeholders; reviewers and approvers are assigned from organizational roles.
attestation period
The time window during which attestation is active, with fixed start and end dates. Each period creates an unchanging record (model) snapshot. Periods are usually scheduled quarterly or annually and can align with regulatory or internal cycles.
attestation questionnaire
A structured form that submitters use to confirm record (model) status, documentation and compliance. It supports formatted inputs like checkboxes and text fields, serving as both a compliance check and formal review record.
execution schedule
The mechanism, manual or automated, that starts the attestation process based on set periods. It creates attestation instances, triggers snapshots and begins the workflow for attestation participants.
group
An organizational unit that associates records (models) with specific teams or functions. When reviewers or approvers are assigned by role, they can only act on records within groups they belong to — resulting in one attestation submission per owner per group.
inventory scope
The filter conditions that define which records (models) are included in an attestation. Scope can be set using rules based on fields, stages, or custom attributes.
snapshot
A fixed capture of record (model) data at a specific time. It includes optional custom fields and related artifacts and stays unchanged throughout the attestation, ensuring historical accuracy.

Integrations

Integrations

integration
The part of ​ValidMind designed to exchange data or actions with another system, such as REST APIs or webhooks.
secret
A credential that stores the API key, password, or token required by an external service for authentication. There are two types: integration secrets for authenticating persistent connections to external platforms, and webhook secrets for authenticating outgoing HTTP Request steps in workflows via {secret:name} syntax.
connection
A packaged adapter that knows how to authenticate, map data, and orchestrate workflows with a specific third-party platform.
HTTP request
A workflow step that sends data from ​ValidMind to another service. Header values can reference webhook secrets using {secret:name} syntax to avoid storing credentials in plaintext.
webhook
A workflow step that pauses workflow execution and waits for an external system to send a POST request to a secure ​ValidMind URL. Another system sends a POST request to start the workflow or continue a paused workflow.

Developer tools

decorator, Python decorator
A design pattern in Python21 that allows a user to add new functionality to an existing object without modifying its structure.

Decorators are a simpler way for users to run their own code as a ​ValidMind test.

inputs
Objects to be evaluated and documented in the ValidMind Library. They can be any of the following:
  • model: A single record (such as a model) that has been initialized in ​ValidMind. Despite the naming convention, model objects can be any type of record you want to test, document, validate, or monitor with ​ValidMind.22
  • dataset: A single dataset that has been initialized in ​ValidMind.23
  • models: A list of ​ValidMind records — usually this is used when you want to compare multiple records in your custom tests.
  • datasets: A list of ​ValidMind datasets — usually this is used when you want to compare multiple datasets in your custom tests.24

22 Refer to: init_model()

23 Refer to: init_dataset()

outputs
Custom tests can return elements like tables or plots. Tables may be a list of dictionaries (each representing a row) or a pandas DataFrame. Plots may be matplotlib or plotly figures.
parameters
Additional arguments that can be passed when running a ​ValidMind test, used to pass additional information to a test, customize its behavior, or provide additional context.
pip
A package manager for Python, used to install and manage software packages written in the Python programming language.

​ValidMind uses the pip command to install the Python client library that is part of the ValidMind Library so that developers can make use of its features.

JupyterHub
A multi-user server provides a platform for users to interactively work with data science and scientific computing tools in a collaborative environment.

​ValidMind uses JupyterHub to share live code, how-to instructions, and visualizations via notebooks as part of our getting started experience for new users.

Jupyter Notebook
Allows users to create and share documents containing live code, data visualizations, and narrative text. Supports various programming languages, most notably Python, and is widely used for data analysis, machine learning, scientific research, and educational purposes.

​ValidMind uses notebooks to share sample code and how-to instructions with users that you can adapt to your own use case.

metrics, custom metrics
Metrics are a subset of tests that do not have thresholds. Custom metrics are functions that you define to evaluate your record (such as a model) or dataset. These functions can be registered via the ValidMind Library to be used with the ValidMind Platform.

In the context of ​ValidMind’s Jupyter Notebooks, metrics and tests can be thought of as interchangeable concepts.25

25 Refer also to: test

GitHub
A cloud-based platform that provides hosting for software development and version control using Git. GitHub26 offers collaboration tools such as bug tracking, feature requests, task management, and continuous integration pipelines.

26 GitHub

​ValidMind uses GitHub to share open-source software27 with you.

27 GitHub: validmind