Using ​ValidMind
for Model Validation

Validator Fundamentals — Module 1 of 4

Click to start

Learning objectives

“As a validator who is new to ​ValidMind, I want to learn how to connect to a champion model in the ValidMind Platform using the ValidMind Library installed in my local environment, preview the model’s validation report template, and review submitted documentation for the model.”


This first module is part of a four-part series:

Validator Fundamentals

Module 1 — Contents

  1. ​ValidMind for model validation
  2. The ValidMind Platform
  3. Connect to your champion model
  4. Preview your model’s validation report template
  5. Review submitted model documentation

First, let’s make sure you can log in to ​ValidMind.

Training is interactive — you explore ​ValidMind live. Try it!

→ , ↓ , SPACE , N — next slide     ← , ↑ , P , H — previous slide     ? — all keyboard shortcuts

Before you begin

To continue, you need to have been onboarded onto ValidMind Academy with the Validator role.


Already logged in and refreshed this module? Click to continue.

  1. Log in to check your access:

Log in to JupyterHub

Log in to ​ValidMind

Be sure to return to this page afterwards.

  1. After you successfully log in, refresh the page to connect this training module up to the ValidMind Platform:


ValidMind for model validation

Jupyter Notebook series

These notebooks walk you through how to validate a model using ​ValidMind, complete with supporting test results attached as evidence to your validation report.

​ValidMind for model validation

Our series of four introductory notebooks for model validators include sample code and how-to information to get you started with ​ValidMind:

1 — Set up the ValidMind Library for validation
2 — Start the model validation process
3 — Developing a potential challenger model
4 — Finalize testing and reporting

In this first module, we’ll run through 1 — Set up the ValidMind Library for validation together.


Let’s start our journey with 1 — Set up the ValidMind Library for validation on the next page.

1 — Set up the ValidMind Library for validation

During this course, we’ll run through these notebooks together, and at the end of your learning journey you’ll have a fully supported sample validation report ready for review.

For now, scroll through this notebook to explore. When you are done, click to continue.

The ValidMind Platform

Welcome to the ValidMind Platform

From here, you can:

  • Keep track of your models in the customizable inventory …
  • Review model documentation submitted by model developers …
  • Assess the compliance of models by logging findings and test results as evidence on validation reports …
  • View analytics on your models and findings, and create custom reports …
  • … and much more!

Can’t load the ValidMind Platform?

Make sure you’re logged in and have refreshed the page in a Chromium-based browser.

When you’re done navigating around, click to continue.

Welcome to the Inventory

Use the model inventory to track comprehensive details for all your models throughout the model lifecycle. The model inventory is customizable and extensible, with a layout that can be configured to suit your needs.

In a usual model lifecycle, a champion model will have been independently registered in your model inventory and submitted to you for validation.

For this training course, we’ll have you register a dummy model via the Register Model modal and assign yourself as the validator to familiarize you with the ValidMind Platform interface.

Let’s register a model together for use with validation on the next page.

Register a binary classification model

  1. Select the option for a new model:
  • Documentation template —
    Binary classification
  • Use case —
    Attrition/Churn Management

    You can fill in other options according to your preference.
  1. Click Register Model to add the model to your inventory.

When you’re done, click to continue.

Assign validator credentials

In order to log tests as a validator instead of as a developer, we’ll need to adjust your model stakeholder permissions:

  1. Select the name of your model you registered for this course to open up the model details page.
  2. Remove yourself as a model owner by clicking on the owners tile, then clicking on the x next to your name. Click Save to apply your changes to that role.
  3. Remove yourself as a developer by clicking on the developers tile, then clicking on the x next to your name. Click Save to apply your changes to that role.
  4. Add yourself as a validator by clicking on the validators tile, and selecting your name from the drop-down menu. Click Save to apply your changes to that role.

When you’re done, click to continue.

Connect to a model

Get your code snippet

​ValidMind generates a unique code snippet for each registered model to connect with your validation environment:

  1. Select the name of your model you registered for this course to open up the model details page.
  2. On the left sidebar that appears for your model, click Getting Started.
  3. Locate the code snippet and click Copy snippet to clipboard.

When you’re done, click to continue.

Can’t load the ValidMind Platform?

Make sure you’re logged in and have refreshed the page in a Chromium-based web browser.

Install & initialize the ValidMind Library

With your code snippet copied to your clipboard:

  1. Open 1 — Set up the ValidMind Library for validation: JupyterHub
  2. Run all the cells under the Setting up sections:
    • Install the ValidMind Library
    • Initialize the ValidMind Library

When you’re done, return to this page and click to continue.

Preview validation report

Preview the validation report template

You can preview your model’s validation report template right from the ValidMind Library:

  1. Continue with 1 — Set up the ValidMind Library for validation: JupyterHub
  2. Run all the cells in the sections under Getting to know ValidMind.

When you’re done, return to this page and click to continue.

Verify the report template

Once you’ve called preview_template():

  1. Select the name of your model you registered for this course to open up the model details page.
  2. On the left sidebar that appears for your model, click Validation Report.
  3. Note how the structure of the validation report reflects the previewed template.

When you’re done taking a look around, click to continue.

Review model documentation

Document Overview

Learn more…


Try it live on the next page.

The Document Overview shows a section-by-section outline of your model’s documentation, as well as summaries of:

  • Any unresolved conversations
  • The number of model findings
  • The completion status for your model’s documentation
  • Recent activity on your model

To locate your document overview for a model:

  1. In the left sidebar, click Inventory.

  2. Select a model or find your model by applying a filter or searching for it.

  3. In the left sidebar that appears for your model, click Documentation.

Explore sample model documentation

In a usual model lifecycle, a champion model will have been submitted to you for validation with completed model documentation.

Here, review the empty sample model documentation for the model you registered earlier to familiarize you with what the structure of documentation could look like when presented to you for review:

  1. Select the name of your model you registered for this course to open up the model details page.
  2. In the left sidebar that appears for your model, click Documentation.

The structure of the model documentation reflects the documentation template applied to the model, just like the validation report template.

When you’re done taking a look around the model documentation, click to continue.

In summary

Using ​ValidMind for model validation

In this first module, you learned how to:


Continue your model development journey with:

Running Data Quality Tests