October 25, 2023

Release highlights

We’ve introduced new features to the platform that enable you to remove blocks of content from documentation and work with your settings more effectively.

ValidMind Platform (v1.8.0)

Remove blocks from documentation

You can now remove blocks of text or test-related content from model documentation in the editor of the platform.

  • This new feature gives you more control over your documentation and enables you to remove content that is no longer needed.
  • To remove text blocks and test-driven blocks from model documentation, you first select the block you want to remove and click , either in the text-block’s toolbar or in the test-driven’s block single-button toolbar.

Remove content blocks

A gif showcasing the process of removing a test-driven block

Removing a test-driven block

New Settings landing page

A new Settings landing page now organizes more of your settings for the ValidMind Platform in one convenient place.

From this page you can manage:

  • Your organization, including the name and the API and secret key you use to connect to the ValidMind Platform.
  • The documentation templates that standardize the documentation table of contents for your projects and configure the required validation tests for specific model use cases.
  • Workflows that determine the statuses of your model and how it transitions through your model risk management process according to your requirements.

A screenshot showing the new ValidMind Platform settings page

New Settings page

Enhancements

Rouge and Bert Score tests now show average scores

  • Introduced RougeMetricsAggregate and BertScoreAggregate to offer a high-level overview of model performance across a large number of text rows.
  • These tests complement the detailed row-by-row analysis provided by RougeScore and BertScore.

RougeScore

BertScore

Tests for safety toxicity and bias in text summarization

We introduced several new tests to evaluate safety and bias risks in text summarization:

  • ToxicityScore: Measures safety risk
  • ToxicityHistogram: Provides a distribution of safety risk scores
  • RegardScore: Evaluates bias risk
  • RegardHistogram: Shows distribution of bias risk scores

ToxicityScore

RegardScore

Bug fixes

Shap test issue resolved

  • We set matplotlib to version 3.7.x in pyproject.toml to fix an incompatibility with the latest matplotlib version (3.8.0).
  • This incompatibility was causing SHAP plot errors. We will keep track of matplotlib releases for future updates. Once fixed, we will consider updating the version.

Documentation updates

Product overview rewrite

We expanded our platform overview to provide more background information about what ValidMind offers and how we enable you to comply with policies and regulations such as SR 11-7 and SS1/23.

About ValidMind

QuickStart updates for the closed beta

We updated the QuickStart section of our documentation to reflect recent ValidMind Platform and sign-up flow changes.

QuickStart

Improved handling of Jupyter Notebooks

We now programmatically embed our Jupyter Notebooks in our documentation site and generate a downloadable notebooks.zip file with all notebooks and supporting datasets.

Code samples

Run tests and test suites

How to upgrade

ValidMind Platform

To access the latest version of the ValidMind Platform,1 hard refresh your browser tab:

  • Windows: Ctrl + Shift + R OR Ctrl + F5
  • MacOS: ⌘ Cmd + Shift + R OR hold down ⌘ Cmd and click the Reload button

ValidMind Library

To upgrade the ValidMind Library:2

  1. In your Jupyter Notebook:

    • Using JupyterHub:3 Hard refresh your browser tab.
    • In your own developer environment:4 Restart your notebook.

  2. Then within a code cell or your terminal, run:

    %pip install --upgrade validmind

You may need to restart your kernel after running the upgrade package for changes to be applied.