Enable monitoring

To start uploading ongoing monitoring results for a model to ValidMind, you enable monitoring in your code snippet and then select a monitoring template. You can enable monitoring for both new and existing models.

The process requires two steps:

1. Add monitoring=True to your code snippet
2. Select a monitoring template

To try out monitoring, check out the code sample for ongoing monitoring of models.

Quickstart for ongoing monitoring of models with ValidMind

Prerequisites

• You are logged in to ValidMind.
• You are a Developer or Validator, or assigned another role with sufficient permissions to perform the tasks in this guide.¹
• The model is in the pre-approval stage for performance testing or the model has been approved and is in production.

¹ Manage permissions

Add monitoring=True to your code snippet

ValidMind generates a unique code snippet for each registered model to connect with your developer environment. You initialize the client library with this code snippet, which ensures that your documentation and tests are uploaded to the correct model when you run the notebook.

To enable ongoing monitoring for a model, add monitoring=True to your code snippet:

1. In the ValidMind Platform UI, either register a new model² or locate an existing model:

   • In the left sidebar, click Model Inventory.
   • Select a model by clicking on it or find your model by applying a filter or searching for it.³

2. Copy the code snippet for the model:

   • In the left sidebar that appears for your model, click Getting Started.
   • Locate the code snippet and click Copy snippet to clipboard.

3. Paste the code snippet into your code and add monitoring=True to the vm.init method, similar to this example:

   import validmind as vm
   
   vm.init(
       api_host="https://api.prod.validmind.ai/api/v1/tracking",
       api_key="...",
       api_secret="...",
       project="...",
       monitoring=True
   )

² Register models in the inventory

³ Working with the model inventory

Select a monitoring template

Before you can start sending ongoing monitoring data from your developer environment to ValidMind, you must select a monitoring template:

1. In the ValidMind Platform UI, click Model Inventory in the left sidebar.

2. Select a model by clicking on it or find your model by applying a filter or searching for it.⁴

3. In the left sidebar, click Ongoing Monitoring.

4. From the Template drop-down, select one of the available monitoring templates.

5. Click Use Template.

⁴ Working with the model inventory

After you select a template, the initial document where your ongoing monitoring results will be stored becomes available. You can review this documents structure and then start running your code to populate the template with monitoring results.

Troubleshooting

If you see errors similar to this one when running your code, make sure that you have selected your monitoring template and then re-run your code:

APIRequestError: Please select an ongoing monitoring template on the ValidMind platform first

What’s next

After you have enabled ongoing monitoring and run your code to generate some output, you can start reviewing the monitoring results.⁵

⁵ Review monitoring results