Work with metrics over time

Published

October 24, 2024

Once generated via the ValidMind Developer Framework, view and add metrics over time to your ongoing monitoring plans in the ValidMind Platform UI.

Metrics over time refers to the continued monitoring of a model’s performance once it is deployed. Tracking how a model performs as new data is introduced or conditions change ensures that it remains accurate and reliable in real-world environments where data distributions or market conditions shift.

Prerequisites

Add metrics over time

  1. In the left sidebar, click Model Inventory.

  2. Select a model by clicking on it or find your model by applying a filter or searching for it.3

  3. In the left sidebar that appears for your model, click Documentation or Ongoing Monitoring.

    You can now jump to any section of the model documentation or ongoing monitoring plan by expanding the table of contents on the left and selecting the relevant section you would like to add content to, such as 1.1 Model Monitoring Performance History.

  4. Hover your mouse over the space where you want your new block to go until a horizontal dashed line with a sign appears that indicates you can insert a new block:

    A gif showing the process of adding a content block in the UI

    Adding a content block in the UI
  5. Click and then select Metric Over Time.4

    By default, only the Developer role can add metrics over time within model documentation or ongoing monitoring plans.

  6. Select metric over time results:

    • Select the metric over time to insert into the model documentation from the list of available metrics.
    • Search by name using Search on the top-left to locate specific metrics.

    A screenshot showing several metric over time blocks that have been selected for insertion

    Metric over time blocks that have been selected for insertion

    To preview what is included in a metric, click on it. By default, the actively selected metric is previewed.

  7. Click Insert # Metrics(s) Over Time to Document when you are ready.

  8. After inserting the metrics into your document, review the data to confirm that it is accurate and relevant.

View metric over time metadata

After you have added metrics over time to your document, you can view the following information attached to the result:

  • Date and time the metric was recorded
  • Who updated the metric
  • The numeric value of the metric
  • Any additional parameters
  1. In the left sidebar, click Model Inventory.

  2. Select a model by clicking on it or find your model by applying a filter or searching for it.5

  3. In the left sidebar that appears for your model, click Documentation or Ongoing Monitoring.

  4. Locate the metric whose metadata you want to view.

  5. Under the metric’s name, click on Data tab.

    A screenshot showing the Data tab within a metric over time

What’s next