Work with metrics over time
Once generated via the ValidMind Developer Framework, view and add metrics over time to your ongoing monitoring plans in the ValidMind Platform UI.
Metrics over time refers to the continued monitoring of a model’s performance once it is deployed. Tracking how a model performs as new data is introduced or conditions change ensures that it remains accurate and reliable in real-world environments where data distributions or market conditions shift.
- Model performance is determined by continuously measuring metrics and comparing them over time to detect degradation, bias, or shifts in the model’s output.
- Performance data is collected and tracked over time, often using a rolling window approach or real-time monitoring tools with the same metrics used in testing, but observed across different periods.
- Continuous tracking helps to identify if and when a model needs to be recalibrated, retrained, or even replaced due to performance deterioration or changing conditions.
Prerequisites
Add metrics over time
In the left sidebar, click Model Inventory.
Select a model by clicking on it or find your model by applying a filter or searching for it.3
In the left sidebar that appears for your model, click Documentation or Ongoing Monitoring.
You can now jump to any section of the model documentation or ongoing monitoring plan by expanding the table of contents on the left and selecting the relevant section you would like to add content to, such as 1.1 Model Monitoring Performance History.
Hover your mouse over the space where you want your new block to go until a horizontal dashed line with a sign appears that indicates you can insert a new block:
Adding a content block in the UI Click and then select Metric Over Time.4
By default, only the
Developerrole can add metrics over time within model documentation or ongoing monitoring plans.Select metric over time results:
- Select the metric over time to insert into the model documentation from the list of available metrics.
- Search by name using Search on the top-left to locate specific metrics.
Metric over time blocks that have been selected for insertion To preview what is included in a metric, click on it. By default, the actively selected metric is previewed.
Click Insert # Metrics(s) Over Time to Document when you are ready.
After inserting the metrics into your document, review the data to confirm that it is accurate and relevant.
View metric over time metadata
After you have added metrics over time to your document, you can view the following information attached to the result:
- Date and time the metric was recorded
- Who updated the metric
- The numeric value of the metric
- Any additional parameters
In the left sidebar, click Model Inventory.
Select a model by clicking on it or find your model by applying a filter or searching for it.5
In the left sidebar that appears for your model, click Documentation or Ongoing Monitoring.
Locate the metric whose metadata you want to view.
Under the metric’s name, click on Data tab.
