ValidMind for validation 4 — Finalize testing and reporting

Learn how to use ValidMind for your end-to-end validation process with our series of four introductory notebooks. In this last notebook, finalize the compliance assessment process and have a complete validation report ready for review.

This notebook will walk you through how to supplement ValidMind tests with your own custom tests and include them as additional evidence in your validation report. A custom test is any function that takes a set of inputs and parameters as arguments and returns one or more outputs:

• The function can be as simple or as complex as you need it to be — it can use external libraries, make API calls, or do anything else that you can do in Python.
• The only requirement is that the function signature and return values can be "understood" and handled by the ValidMind Library. As such, custom tests offer added flexibility by extending the default tests provided by ValidMind, enabling you to document any type of record (model) or use case.

For a more in-depth introduction to custom tests, refer to our Implement custom tests notebook.

Learn by doing

Our course tailor-made for validators new to ValidMind combines this series of notebooks with more a more in-depth introduction to the ValidMind Platform — Validator Fundamentals

Prerequisites

In order to finalize validation and reporting, you'll need to first have:

• Registered a model within the ValidMind Platform and granted yourself access to the model as a validator
• Installed the ValidMind Library in your local environment, allowing you to access all its features
• Learned how to import and initialize datasets and models for use with ValidMind
• Understood the basics of how to identify and run validation tests
• Run validation tests for your champion and challengers, and logged the results of those tests to the ValidMind Platform
• Inserted your logged test results into your validation report
• Added some preliminary artifacts (findings) to your validation report

Need help with the above steps?

Refer to the first three notebooks in this series:

• 1 — Set up the ValidMind Library for validation
• 2 — Start the validation process
• 3 — Developing a potential challenger

Setting up

This section should be very familiar to you now — as we performed the same actions in the previous two notebooks in this series.

Initialize the ValidMind Library

As usual, let's first connect up the ValidMind Library to our model we previously registered in the ValidMind Platform:

1. On the left sidebar that appears for your model, select Getting Started and select Validation from the Document drop-down menu.

2. Click Copy snippet to clipboard.

3. Next, load your model identifier credentials from an .env file or replace the placeholder with your own code snippet:

# Make sure the ValidMind Library is installed

%pip install -q validmind

# Load your model identifier credentials from an `.env` file

%load_ext dotenv
%dotenv .env

# Or replace with your code snippet

import validmind as vm

vm.init(
    # api_host="...",
    # api_key="...",
    # api_secret="...",
    # model="...",
    document="validation-report",
)

Note: you may need to restart the kernel to use updated packages.

2026-10-30 21:02:52,684 - INFO(validmind.api_client): 🎉 Connected to ValidMind!
📊 Model: [ValidMind Academy] Model validation (ID: cmalguc9y02ok199q2db381ib)
📁 Document Type: validation_report

Import the sample dataset

Next, we'll load in the same sample Bank Customer Churn Prediction dataset used to develop the champion that we will independently preprocess:

# Load the sample dataset
from validmind.datasets.classification import customer_churn as demo_dataset

print(
    f"Loaded demo dataset with: \n\n\t• Target column: '{demo_dataset.target_column}' \n\t• Class labels: {demo_dataset.class_labels}"
)

raw_df = demo_dataset.load_data()

Loaded demo dataset with: 

    • Target column: 'Exited' 
    • Class labels: {'0': 'Did not exit', '1': 'Exited'}

# Initialize the raw dataset for use in ValidMind tests
vm_raw_dataset = vm.init_dataset(
    dataset=raw_df,
    input_id="raw_dataset",
    target_column="Exited",
)

import pandas as pd

raw_copy_df = raw_df.sample(frac=1)  # Create a copy of the raw dataset

# Create a balanced dataset with the same number of exited and not exited customers
exited_df = raw_copy_df.loc[raw_copy_df["Exited"] == 1]
not_exited_df = raw_copy_df.loc[raw_copy_df["Exited"] == 0].sample(n=exited_df.shape[0])

balanced_raw_df = pd.concat([exited_df, not_exited_df])
balanced_raw_df = balanced_raw_df.sample(frac=1, random_state=42)

Let’s also quickly remove highly correlated features from the dataset using the output from a ValidMind test:

# Register new data and now 'balanced_raw_dataset' is the new dataset object of interest
vm_balanced_raw_dataset = vm.init_dataset(
    dataset=balanced_raw_df,
    input_id="balanced_raw_dataset",
    target_column="Exited",
)

# Run HighPearsonCorrelation test with our balanced dataset as input and return a result object
corr_result = vm.tests.run_test(
    test_id="validmind.data_validation.HighPearsonCorrelation",
    params={"max_threshold": 0.3},
    inputs={"dataset": vm_balanced_raw_dataset},
)

❌ High Pearson Correlation

The High Pearson Correlation test evaluates pairwise linear relationships between features to identify highly correlated variable pairs that may indicate redundancy or multicollinearity. The results table lists the top feature pairs by Pearson correlation coefficient together with a Pass/Fail assessment using the configured absolute-value threshold of 0.3. Across the ten reported pairs, coefficient values range from -0.1862 to 0.3427. Only one pair, (Age, Exited), exceeds the threshold and is marked as Fail, while all remaining reported pairs are marked as Pass.

Key insights:

• Single threshold breach identified: The pair (Age, Exited) has a Pearson correlation coefficient of 0.3427, exceeding the configured threshold of 0.3 and receiving a Fail designation.
• All other reported pairs pass: The remaining nine reported feature pairs have absolute correlation values below the threshold, with coefficients ranging from -0.1862 to 0.1616.
• Observed relationships are generally weak: Aside from (Age, Exited), the largest absolute correlations in the reported output are -0.1862 for (IsActiveMember, Exited) and -0.1752 for (Balance, NumOfProducts), both well below the threshold.
• Both positive and negative correlations appear: The reported coefficients include positive values such as 0.1616 for (Balance, Exited) and negative values such as -0.0777 for (NumOfProducts, Exited), indicating mixed linear relationship directions among the listed pairs.

The reported correlation structure is limited to one pair exceeding the configured threshold, with (Age, Exited) standing out as the only failed result. All other displayed feature pairs remain below the threshold and show comparatively small absolute correlation magnitudes. Overall, the output indicates a predominantly low pairwise linear correlation pattern within the set of top reported relationships, with one exception identified by the test.

Parameters:

{
  "max_threshold": 0.3
}
            

Tables

Columns	Coefficient	Pass/Fail
(Age, Exited)	0.3427	Fail
(IsActiveMember, Exited)	-0.1862	Pass
(Balance, NumOfProducts)	-0.1752	Pass
(Balance, Exited)	0.1616	Pass
(NumOfProducts, Exited)	-0.0777	Pass
(Age, NumOfProducts)	-0.0610	Pass
(Tenure, IsActiveMember)	-0.0573	Pass
(Age, Balance)	0.0455	Pass
(NumOfProducts, IsActiveMember)	0.0324	Pass
(CreditScore, Exited)	-0.0296	Pass

# From result object, extract table from `corr_result.tables`
features_df = corr_result.tables[0].data
features_df

	Columns	Coefficient	Pass/Fail
0	(Age, Exited)	0.3427	Fail
1	(IsActiveMember, Exited)	-0.1862	Pass
2	(Balance, NumOfProducts)	-0.1752	Pass
3	(Balance, Exited)	0.1616	Pass
4	(NumOfProducts, Exited)	-0.0777	Pass
5	(Age, NumOfProducts)	-0.0610	Pass
6	(Tenure, IsActiveMember)	-0.0573	Pass
7	(Age, Balance)	0.0455	Pass
8	(NumOfProducts, IsActiveMember)	0.0324	Pass
9	(CreditScore, Exited)	-0.0296	Pass

# Extract list of features that failed the test
high_correlation_features = features_df[features_df["Pass/Fail"] == "Fail"]["Columns"].tolist()
high_correlation_features

['(Age, Exited)']

# Extract feature names from the list of strings
high_correlation_features = [feature.split(",")[0].strip("()") for feature in high_correlation_features]
high_correlation_features

['Age']

# Remove the highly correlated features from the dataset
balanced_raw_no_age_df = balanced_raw_df.drop(columns=high_correlation_features)

# Re-initialize the dataset object
vm_raw_dataset_preprocessed = vm.init_dataset(
    dataset=balanced_raw_no_age_df,
    input_id="raw_dataset_preprocessed",
    target_column="Exited",
)

# Re-run the test with the reduced feature set
corr_result = vm.tests.run_test(
    test_id="validmind.data_validation.HighPearsonCorrelation",
    params={"max_threshold": 0.3},
    inputs={"dataset": vm_raw_dataset_preprocessed},
)

✅ High Pearson Correlation

The High Pearson Correlation test evaluates pairwise linear relationships among features to identify potentially redundant variables or multicollinearity. The reported output lists the top 10 strongest correlations after removing duplicate and self-correlations, with each pair labeled against the configured threshold of 0.3. In this result set, all reported coefficients fall between -0.1862 and 0.1616 in magnitude, and each pair is marked as Pass.

Key insights:

• No correlations exceed threshold: All 10 reported feature pairs pass the test against the 0.3 threshold, with no absolute correlation coefficient reaching the fail condition.

• Largest observed relationship is modest: The strongest correlation in the output is between IsActiveMember and Exited at -0.1862, which remains below the configured threshold.

• Top relationships are weak in magnitude: Other comparatively larger coefficients include Balance with NumOfProducts at -0.1752 and Balance with Exited at 0.1616, with remaining reported values closer to zero.

• Reported pairs show limited linear dependence: Several feature pairs, including NumOfProducts with IsActiveMember (0.0324), Tenure with EstimatedSalary (0.0273), and HasCrCard with IsActiveMember (-0.0219), exhibit near-zero Pearson correlation.

The reported correlation structure shows limited linear association among the top-ranked feature pairs under the High Pearson Correlation test. No pair exceeds the 0.3 threshold, and the strongest observed relationships remain modest in magnitude. Within the set of reported results, the evidence does not indicate pronounced pairwise linear redundancy.

Parameters:

{
  "max_threshold": 0.3
}
            

Tables

Columns	Coefficient	Pass/Fail
(IsActiveMember, Exited)	-0.1862	Pass
(Balance, NumOfProducts)	-0.1752	Pass
(Balance, Exited)	0.1616	Pass
(NumOfProducts, Exited)	-0.0777	Pass
(Tenure, IsActiveMember)	-0.0573	Pass
(NumOfProducts, IsActiveMember)	0.0324	Pass
(CreditScore, Exited)	-0.0296	Pass
(Tenure, EstimatedSalary)	0.0273	Pass
(CreditScore, HasCrCard)	-0.0228	Pass
(HasCrCard, IsActiveMember)	-0.0219	Pass

Split the preprocessed dataset

With our raw dataset rebalanced with highly correlated features removed, let's now spilt our dataset into train and test in preparation for model evaluation testing:

# Encode categorical features in the dataset
balanced_raw_no_age_df = pd.get_dummies(
    balanced_raw_no_age_df, columns=["Geography", "Gender"], drop_first=True
)
balanced_raw_no_age_df.head()

	CreditScore	Tenure	Balance	NumOfProducts	HasCrCard	IsActiveMember	EstimatedSalary	Exited	Geography_Germany	Geography_Spain	Gender_Male
1414	555	9	124969.13	1	1	0	60194.05	0	False	False	True
1187	576	6	0.00	2	1	1	48314.00	0	False	False	True
2404	774	3	137781.65	1	0	0	199316.19	0	False	False	False
6983	655	6	146498.76	1	1	0	64853.51	1	True	False	False
897	434	9	125801.03	2	1	0	60891.80	1	True	False	False

from sklearn.model_selection import train_test_split

# Split the dataset into train and test
train_df, test_df = train_test_split(balanced_raw_no_age_df, test_size=0.20)

X_train = train_df.drop("Exited", axis=1)
y_train = train_df["Exited"]
X_test = test_df.drop("Exited", axis=1)
y_test = test_df["Exited"]

# Initialize the split datasets
vm_train_ds = vm.init_dataset(
    input_id="train_dataset_final",
    dataset=train_df,
    target_column="Exited",
)

vm_test_ds = vm.init_dataset(
    input_id="test_dataset_final",
    dataset=test_df,
    target_column="Exited",
)

Import the champion model

With our raw dataset assessed and preprocessed, let's go ahead and import the champion submitted by the development team in the format of a .pkl file: lr_model_champion.pkl

# Import the champion model
import pickle as pkl

with open("lr_model_champion.pkl", "rb") as f:
    log_reg = pkl.load(f)

/opt/hostedtoolcache/Python/3.11.17/x64/lib/python3.11/site-packages/sklearn/base.py:442: InconsistentVersionWarning: Trying to unpickle estimator LogisticRegression from version 1.3.2 when using version 1.7.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(

Train potential challenger model

We'll also train our random forest classification challenger to see how it compares:

# Import the Random Forest Classification model
from sklearn.ensemble import RandomForestClassifier

# Create the model instance with 50 decision trees
rf_model = RandomForestClassifier(
    n_estimators=50,
    random_state=42,
)

# Train the model
rf_model.fit(X_train, y_train)

RandomForestClassifier(n_estimators=50, random_state=42)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Initialize the ValidMind models

In addition to the initialized datasets, you'll also need to initialize a ValidMind model object (vm_model) that can be passed to other functions for analysis and tests on the data for each of our two models:

# Initialize the champion logistic regression model
vm_log_model = vm.init_model(
    log_reg,
    input_id="log_model_champion",
)

# Initialize the challenger random forest classification model
vm_rf_model = vm.init_model(
    rf_model,
    input_id="rf_model",
)

# Assign predictions to Champion — Logistic regression model
vm_train_ds.assign_predictions(model=vm_log_model)
vm_test_ds.assign_predictions(model=vm_log_model)

# Assign predictions to Challenger — Random forest classification model
vm_train_ds.assign_predictions(model=vm_rf_model)
vm_test_ds.assign_predictions(model=vm_rf_model)

2026-10-30 21:03:09,800 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-10-30 21:03:09,802 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-10-30 21:03:09,802 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-10-30 21:03:09,805 - INFO(validmind.vm_models.dataset.utils): Done running predict()
2026-10-30 21:03:09,807 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-10-30 21:03:09,808 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-10-30 21:03:09,809 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-10-30 21:03:09,809 - INFO(validmind.vm_models.dataset.utils): Done running predict()
2026-10-30 21:03:09,811 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-10-30 21:03:09,826 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-10-30 21:03:09,826 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-10-30 21:03:09,841 - INFO(validmind.vm_models.dataset.utils): Done running predict()
2026-10-30 21:03:09,842 - INFO(validmind.vm_models.dataset.utils): Running predict_proba()... This may take a while
2026-10-30 21:03:09,848 - INFO(validmind.vm_models.dataset.utils): Done running predict_proba()
2026-10-30 21:03:09,849 - INFO(validmind.vm_models.dataset.utils): Running predict()... This may take a while
2026-10-30 21:03:09,855 - INFO(validmind.vm_models.dataset.utils): Done running predict()

Implementing custom tests

Thanks to the documentation (Learn more: ValidMind for development), we know that the development team implemented a custom test to further evaluate the performance of the champion.

In a usual validation situation, you would load a saved custom test provided by the development team. In the following section, we'll have you implement the same custom test and make it available for reuse, to familiarize you with the processes.

Want to learn more about custom tests?

Refer to our in-depth introduction to custom tests: Implement custom tests

Implement a custom inline test

Let's implement the same custom inline test that calculates the confusion matrix for a binary classification model that the development team used in their performance evaluations.

• An inline test refers to a test written and executed within the same environment as the code being tested — in this case, right in this Jupyter Notebook — without requiring a separate test file or framework.
• You'll note that the custom test function is just a regular Python function that can include and require any Python library as you see fit.

Create a confusion matrix plot

Let's first create a confusion matrix plot using the confusion_matrix function from the sklearn.metrics module:

import matplotlib.pyplot as plt
from sklearn import metrics

# Get the predicted classes
y_pred = log_reg.predict(vm_test_ds.x)

confusion_matrix = metrics.confusion_matrix(y_test, y_pred)

cm_display = metrics.ConfusionMatrixDisplay(
    confusion_matrix=confusion_matrix, display_labels=[False, True]
)
cm_display.plot()

Next, create a @vm.test wrapper that will allow you to create a reusable test. Note the following changes in the code below:

• The function confusion_matrix takes two arguments dataset and model. This is a VMDataset and VMModel object respectively.
  • VMDataset objects allow you to access the dataset's true (target) values by accessing the .y attribute.
  • VMDataset objects allow you to access the predictions for a given record (model) by accessing the .y_pred() method.
• The function docstring provides a description of what the test does. This will be displayed along with the result in this notebook as well as in the ValidMind Platform.
• The function body calculates the confusion matrix using the sklearn.metrics.confusion_matrix function as we just did above.
• The function then returns the ConfusionMatrixDisplay.figure_ object — this is important as the ValidMind Library expects the output of the custom test to be a plot or a table.
• The @vm.test decorator is doing the work of creating a wrapper around the function that will allow it to be run by the ValidMind Library. It also registers the test so it can be found by the ID my_custom_tests.ConfusionMatrix.

@vm.test("my_custom_tests.ConfusionMatrix")
def confusion_matrix(dataset, model):
    """The confusion matrix is a table that is often used to describe the performance of a classification model on a set of data for which the true values are known.

    The confusion matrix is a 2x2 table that contains 4 values:

    - True Positive (TP): the number of correct positive predictions
    - True Negative (TN): the number of correct negative predictions
    - False Positive (FP): the number of incorrect positive predictions
    - False Negative (FN): the number of incorrect negative predictions

    The confusion matrix can be used to assess the holistic performance of a classification model by showing the accuracy, precision, recall, and F1 score of the model on a single figure.
    """
    y_true = dataset.y
    y_pred = dataset.y_pred(model=model)

    confusion_matrix = metrics.confusion_matrix(y_true, y_pred)

    cm_display = metrics.ConfusionMatrixDisplay(
        confusion_matrix=confusion_matrix, display_labels=[False, True]
    )
    cm_display.plot()

    plt.close()  # close the plot to avoid displaying it

    return cm_display.figure_  # return the figure object itself

You can now run the newly created custom test on both the training and test datasets for both models using the run_test() function:

# Champion train and test
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:champion",
    input_grid={
        "dataset": [vm_train_ds,vm_test_ds],
        "model" : [vm_log_model]
    }
).log()

Confusion Matrix Champion

The Confusion Matrix test evaluates classification outcomes by comparing predicted labels against true labels and presenting the counts of true positives, true negatives, false positives, and false negatives. Results are shown separately for the training dataset and the test dataset. In the training dataset, the matrix entries are 825 true negatives, 476 false positives, 480 false negatives, and 804 true positives. In the test dataset, the corresponding counts are 198 true negatives, 117 false positives, 127 false negatives, and 205 true positives.

Key insights:

• Correct classifications exceed errors: In both datasets, the diagonal cells are larger than the off-diagonal cells. Training results show 1,629 correct classifications versus 956 errors, while test results show 403 correct classifications versus 244 errors.

• Balanced error counts across classes: The two error types are similar in magnitude within each dataset. Training false positives and false negatives are 476 and 480 respectively, and test false positives and false negatives are 117 and 127 respectively.

• Class outcomes are similarly distributed: Correct negative and correct positive classifications are of similar scale in both datasets. Training true negatives and true positives are 825 and 804, while test true negatives and true positives are 198 and 205.

• Train and test confusion patterns are aligned: The relative structure of the confusion matrices is similar across datasets, with comparable balance between true negatives and true positives and no single error type dominating the results.

The confusion matrix results show that the model produces more correct than incorrect classifications in both the training and test datasets. Error counts are distributed relatively evenly between false positives and false negatives, and correct classifications are also balanced between the negative and positive classes. The similarity in matrix structure across training and test samples indicates consistent classification behavior between the two datasets.

Figures

2026-10-30 21:03:18,794 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:champion does not exist in model's document

# Challenger train and test
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:challenger",
    input_grid={
        "dataset": [vm_train_ds,vm_test_ds],
        "model" : [vm_rf_model]
    }
).log()

Confusion Matrix Challenger

The Confusion Matrix test evaluates classification performance by comparing predicted labels with observed labels across the train and test datasets. The results are presented as 2x2 confusion matrices with counts for true negatives, false positives, false negatives, and true positives. For the train dataset, the matrix shows 1301 true negatives, 0 false positives, 1 false negative, and 1283 true positives. For the test dataset, the matrix shows 230 true negatives, 85 false positives, 106 false negatives, and 226 true positives.

Key insights:

• Near-perfect training classification: The train confusion matrix contains 1301 true negatives and 1283 true positives, with only 1 false negative and 0 false positives. Misclassification on the training sample is effectively absent.
• Higher error counts on test data: The test confusion matrix shows 191 total misclassifications, comprising 85 false positives and 106 false negatives. This contrasts materially with the train matrix, which contains only 1 total error.
• Both error types are present in testing: On the test dataset, errors occur in both directions rather than being concentrated in a single class. False negatives (106) exceed false positives (85), indicating slightly more missed positive cases than incorrect positive assignments.
• Class counts are relatively balanced: In the train dataset, observed negatives and positives total 1301 and 1284 respectively; in the test dataset, they total 315 and 332 respectively. The confusion matrices therefore reflect performance on samples with similar class representation across the two labels.

The confusion matrix results show extremely strong fit on the training dataset and materially lower classification accuracy on the test dataset. Test-set errors are distributed across both false positives and false negatives, with a modestly higher count of false negatives. Across both datasets, the underlying class totals are relatively balanced, so the observed differences primarily reflect changes in classification outcomes rather than shifts in class composition.

Figures

2026-10-30 21:03:27,365 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:challenger does not exist in model's document

Note the output returned indicating that a test-driven block doesn't currently exist in your documentation for some test IDs.

That's expected, as when we run validations tests the results logged need to be manually added to your report as part of your compliance assessment process within the ValidMind Platform.

Add parameters to custom tests

Custom tests can take parameters just like any other function. To demonstrate, let's modify the confusion_matrix function to take an additional parameter normalize that will allow you to normalize the confusion matrix:

@vm.test("my_custom_tests.ConfusionMatrix")
def confusion_matrix(dataset, model, normalize=False):
    """The confusion matrix is a table that is often used to describe the performance of a classification model on a set of data for which the true values are known.

    The confusion matrix is a 2x2 table that contains 4 values:

    - True Positive (TP): the number of correct positive predictions
    - True Negative (TN): the number of correct negative predictions
    - False Positive (FP): the number of incorrect positive predictions
    - False Negative (FN): the number of incorrect negative predictions

    The confusion matrix can be used to assess the holistic performance of a classification model by showing the accuracy, precision, recall, and F1 score of the model on a single figure.
    """
    y_true = dataset.y
    y_pred = dataset.y_pred(model=model)

    if normalize:
        confusion_matrix = metrics.confusion_matrix(y_true, y_pred, normalize="all")
    else:
        confusion_matrix = metrics.confusion_matrix(y_true, y_pred)

    cm_display = metrics.ConfusionMatrixDisplay(
        confusion_matrix=confusion_matrix, display_labels=[False, True]
    )
    cm_display.plot()

    plt.close()  # close the plot to avoid displaying it

    return cm_display.figure_  # return the figure object itself

Pass parameters to custom tests

You can pass parameters to custom tests by providing a dictionary of parameters to the run_test() function.

• The parameters will override any default parameters set in the custom test definition. Note that dataset and model are still passed as inputs.
• Since these are VMDataset or VMModel inputs, they have a special meaning.

Re-running and logging the custom confusion matrix with normalize=True for both models and our testing dataset looks like this:

# Champion with test dataset and normalize=True
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:test_normalized_champion",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_log_model]
    },
    params={"normalize": True}
).log()

Confusion Matrix Test Normalized Champion

The Confusion Matrix test evaluates classification performance by comparing predicted labels against true labels and displaying the normalized shares of true negatives, false positives, false negatives, and true positives. For the test dataset and champion logistic model, the normalized matrix shows values of 0.31 for true negatives, 0.18 for false positives, 0.20 for false negatives, and 0.32 for true positives. The diagonal cells therefore represent the correctly classified share, while the off-diagonal cells represent the misclassified share.

Key insights:

• Correct classifications dominate overall: The diagonal cells sum to 0.63, comprising 0.31 true negatives and 0.32 true positives, which exceeds the combined off-diagonal error share of 0.38.
• Performance is balanced across classes: Correct classification shares are similar for the negative and positive classes, with 0.31 true negatives and 0.32 true positives.
• False negatives slightly exceed false positives: The normalized false negative share is 0.20 versus 0.18 for false positives, indicating marginally more missed positive cases than incorrect positive assignments.
• Error mass remains material in both directions: Both misclassification cells are non-trivial, with 0.18 in the false positive cell and 0.20 in the false negative cell, showing that errors occur for both classes rather than being concentrated in one outcome.

The normalized confusion matrix indicates that the model classifies a larger share of observations correctly than incorrectly, with similar contributions from true negative and true positive outcomes. Misclassifications are present in both directions at comparable levels, with a slightly higher share of false negatives than false positives. Overall, the observed pattern reflects relatively balanced classification behavior across the two classes.

Parameters:

{
  "normalize": true
}
            

Figures

2026-10-30 21:03:36,277 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:test_normalized_champion does not exist in model's document

# Challenger with test dataset and normalize=True
vm.tests.run_test(
    test_id="my_custom_tests.ConfusionMatrix:test_normalized_challenger",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_rf_model]
    },
    params={"normalize": True}
).log()

Confusion Matrix Test Normalized Challenger

The Confusion Matrix test evaluates classification outcomes by comparing predicted labels with true labels, and this result presents a normalized confusion matrix for the challenger model on the test dataset. The matrix shows the relative share of observations in each outcome category, with values displayed for true negatives, false positives, false negatives, and true positives. In this result, the four normalized cell values are 0.36 for true negatives, 0.13 for false positives, 0.16 for false negatives, and 0.35 for true positives.

Key insights:

• Correct classifications dominate: The diagonal cells account for 0.36 (true negatives) and 0.35 (true positives), for a combined normalized share of 0.71.
• Error rates are lower than correct rates: The off-diagonal cells are 0.13 for false positives and 0.16 for false negatives, for a combined normalized share of 0.29.
• Class-specific outcomes are balanced: The two correct classification cells are closely matched at 0.36 and 0.35, indicating similar normalized shares of correct predictions for the negative and positive classes.
• False negatives exceed false positives slightly: The false negative cell is 0.16 versus 0.13 for false positives, a difference of 0.03 in normalized terms.

The normalized confusion matrix indicates that most observations fall into correct classification categories, with 71% of outcomes on the diagonal and 29% in misclassification cells. Correct negative and correct positive outcomes are nearly equal in share, while the misclassification pattern is also relatively balanced, with a modestly higher proportion of false negatives than false positives. Overall, the result shows broadly symmetric classification behavior across the two classes in normalized terms.

Parameters:

{
  "normalize": true
}
            

Figures

2026-10-30 21:03:44,855 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_custom_tests.ConfusionMatrix:test_normalized_challenger does not exist in model's document

Use external test providers

Sometimes you may want to reuse the same set of custom tests across multiple records (models) and share them with others in your organization, like the development team would have done with you in this example workflow featured in this series of notebooks. In this case, you can create an external custom test provider that will allow you to load custom tests from a local folder or a Git repository.

In this section you will learn how to declare a local filesystem test provider that allows loading tests from a local folder following these high level steps:

1. Create a folder of custom tests from existing inline tests (tests that exist in your active Jupyter Notebook)
2. Save an inline test to a file
3. Define and register a LocalTestProvider that points to that folder
4. Run test provider tests
5. Add the test results to your documentation

Create custom tests folder

Let's start by creating a new folder that will contain reusable custom tests from your existing inline tests.

The following code snippet will create a new my_tests directory in the current working directory if it doesn't exist:

tests_folder = "my_tests"

import os

# create tests folder
os.makedirs(tests_folder, exist_ok=True)

# remove existing tests
for f in os.listdir(tests_folder):
    # remove files and pycache
    if f.endswith(".py") or f == "__pycache__":
        os.system(f"rm -rf {tests_folder}/{f}")

After running the command above, confirm that a new my_tests directory was created successfully. For example:

~/notebooks/tutorials/validation/my_tests/

Save an inline test

The @vm.test decorator we used in Implement a custom inline test above to register one-off custom tests also includes a convenience method on the function object that allows you to simply call <func_name>.save() to save the test to a Python file at a specified path.

While save() will get you started by creating the file and saving the function code with the correct name, it won't automatically include any imports, or other functions or variables, outside of the functions that are needed for the test to run. To solve this, pass in an optional imports argument ensuring necessary imports are added to the file.

The confusion_matrix test requires the following additional imports:

import matplotlib.pyplot as plt
from sklearn import metrics

Let's pass these imports to the save() method to ensure they are included in the file with the following command:

confusion_matrix.save(
    # Save it to the custom tests folder we created
    tests_folder,
    imports=["import matplotlib.pyplot as plt", "from sklearn import metrics"],
)

2026-10-30 21:03:45,676 - INFO(validmind.tests.decorator): Saved to /home/runner/work/documentation/documentation/site/notebooks/EXECUTED/validation/my_tests/ConfusionMatrix.py!Be sure to add any necessary imports to the top of the file.
2026-10-30 21:03:45,676 - INFO(validmind.tests.decorator): This metric can be run with the ID: <test_provider_namespace>.ConfusionMatrix

• Confirm that the save() method saved the confusion_matrix function to a file named ConfusionMatrix.py in the my_tests folder.

• Note that the new file provides some context on the origin of the test, which is useful for traceability:

  # Saved from __main__.confusion_matrix
  # Original Test ID: my_custom_tests.ConfusionMatrix
  # New Test ID: <test_provider_namespace>.ConfusionMatrix

• Additionally, the new test function has been stripped off its decorator, as it now resides in a file that will be loaded by the test provider:

  def ConfusionMatrix(dataset, model, normalize=False):

Register a local test provider

Now that your my_tests folder has a sample custom test, let's initialize a test provider that will tell the ValidMind Library where to find your custom tests:

• ValidMind offers out-of-the-box test providers for local tests (tests in a folder) or a Github provider for tests in a Github repository.
• You can also create your own test provider by creating a class that has a load_test method that takes a test ID and returns the test function matching that ID.

Want to learn more about test providers?

An extended introduction to test providers can be found in: Integrate external test providers

Initialize a local test provider

For most use cases, using a LocalTestProvider that allows you to load custom tests from a designated directory should be sufficient.

The most important attribute for a test provider is its namespace. This is a string that will be used to prefix test IDs in documentation. This allows you to have multiple test providers with tests that can even share the same ID, but are distinguished by their namespace.

Let's go ahead and load the custom tests from our my_tests directory:

from validmind.tests import LocalTestProvider

# initialize the test provider with the tests folder we created earlier
my_test_provider = LocalTestProvider(tests_folder)

vm.tests.register_test_provider(
    namespace="my_test_provider",
    test_provider=my_test_provider,
)
# `my_test_provider.load_test()` will be called for any test ID that starts with `my_test_provider`
# e.g. `my_test_provider.ConfusionMatrix` will look for a function named `ConfusionMatrix` in `my_tests/ConfusionMatrix.py` file

Run test provider tests

Now that we've set up the test provider, we can run any test that's located in the tests folder by using the run_test() method as with any other test:

• For tests that reside in a test provider directory, the test ID will be the namespace specified when registering the provider, followed by the path to the test file relative to the tests folder.
• For example, the Confusion Matrix test we created earlier will have the test ID my_test_provider.ConfusionMatrix. You could organize the tests in subfolders, say classification and regression, and the test ID for the Confusion Matrix test would then be my_test_provider.classification.ConfusionMatrix.

Let's go ahead and re-run the confusion matrix test with our testing dataset for our two models by using the test ID my_test_provider.ConfusionMatrix. This should load the test from the test provider and run it as before.

# Champion with test dataset and test provider custom test
vm.tests.run_test(
    test_id="my_test_provider.ConfusionMatrix:champion",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_log_model]
    }
).log()

Confusion Matrix Champion

The Confusion Matrix test evaluates classification performance by comparing predicted labels with observed labels in a 2x2 table. For the test_dataset_final results of log_model_champion, the matrix reports counts for true negatives, false positives, false negatives, and true positives across the two classes. The displayed counts are 198 for true negatives, 117 for false positives, 127 for false negatives, and 205 for true positives, allowing direct comparison of correct and incorrect classifications by class.

Key insights:

• Correct classifications exceed errors: The model records 198 true negatives and 205 true positives, compared with 117 false positives and 127 false negatives. Both diagonal cells are larger than their corresponding off-diagonal cells.
• Positive and negative outcomes are similarly represented: Actual negative cases total 315 (198 + 117), while actual positive cases total 332 (127 + 205). Predicted negative cases total 325 (198 + 127), and predicted positive cases total 322 (117 + 205).
• Error counts are relatively balanced: False positives and false negatives are close in magnitude, at 117 and 127 respectively. This indicates that misclassification is not concentrated predominantly in one error type.
• True positives slightly exceed true negatives: The model correctly identifies 205 positive cases versus 198 negative cases. The difference between the two correct-classification counts is modest.

The confusion matrix shows that correct predictions occur more frequently than incorrect predictions in both classes, with 403 correct classifications versus 244 misclassifications overall. Class representation is similar across actual and predicted outcomes, and the two error types are of comparable size. Taken together, the result indicates a relatively balanced classification pattern without a pronounced skew toward either false positives or false negatives.

Figures

2026-10-30 21:03:54,812 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_test_provider.ConfusionMatrix:champion does not exist in model's document

# Challenger with test dataset  and test provider custom test
vm.tests.run_test(
    test_id="my_test_provider.ConfusionMatrix:challenger",
    input_grid={
        "dataset": [vm_test_ds],
        "model" : [vm_rf_model]
    }
).log()

Confusion Matrix Challenger

The Confusion Matrix test evaluates classification outcomes by comparing predicted labels against observed labels for the challenger model on the test dataset. The matrix reports counts for correct and incorrect predictions across both classes, with true negatives in the upper-left cell, false positives in the upper-right cell, false negatives in the lower-left cell, and true positives in the lower-right cell. In this result, the four cell counts are 230 true negatives, 85 false positives, 106 false negatives, and 226 true positives.

Key insights:

• Correct predictions exceed errors: The diagonal cells contain 230 true negatives and 226 true positives, for a combined 456 correct predictions, while the off-diagonal cells contain 85 false positives and 106 false negatives, for a combined 191 incorrect predictions.
• Negative class is identified more accurately: Among observations with a true label of False, 230 are correctly classified as False and 85 are incorrectly classified as True. This indicates fewer errors for the negative class than for the positive class row.
• False negatives exceed false positives: The model produces 106 false negatives compared with 85 false positives. Misclassification is therefore somewhat more concentrated in cases where true positive observations are predicted as negative.
• Class support is relatively balanced: The observed class totals are 315 for the False class (230 + 85) and 332 for the True class (106 + 226). Predicted class totals are also similar, at 336 predicted False (230 + 106) and 311 predicted True (85 + 226).

The confusion matrix shows that the challenger model classifies both classes with a comparable volume of correct predictions, with total correct predictions materially exceeding total errors. Performance differs modestly by error type, as false negatives are more frequent than false positives and the negative class has fewer misclassifications than the positive class. The underlying test sample and prediction outputs are relatively balanced across the two classes, which supports direct comparison of the error counts between positive and negative outcomes.

Figures

2026-10-30 21:04:05,885 - INFO(validmind.vm_models.result.result): Test driven block with result_id my_test_provider.ConfusionMatrix:challenger does not exist in model's document

Verify test runs

Our final task is to verify that all the tests provided by the development team were run and reported accurately. Note the appended result_ids to delineate which dataset we ran the test with for the relevant tests.

Here, we'll specify all the tests we'd like to independently rerun in a dictionary called test_config. Note here that inputs and input_grid expect the input_id of the dataset or model as the value rather than the variable name we specified:

test_config = {
    # Run with the raw dataset
    'validmind.data_validation.DatasetDescription:raw_data': {
        'inputs': {'dataset': 'raw_dataset'}
    },
    'validmind.data_validation.DescriptiveStatistics:raw_data': {
        'inputs': {'dataset': 'raw_dataset'}
    },
    'validmind.data_validation.MissingValues:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_percentage_threshold': 1}
    },
    'validmind.data_validation.ClassImbalance:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_percent_threshold': 10}
    },
    'validmind.data_validation.Duplicates:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_threshold': 1}
    },
    'validmind.data_validation.HighCardinality:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {
            'num_threshold': 100,
            'percent_threshold': 0.1,
            'threshold_type': 'percent'
        }
    },
    'validmind.data_validation.Skewness:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'max_threshold': 1}
    },
    'validmind.data_validation.UniqueRows:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'min_percent_threshold': 1}
    },
    'validmind.data_validation.TooManyZeroValues:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'max_percent_threshold': 0.03}
    },
    'validmind.data_validation.IQROutliersTable:raw_data': {
        'inputs': {'dataset': 'raw_dataset'},
        'params': {'threshold': 5}
    },
    # Run with the preprocessed dataset
    'validmind.data_validation.DescriptiveStatistics:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.TabularDescriptionTables:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.MissingValues:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'},
        'params': {'min_percentage_threshold': 1}
    },
    'validmind.data_validation.TabularNumericalHistograms:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'}
    },
    'validmind.data_validation.TargetRateBarPlots:preprocessed_data': {
        'inputs': {'dataset': 'raw_dataset_preprocessed'},
        'params': {'default_column': 'loan_status'}
    },
    # Run with the training and test datasets
    'validmind.data_validation.DescriptiveStatistics:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.TabularDescriptionTables:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.ClassImbalance:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'min_percent_threshold': 10}
    },
    'validmind.data_validation.UniqueRows:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'min_percent_threshold': 1}
    },
    'validmind.data_validation.TabularNumericalHistograms:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.MutualInformation:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'min_threshold': 0.01}
    },
    'validmind.data_validation.PearsonCorrelationMatrix:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']}
    },
    'validmind.data_validation.HighPearsonCorrelation:development_data': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final']},
        'params': {'max_threshold': 0.3, 'top_n_correlations': 10}
    },
    'validmind.model_validation.ModelMetadata': {
        'input_grid': {'model': ['log_model_champion', 'rf_model']}
    },
    'validmind.model_validation.sklearn.ModelParameters': {
        'input_grid': {'model': ['log_model_champion', 'rf_model']}
    },
    'validmind.model_validation.sklearn.ROCCurve': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final'], 'model': ['log_model_champion']}
    },
    'validmind.model_validation.sklearn.MinimumROCAUCScore': {
        'input_grid': {'dataset': ['train_dataset_final', 'test_dataset_final'], 'model': ['log_model_champion']},
        'params': {'min_threshold': 0.5}
    }
}

Then batch run and log our tests in test_config:

for t in test_config:
    print(t)
    try:
        # Check if test has input_grid
        if 'input_grid' in test_config[t]:
            # For tests with input_grid, pass the input_grid configuration
            if 'params' in test_config[t]:
                vm.tests.run_test(t, input_grid=test_config[t]['input_grid'], params=test_config[t]['params']).log()
            else:
                vm.tests.run_test(t, input_grid=test_config[t]['input_grid']).log()
        else:
            # Original logic for regular inputs
            if 'params' in test_config[t]:
                vm.tests.run_test(t, inputs=test_config[t]['inputs'], params=test_config[t]['params']).log()
            else:
                vm.tests.run_test(t, inputs=test_config[t]['inputs']).log()
    except Exception as e:
        print(f"Error running test {t}: {str(e)}")

validmind.data_validation.DatasetDescription:raw_data

Dataset Description Raw Data

The Dataset Description test evaluates the structure, completeness, and column-level characteristics of the raw dataset. The results summarize 11 fields across numeric and categorical types, reporting record counts, missingness, and distinct-value counts for each column. All listed variables have 8,000 observed records with no missing values, and the table highlights the relative cardinality of each feature through distinct-count and distinct-percentage measures.

Key insights:

• No missing values across fields: All 11 columns show a count of 8,000 and missingness of 0.0%, indicating complete population of the raw dataset for the variables reported.

• EstimatedSalary is fully unique: EstimatedSalary has 8,000 distinct values out of 8,000 records, corresponding to a distinct rate of 1.0, making it the highest-cardinality field in the dataset.

• Balance also has high cardinality: Balance contains 5,088 distinct values, representing 63.6% of records, which is materially higher than the distinctness observed in the other numeric variables aside from EstimatedSalary.

• Most categorical fields are low-cardinality: Geography has 3 distinct values, while Gender, HasCrCard, IsActiveMember, and Exited each have 2 distinct values, indicating compact categorical domains across these fields.

• Numeric variables show varied granularity: Among numeric columns, CreditScore has 452 distinct values, Age has 69, Tenure has 11, and NumOfProducts has 4, showing a mix of more continuous-like and more discretized numeric fields.

The dataset description indicates a complete raw dataset with no observed missing values in any reported field. The most pronounced structural characteristic is the contrast in feature cardinality: EstimatedSalary is fully unique and Balance is highly distinct, while the remaining variables, particularly the categorical fields and some numeric count-like fields, have relatively small value sets. Overall, the table reflects a mix of continuous-style numeric variables and low-cardinality categorical or discretized fields across the 8,000-record sample.

Tables

Dataset Description

Name	Type	Count	Missing	Missing %	Distinct	Distinct %
CreditScore	Numeric	8000.0	0	0.0	452	0.0565
Geography	Categorical	8000.0	0	0.0	3	0.0004
Gender	Categorical	8000.0	0	0.0	2	0.0002
Age	Numeric	8000.0	0	0.0	69	0.0086
Tenure	Numeric	8000.0	0	0.0	11	0.0014
Balance	Numeric	8000.0	0	0.0	5088	0.6360
NumOfProducts	Numeric	8000.0	0	0.0	4	0.0005
HasCrCard	Categorical	8000.0	0	0.0	2	0.0002
IsActiveMember	Categorical	8000.0	0	0.0	2	0.0002
EstimatedSalary	Numeric	8000.0	0	0.0	8000	1.0000
Exited	Categorical	8000.0	0	0.0	2	0.0002

2026-10-30 21:04:16,832 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DatasetDescription:raw_data does not exist in model's document

validmind.data_validation.DescriptiveStatistics:raw_data

Descriptive Statistics Raw Data

The Descriptive Statistics test evaluates the distributional characteristics of numerical and categorical variables in the raw dataset. The results provide summary statistics for eight numerical variables and frequency-based summaries for two categorical variables, each with 8,000 observed records. Numerical output includes central tendency, dispersion, and percentile values from the minimum through the 95th percentile and maximum, while categorical output reports the number of unique values, the most frequent category, and its share of total observations.

Key insights:

• Balance shows pronounced asymmetry: Balance has a mean of 76,434.10 and a median of 97,264.00, with the 25th percentile at 0.00 and the minimum also 0.00. This indicates a substantial concentration of zero-balance observations alongside a wide positive range up to 250,898.00.

• Age is moderately right-skewed: Age has a mean of 38.95 versus a median of 37.00, with upper percentiles extending to 53.00 at the 90th percentile, 60.00 at the 95th percentile, and a maximum of 92.00. The distribution therefore extends materially into older values relative to its center.

• Tenure is broadly centered: Tenure has a mean of 5.03 and a median of 5.00, with an interquartile range from 3.00 to 8.00 and a bounded range from 0.00 to 10.00. The close alignment of mean and median indicates a relatively balanced distribution within its defined limits.

• NumOfProducts is concentrated at low values: NumOfProducts has a mean of 1.53 and a median of 1.00, with the 75th percentile at 2.00, the 95th percentile also at 2.00, and a maximum of 4.00. Most observations are therefore concentrated in the lower end of the variable range.

• Binary indicators are imbalanced to different degrees: HasCrCard has a mean of 0.7026, indicating that the value 1 occurs more frequently than 0, while IsActiveMember has a mean of 0.5199, indicating a near-even split. Both variables are bounded between 0 and 1, with medians of 1.00.

• Categorical concentration is moderate: Geography contains 3 unique values, with France as the top category at 4,010 records or 50.12% of observations. Gender contains 2 unique values, with Male as the top category at 4,396 records or 54.95% of observations.

The descriptive statistics show a dataset with complete reported counts across all listed variables and differing distributional shapes across features. The most notable concentration appears in Balance, where zero values coexist with a wide upper range, while NumOfProducts is concentrated in lower categories and Tenure remains comparatively centered. The categorical variables show moderate concentration in their top categories rather than extreme dominance, and the binary indicators differ in balance, with HasCrCard more skewed than IsActiveMember.

Tables

Numerical Variables

Name	Count	Mean	Std	Min	25%	50%	75%	90%	95%	Max
CreditScore	8000.0	650.1596	96.8462	350.0	583.0	652.0	717.0	778.0	813.0	850.0
Age	8000.0	38.9489	10.4590	18.0	32.0	37.0	44.0	53.0	60.0	92.0
Tenure	8000.0	5.0339	2.8853	0.0	3.0	5.0	8.0	9.0	9.0	10.0
Balance	8000.0	76434.0965	62612.2513	0.0	0.0	97264.0	128045.0	149545.0	162488.0	250898.0
NumOfProducts	8000.0	1.5325	0.5805	1.0	1.0	1.0	2.0	2.0	2.0	4.0
HasCrCard	8000.0	0.7026	0.4571	0.0	0.0	1.0	1.0	1.0	1.0	1.0
IsActiveMember	8000.0	0.5199	0.4996	0.0	0.0	1.0	1.0	1.0	1.0	1.0
EstimatedSalary	8000.0	99790.1880	57520.5089	12.0	50857.0	99505.0	149216.0	179486.0	189997.0	199992.0

Categorical Variables

Name	Count	Number of Unique Values	Top Value	Top Value Frequency	Top Value Frequency %
Geography	8000.0	3.0	France	4010.0	50.12
Gender	8000.0	2.0	Male	4396.0	54.95

2026-10-30 21:04:29,877 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DescriptiveStatistics:raw_data does not exist in model's document

validmind.data_validation.MissingValues:raw_data

✅ Missing Values Raw Data

The Missing Values test evaluates dataset completeness by measuring the proportion of missing values in each feature against the configured 1% threshold. The result table reports the number and percentage of missing values for each column in the raw dataset, together with a pass/fail outcome. Across all 11 listed columns, the observed number of missing values is 0 and the percentage of missing values is 0.0%, resulting in a passing status for every feature.

Key insights:

• No missing values detected: All 11 columns show 0 missing values and 0.0% missingness in the raw dataset.
• Universal pass outcome: Every feature passes the test under the configured 1% missing-value threshold, with no column exceeding the limit.
• Completeness is consistent across features: The result is uniform across all listed variables, including input features such as CreditScore, Geography, Age, and Balance, as well as the target variable Exited.

The test results show complete observed data coverage across all evaluated columns in the raw dataset. Missingness does not vary by feature, and no threshold breaches are present under the 1% criterion. Collectively, the results indicate that this dataset passed the missing-values assessment without any feature-level exceptions.

Parameters:

{
  "min_percentage_threshold": 1
}
            

Tables

Column	Number of Missing Values	Percentage of Missing Values (%)	Pass/Fail
CreditScore	0	0.0	Pass
Geography	0	0.0	Pass
Gender	0	0.0	Pass
Age	0	0.0	Pass
Tenure	0	0.0	Pass
Balance	0	0.0	Pass
NumOfProducts	0	0.0	Pass
HasCrCard	0	0.0	Pass
IsActiveMember	0	0.0	Pass
EstimatedSalary	0	0.0	Pass
Exited	0	0.0	Pass

2026-10-30 21:04:36,358 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.MissingValues:raw_data does not exist in model's document

validmind.data_validation.ClassImbalance:raw_data

✅ Class Imbalance Raw Data

The Class Imbalance test evaluates the distribution of target classes in the dataset by measuring each class’s share of total records against the configured minimum threshold of 10%. The results for the Exited target show two classes, with class 0 representing 79.80% of rows and class 1 representing 20.20% of rows. The accompanying table and bar chart present the relative frequency of each class and the pass/fail outcome for the threshold check.

Key insights:

• Both classes exceed threshold: Class 0 at 79.80% and class 1 at 20.20% both exceed the 10% minimum threshold, and both are marked as passing.
• Majority class dominates distribution: The target distribution is concentrated in class 0, which accounts for nearly four times the share of class 1.
• Minority class remains materially represented: Although class 1 is the smaller class, its 20.20% share remains above the configured imbalance cutoff used in this test.

The observed class distribution is uneven, with a clear majority in class 0 and a smaller but still substantial representation in class 1. Under the configured 10% threshold, neither class is flagged as under-represented, and the test outcome indicates that both classes satisfy the defined minimum frequency criterion.

Parameters:

{
  "min_percent_threshold": 10
}
            

Tables

Exited Class Imbalance

Exited	Percentage of Rows (%)	Pass/Fail
0	79.80%	Pass
1	20.20%	Pass

Figures

2026-10-30 21:04:44,397 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.ClassImbalance:raw_data does not exist in model's document

validmind.data_validation.Duplicates:raw_data

✅ Duplicates Raw Data

The Duplicates test evaluates whether the dataset contains repeated rows that could affect data quality and training behavior. The result table reports the number of duplicate rows and their share of the dataset. For the evaluated raw dataset, the table shows 0 duplicate rows, corresponding to 0.0% of all rows.

Key insights:

• No duplicate rows detected: The test identified 0 duplicate entries in the raw dataset, indicating that no exact row repetitions were found in the evaluated data.
• Duplicate share is zero: Duplicate rows account for 0.0% of the dataset, showing that repeated records were absent in proportional as well as absolute terms.
• Result is below threshold: The reported duplicate count of 0 is below the configured minimum threshold parameter of 1.

The duplicate row assessment shows no exact duplication in the raw dataset. Both the absolute count and percentage of duplicate rows are zero, indicating that this data quality check did not identify repeated records in the evaluated sample.

Parameters:

{
  "min_threshold": 1
}
            

Tables

Duplicate Rows Results for Dataset

Number of Duplicates	Percentage of Rows (%)
0	0.0

2026-10-30 21:04:49,475 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.Duplicates:raw_data does not exist in model's document

validmind.data_validation.HighCardinality:raw_data

✅ High Cardinality Raw Data

The High Cardinality test evaluates the number of unique values in categorical columns to identify columns with large numbers of distinct categories. The results table reports, for each categorical column assessed, the number of distinct values, the percentage of distinct values, and the corresponding pass/fail outcome against the configured threshold. In this run, two categorical columns were evaluated: Geography with 3 distinct values and Gender with 2 distinct values. Both columns are recorded as passing the test.

Key insights:

• No categorical columns failed: Both evaluated categorical features, Geography and Gender, have a Pass result in the test output, indicating that neither exceeded the configured high-cardinality threshold used in this run.
• Distinct counts are low: Geography contains 3 distinct values and Gender contains 2 distinct values, reflecting a limited set of categories across the evaluated categorical inputs.
• Distinct-value percentages remain small: The reported percentage of distinct values is 0.0375 for Geography and 0.025 for Gender, showing low distinct-value concentration for both assessed columns.

The results show that all categorical columns evaluated in this test remained below the high-cardinality threshold. The assessed categorical feature set is limited to two columns, and both exhibit low distinct counts and low distinct-value percentages. Taken together, the output indicates no high-cardinality findings within the categorical columns included in this test run.

Parameters:

{
  "num_threshold": 100,
  "percent_threshold": 0.1,
  "threshold_type": "percent"
}
            

Tables

Column	Number of Distinct Values	Percentage of Distinct Values (%)	Pass/Fail
Geography	3	0.0375	Pass
Gender	2	0.0250	Pass

2026-10-30 21:04:55,976 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.HighCardinality:raw_data does not exist in model's document

validmind.data_validation.Skewness:raw_data

❌ Skewness Raw Data

The Skewness test evaluates the asymmetry of numerical feature distributions by calculating skewness for each numeric column and comparing the result against the maximum threshold of 1. The results table reports skewness values and pass/fail outcomes for nine numeric columns in the raw dataset. Observed skewness values range from -0.8867 to 1.4847, with two columns exceeding the threshold and seven remaining within it.

Key insights:

• Two columns exceed threshold: Age has skewness of 1.0245 and Exited has skewness of 1.4847, and both are marked as failing the test against the threshold of 1.
• Most numeric columns pass: Seven of the nine evaluated columns are below the maximum threshold, including CreditScore (-0.062), Tenure (0.0077), Balance (-0.1353), NumOfProducts (0.7172), HasCrCard (-0.8867), IsActiveMember (-0.0796), and EstimatedSalary (0.0095).
• Exited shows the highest skewness: Exited records the largest absolute skewness value at 1.4847, making it the most asymmetric distribution among the evaluated numeric columns.
• Several variables are near symmetric: CreditScore, Tenure, Balance, IsActiveMember, and EstimatedSalary all have skewness values close to zero, indicating limited distributional asymmetry in those fields.

The test results show that skewness is limited for most evaluated numeric columns, with seven variables remaining within the defined threshold. The only threshold breaches occur in Age and Exited, with Exited exhibiting the strongest asymmetry in the dataset. Overall, the distributional profile is largely balanced across the numeric inputs included in the test, with isolated exceptions captured by the failing columns.

Parameters:

{
  "max_threshold": 1
}
            

Tables

Skewness Results for Dataset

Column	Skewness	Pass/Fail
CreditScore	-0.0620	Pass
Age	1.0245	Fail
Tenure	0.0077	Pass
Balance	-0.1353	Pass
NumOfProducts	0.7172	Pass
HasCrCard	-0.8867	Pass
IsActiveMember	-0.0796	Pass
EstimatedSalary	0.0095	Pass
Exited	1.4847	Fail

2026-10-30 21:05:04,487 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.Skewness:raw_data does not exist in model's document

validmind.data_validation.UniqueRows:raw_data

❌ Unique Rows Raw Data

The UniqueRows test evaluates dataset diversity by comparing the percentage of unique values in each column against the configured minimum threshold of 1%. The result table reports the number and percentage of unique values for each field and indicates whether each column passed or failed that threshold. In this run, 3 of 11 columns passed the test and 8 failed, with reported uniqueness percentages ranging from 0.025% to 100.0%.

Key insights:

• Uniqueness is concentrated in three columns: EstimatedSalary, Balance, and CreditScore are the only columns that pass the 1% threshold, with uniqueness percentages of 100.0%, 63.6%, and 5.65%, respectively.
• EstimatedSalary is fully unique: EstimatedSalary contains 8,000 unique values and a reported uniqueness rate of 100.0%, making it the most distinct column in the dataset.
• Balance shows substantial variation: Balance records 5,088 unique values, corresponding to 63.6% uniqueness, which is the second-highest value in the result set.
• Most fields fall below threshold: Geography, Gender, Age, Tenure, NumOfProducts, HasCrCard, IsActiveMember, and Exited all fail the test, with reported uniqueness percentages between 0.025% and 0.8625%.
• Several columns have very low cardinality: Gender, HasCrCard, IsActiveMember, and Exited each have 2 unique values, while Geography has 3 and NumOfProducts has 4, producing uniqueness percentages well below the configured threshold.

The results show that uniqueness within the dataset is unevenly distributed across columns. A small subset of fields exhibits moderate to complete distinctness, while most columns have uniqueness percentages below the 1% threshold and are therefore marked as failing under this test configuration. Overall, the observed diversity is driven primarily by EstimatedSalary, Balance, and CreditScore, with the remaining variables contributing limited uniqueness by this measure.

Parameters:

{
  "min_percent_threshold": 1
}
            

Tables

Column	Number of Unique Values	Percentage of Unique Values (%)	Pass/Fail
CreditScore	452	5.6500	Pass
Geography	3	0.0375	Fail
Gender	2	0.0250	Fail
Age	69	0.8625	Fail
Tenure	11	0.1375	Fail
Balance	5088	63.6000	Pass
NumOfProducts	4	0.0500	Fail
HasCrCard	2	0.0250	Fail
IsActiveMember	2	0.0250	Fail
EstimatedSalary	8000	100.0000	Pass
Exited	2	0.0250	Fail

2026-10-30 21:05:14,136 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.UniqueRows:raw_data does not exist in model's document

validmind.data_validation.TooManyZeroValues:raw_data

❌ Too Many Zero Values Raw Data

The TooManyZeroValues test evaluates numerical columns for zero-value prevalence above a defined threshold of 0.03%. The result table reports the row count, number of zero values, percentage of zero values, and pass/fail status for each assessed numerical variable in the raw data. Four variables are listed in the output, and each exceeds the configured threshold, resulting in a fail status for all reported columns. Zero-value percentages range from 4.0375% to 48.0125% across the variables shown.

Key insights:

• All assessed variables failed: Each of the four reported numerical variables exceeded the 0.03% threshold and received a fail status, indicating that zero values are present above the configured tolerance in every assessed column.

• IsActiveMember has the highest zero concentration: IsActiveMember contains 3,841 zero values out of 8,000 rows, corresponding to 48.0125%, which is the highest zero-value share among the reported variables.

• Balance and HasCrCard also show elevated zeros: Balance records 2,912 zero values (36.4%), and HasCrCard records 2,379 zero values (29.7375%), placing both variables well above the test threshold.

• Tenure has the lowest reported zero rate: Tenure contains 323 zero values out of 8,000 rows, equal to 4.0375%, which is the smallest zero-value percentage in the result set but still above the configured threshold.

The test results show that all reported numerical variables exceeded the maximum zero-value threshold of 0.03%, with substantial variation in magnitude across columns. The most concentrated zero presence appears in IsActiveMember, followed by Balance and HasCrCard, while Tenure has the lowest zero rate among the reported variables. Collectively, the output indicates that zero values are a pervasive characteristic of the assessed raw-data fields included in this test result.

Parameters:

{
  "max_percent_threshold": 0.03
}
            

Tables

Variable	Row Count	Number of Zero Values	Percentage of Zero Values (%)	Pass/Fail
Tenure	8000	323	4.0375	Fail
Balance	8000	2912	36.4000	Fail
HasCrCard	8000	2379	29.7375	Fail
IsActiveMember	8000	3841	48.0125	Fail

2026-10-30 21:05:23,867 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TooManyZeroValues:raw_data does not exist in model's document

validmind.data_validation.IQROutliersTable:raw_data

IQR Outliers Table Raw Data

The Interquartile Range Outliers Table test evaluates numerical features for observations falling outside the IQR-based outlier bounds. The result table titled Summary of Outliers Detected by IQR Method is empty, indicating that no outlier summary rows were returned in this test output. As presented, the output does not list any numerical features, outlier counts, or outlier summary statistics.

Key insights:

• No outlier rows returned: The outlier summary table contains no data rows, so the test output does not report any detected outliers for numerical features.
• No feature-level detail available: Because the table is empty, there are no reported counts or distributional summaries for individual numerical variables.
• Threshold parameter recorded as 5: The test was executed with an outlier threshold parameter of 5, as shown in the test parameters.

The reported result consists of an empty IQR outlier summary table under a threshold parameter of 5. Based on the provided output, the test result does not include feature-level evidence of detected outliers or associated summary statistics.

Parameters:

{
  "threshold": 5
}
            

Tables

Summary of Outliers Detected by IQR Method

2026-10-30 21:05:29,113 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.IQROutliersTable:raw_data does not exist in model's document

validmind.data_validation.DescriptiveStatistics:preprocessed_data

Descriptive Statistics Preprocessed Data

The Descriptive Statistics test evaluates the distributional characteristics of numerical and categorical variables in the preprocessed dataset. The results present summary statistics for seven numerical variables and frequency-based summaries for two categorical variables, each with 3,232 observations. The numerical table reports central tendency, dispersion, and percentile values, while the categorical table shows the number of unique values, the most frequent category, and its relative share. Together, these tables provide a compact view of the feature distributions represented in the dataset.

Key insights:

• Complete coverage across reported variables: All listed numerical and categorical variables have a count of 3,232, indicating that the reported fields are populated uniformly in this dataset extract.

• Balance shows strong concentration at zero: Balance has a minimum of 0 and a 25th percentile of 0, while the median is 102,731 and the mean is 81,081.98. This gap between the lower quartile and the median indicates a substantial concentration of zero-balance observations alongside a broad positive-value range up to 250,898.

• CreditScore is centered near its median: CreditScore has a mean of 648.17 and a median of 650, with an interquartile range from 579 to 717 and an observed range from 350 to 850. The close alignment of mean and median indicates limited asymmetry in the central portion of the distribution.

• NumOfProducts is concentrated at lower counts: NumOfProducts has a median of 1, a 75th percentile of 2, a 95th percentile of 3, and a maximum of 4. The distribution is therefore concentrated in the lower product-count values.

• Binary indicators are unevenly distributed: HasCrCard has a mean of 0.6949, indicating that the value 1 occurs more frequently than 0, while IsActiveMember has a mean of 0.4623 and a median of 0, indicating a slightly higher frequency of 0 than 1.

• Categorical concentrations are moderate: Geography contains 3 unique values, with France as the top category at 1,483 observations (45.88%). Gender contains 2 unique values, with Male as the top category at 1,635 observations (50.59%), indicating near-even representation across the two gender categories.

The results show that the reported variables are complete at the observation level and span a mix of continuous, count-like, binary, and categorical fields. The most prominent distributional feature is Balance, where zero values occupy at least the lower quartile despite a substantially higher median and a wide upper range. Other variables are more compactly distributed, including CreditScore and Tenure, while NumOfProducts is concentrated at lower counts and the categorical variables show moderate concentration rather than dominance by a single class.

Tables

Numerical Variables

Name	Count	Mean	Std	Min	25%	50%	75%	90%	95%	Max
CreditScore	3232.0	648.1680	98.8311	350.0	579.0	650.0	717.0	779.0	816.0	850.0
Tenure	3232.0	4.9629	2.8965	0.0	2.0	5.0	8.0	9.0	9.0	10.0
Balance	3232.0	81081.9781	61677.3574	0.0	0.0	102731.0	129193.0	149634.0	164484.0	250898.0
NumOfProducts	3232.0	1.5241	0.6690	1.0	1.0	1.0	2.0	2.0	3.0	4.0
HasCrCard	3232.0	0.6949	0.4605	0.0	0.0	1.0	1.0	1.0	1.0	1.0
IsActiveMember	3232.0	0.4623	0.4987	0.0	0.0	0.0	1.0	1.0	1.0	1.0
EstimatedSalary	3232.0	99446.1855	58046.0844	12.0	50202.0	96711.0	149744.0	180599.0	190003.0	199929.0

Categorical Variables

Name	Count	Number of Unique Values	Top Value	Top Value Frequency	Top Value Frequency %
Geography	3232.0	3.0	France	1483.0	45.88
Gender	3232.0	2.0	Male	1635.0	50.59

2026-10-30 21:05:51,341 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DescriptiveStatistics:preprocessed_data does not exist in model's document

validmind.data_validation.TabularDescriptionTables:preprocessed_data

Tabular Description Tables Preprocessed Data

The TabularDescriptionTables:preprocessed_data test summarizes descriptive statistics for numerical and categorical variables in the preprocessed dataset. The result shows eight numerical variables and two categorical variables, with counts, central tendency, observed ranges, missing-value percentages, and data types reported for each field. All listed variables have 3,232 observed records, and the categorical summary also reports the number of unique values and the distinct categories present for each categorical field.

Key insights:

• No missing values observed: Missing Values (%) is 0.0 for all eight numerical variables and both categorical variables, indicating complete coverage across the summarized fields.

• Consistent record counts across fields: Each variable shows 3,232 observations, with no variation in reported counts between numerical and categorical summaries.

• Target variable is evenly split: Exited has a mean of 0.5 with a minimum of 0 and maximum of 1, indicating an equal proportion of the two observed outcome values in the summarized data.

• Binary indicators are numerically encoded: HasCrCard, IsActiveMember, and Exited are stored as int64 and each has a value range from 0 to 1, consistent with binary-encoded fields in the table.

• Categorical fields have low cardinality: Geography contains 3 unique values (France, Germany, Spain) and Gender contains 2 unique values (Male, Female), with both fields stored as object.

The descriptive summary indicates a complete preprocessed dataset with no reported missingness and uniform observation counts across all summarized variables. Numerical fields span a mix of bounded integer variables, binary indicators, and continuous float variables, with reported ranges such as CreditScore from 350 to 850 and EstimatedSalary from 11.58 to 199,929.17. The categorical structure is limited to two low-cardinality fields, while the outcome field Exited is balanced in the summarized sample.

Tables

Numerical Variable	Num of Obs	Mean	Min	Max	Missing Values (%)	Data Type
CreditScore	3232	648.1680	350.00	850.00	0.0	int64
Tenure	3232	4.9629	0.00	10.00	0.0	int64
Balance	3232	81081.9781	0.00	250898.09	0.0	float64
NumOfProducts	3232	1.5241	1.00	4.00	0.0	int64
HasCrCard	3232	0.6949	0.00	1.00	0.0	int64
IsActiveMember	3232	0.4623	0.00	1.00	0.0	int64
EstimatedSalary	3232	99446.1855	11.58	199929.17	0.0	float64
Exited	3232	0.5000	0.00	1.00	0.0	int64

Categorical Variable	Num of Obs	Num of Unique Values	Unique Values	Missing Values (%)	Data Type
Geography	3232.0	3.0	['France' 'Germany' 'Spain']	0.0	object
Gender	3232.0	2.0	['Male' 'Female']	0.0	object

2026-10-30 21:06:01,182 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularDescriptionTables:preprocessed_data does not exist in model's document

validmind.data_validation.MissingValues:preprocessed_data

✅ Missing Values Preprocessed Data

The Missing Values test evaluates dataset completeness by measuring the percentage of missing values in each feature against the configured 1% threshold. The result table reports the number and percentage of missing values for each column in the preprocessed dataset, along with a pass/fail outcome. Across the 10 evaluated columns, all features show 0 missing values and 0.0% missingness, and each column is marked as passing the test.

Key insights:

• No missing values detected: All 10 evaluated columns have 0 missing values, corresponding to 0.0% missingness in every case.
• Universal threshold compliance: Every feature passes the test against the 1% missing-value threshold, with no column approaching or exceeding the limit.
• Completeness is consistent across predictors and target: The reported zero-missingness result applies uniformly to all listed fields, including predictor variables such as CreditScore, Geography, and EstimatedSalary, as well as the target variable Exited.

The test result indicates complete observed data coverage across the evaluated preprocessed dataset. Missingness is absent in every reported column, and no localized or feature-specific completeness issues appear in the output. Based on this test, dataset completeness does not present an identified issue within the fields assessed.

Parameters:

{
  "min_percentage_threshold": 1
}
            

Tables

Column	Number of Missing Values	Percentage of Missing Values (%)	Pass/Fail
CreditScore	0	0.0	Pass
Geography	0	0.0	Pass
Gender	0	0.0	Pass
Tenure	0	0.0	Pass
Balance	0	0.0	Pass
NumOfProducts	0	0.0	Pass
HasCrCard	0	0.0	Pass
IsActiveMember	0	0.0	Pass
EstimatedSalary	0	0.0	Pass
Exited	0	0.0	Pass

2026-10-30 21:06:09,414 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.MissingValues:preprocessed_data does not exist in model's document

validmind.data_validation.TabularNumericalHistograms:preprocessed_data

Tabular Numerical Histograms Preprocessed Data

The TabularNumericalHistograms test evaluates the univariate distributions of numerical features by plotting 50-bin histograms for each variable in the preprocessed dataset. The results display histograms for CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, and EstimatedSalary. These plots show the spread, concentration, and support of each feature, including continuous variables, integer-valued variables, and binary indicators.

Key insights:

• CreditScore is broadly bell-shaped: CreditScore values span roughly from the mid-300s to the mid-800s, with the highest concentration centered around the 600-700 range and lower frequencies at both tails.

• Tenure is distributed across discrete values: Tenure appears on integer values from 0 to 10, with relatively similar bar heights across most categories and visibly lower counts at the endpoints 0 and 10.

• Balance shows a strong zero mass: Balance has a pronounced spike at 0 that is substantially larger than any other bin, alongside a separate broad concentration of nonzero balances centered approximately around 100k-150k.

• NumOfProducts is concentrated at low counts: NumOfProducts takes discrete values from 1 to 4, with the highest frequency at 1, a lower but still substantial frequency at 2, and much smaller counts at 3 and especially 4.

• Binary indicators are imbalanced: HasCrCard and IsActiveMember each contain only 0 and 1 values, with HasCrCard skewed toward 1 and IsActiveMember showing a milder imbalance between the two classes.

• EstimatedSalary is approximately uniform: EstimatedSalary is spread across the full range from near 0 to about 200k with comparatively even bar heights and no dominant central peak.

Overall, the histograms indicate that the preprocessed numerical inputs include a mix of continuous, discrete, and binary distributions with materially different shapes across variables. The most prominent structural features are the large point mass at zero for Balance, the low-count concentration in NumOfProducts, and the class imbalance in the binary indicators. CreditScore and EstimatedSalary exhibit broader continuous coverage, while Tenure remains evenly represented across its discrete support with modest endpoint reduction.

Figures

2026-10-30 21:06:31,859 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularNumericalHistograms:preprocessed_data does not exist in model's document

validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data

Tabular Categorical Bar Plots Preprocessed Data

The TabularCategoricalBarPlots test evaluates the composition of categorical features by displaying the count of observations in each category. The results consist of bar plots for the Geography and Gender variables in the preprocessed dataset. The Geography plot shows three categories—France, Germany, and Spain—with visibly different category counts, while the Gender plot shows two categories—Male and Female—with closely similar counts. These plots provide a direct view of categorical representation across the included features.

Key insights:

• Geography distribution is uneven: France has the highest count, Germany is lower, and Spain has the smallest count. The relative separation between the three bars indicates a materially different representation across geography categories.
• Gender distribution is nearly balanced: The Male and Female bars are very close in height, indicating similar category counts with no pronounced imbalance visible in this feature.
• Category cardinality remains low: Geography contains three categories and Gender contains two categories. The plotted features therefore show limited categorical cardinality in the displayed preprocessed data.

The categorical composition shown in the test result is characterized by an uneven distribution for Geography and a near-balanced distribution for Gender. The observed category counts do not indicate high cardinality in the displayed categorical features, and the most notable variation in representation is concentrated in the Geography variable.

Figures

2026-10-30 21:06:44,472 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularCategoricalBarPlots:preprocessed_data does not exist in model's document

validmind.data_validation.TargetRateBarPlots:preprocessed_data

Target Rate Bar Plots Preprocessed Data

The TargetRateBarPlots test evaluates categorical feature distributions and their associated target rates for a classification dataset. The results are shown as paired bar plots for each categorical variable, with one plot displaying category counts and the other displaying mean target rate by category. In the provided output, these plots are shown for Geography and Gender, allowing direct comparison of category prevalence and observed target rate across each feature’s levels.

Key insights:

• Geography shows uneven category representation: France has the highest count at approximately 1,500 observations, followed by Germany at about 1,000 and Spain at about 750, indicating materially different sample sizes across geographic categories.

• Germany has the highest geographic target rate: The target rate for Germany is approximately 0.64, compared with about 0.42 for France and about 0.45 for Spain, making Germany the clear high-rate category within Geography.

• Gender counts are nearly balanced: Male and Female categories each contain roughly 1,600 observations, with only a small difference in frequency between the two groups.

• Female category has higher target rate: The target rate for Female is approximately 0.55 versus about 0.44 for Male, indicating a visible separation in observed target rates by gender.

The results show that both categorical features exhibit measurable differences in target rate across their categories. Geography combines uneven category frequency with a pronounced elevation in target rate for Germany, while Gender shows relatively balanced representation alongside a moderate target-rate gap between Female and Male. These plots therefore highlight category-level variation in the observed positive-class rate across the categorical inputs included in the output.

Figures

2026-10-30 21:06:57,556 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TargetRateBarPlots:preprocessed_data does not exist in model's document

validmind.data_validation.DescriptiveStatistics:development_data

Descriptive Statistics Development Data

The Descriptive Statistics test evaluates the distributional characteristics of the dataset by summarizing central tendency, dispersion, and percentile structure for each numerical variable. The results are reported separately for train_dataset_final and test_dataset_final across seven variables: CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, and EstimatedSalary. For each variable, the output provides count, mean, standard deviation, minimum, selected percentiles, and maximum, allowing direct comparison of the training and test samples. Counts are 2,585 observations in the training dataset and 647 observations in the test dataset for every reported variable.

Key insights:

• Train and test summaries are broadly aligned: Most variables show similar medians and interquartile ranges across the two datasets. For example, Tenure has a median of 5 in both samples, NumOfProducts has a median of 1 in both samples, and EstimatedSalary medians are 96,929 in train and 95,501 in test.

• Balance shows concentration at zero with wide spread: Balance has a 25th percentile of 0 in both datasets, while medians are substantially higher at 101,610 in train and 105,702 in test. Standard deviations are also large relative to the means, at 61,914 in train and 60,604 in test, indicating substantial dispersion.

• CreditScore remains centered in the mid-600s: CreditScore means are 647.0839 in train and 652.4992 in test, with medians of 651 and 645 respectively. Dispersion is comparable across samples, with standard deviations of 99.5838 and 95.7175, and both datasets reach the same maximum of 850.

• NumOfProducts is concentrated at lower values: The distribution is centered at 1 product in both datasets, with the 75th percentile at 2 and the 95th percentile at 3. Means are 1.5304 in train and 1.4992 in test, and the maximum observed value is 4 in both samples.

• Binary indicators show moderate class imbalance: HasCrCard has means of 0.6851 in train and 0.7342 in test, indicating a higher share of positive values in both datasets. IsActiveMember is closer to balanced, with means of 0.4592 in train and 0.4745 in test, while its median remains 0 in both samples.

• EstimatedSalary spans a wide range: EstimatedSalary ranges from 12 to 199,929 in train and from 123 to 199,454 in test. The spread is broad in both datasets, with standard deviations of 58,159.9587 and 57,627.4038, while central values remain similar across the two samples.

The descriptive statistics indicate that the training and test datasets have closely comparable numerical distributions across the reported variables. The most pronounced structural feature is the Balance distribution, where a zero-valued lower quartile coexists with much higher medians and broad dispersion. Other variables, including CreditScore, Tenure, NumOfProducts, and EstimatedSalary, show similar centers and spreads across samples, while the binary indicators differ modestly in their positive-class proportions.

Tables

dataset	Name	Count	Mean	Std	Min	25%	50%	75%	90%	95%	Max
train_dataset_final	CreditScore	2585.0	647.0839	99.5838	350.0	576.0	651.0	716.0	778.0	814.0	850.0
train_dataset_final	Tenure	2585.0	4.9431	2.8859	0.0	2.0	5.0	8.0	9.0	9.0	10.0
train_dataset_final	Balance	2585.0	80083.5093	61914.4319	0.0	0.0	101610.0	129022.0	149146.0	164756.0	250898.0
train_dataset_final	NumOfProducts	2585.0	1.5304	0.6734	1.0	1.0	1.0	2.0	2.0	3.0	4.0
train_dataset_final	HasCrCard	2585.0	0.6851	0.4646	0.0	0.0	1.0	1.0	1.0	1.0	1.0
train_dataset_final	IsActiveMember	2585.0	0.4592	0.4984	0.0	0.0	0.0	1.0	1.0	1.0	1.0
train_dataset_final	EstimatedSalary	2585.0	99635.2396	58159.9587	12.0	50664.0	96929.0	149914.0	180251.0	189592.0	199929.0
test_dataset_final	CreditScore	647.0	652.4992	95.7175	382.0	586.0	645.0	722.0	788.0	824.0	850.0
test_dataset_final	Tenure	647.0	5.0417	2.9395	0.0	3.0	5.0	8.0	9.0	10.0	10.0
test_dataset_final	Balance	647.0	85071.2238	60604.0859	0.0	0.0	105702.0	129575.0	150161.0	161119.0	222268.0
test_dataset_final	NumOfProducts	647.0	1.4992	0.6510	1.0	1.0	1.0	2.0	2.0	3.0	4.0
test_dataset_final	HasCrCard	647.0	0.7342	0.4421	0.0	0.0	1.0	1.0	1.0	1.0	1.0
test_dataset_final	IsActiveMember	647.0	0.4745	0.4997	0.0	0.0	0.0	1.0	1.0	1.0	1.0
test_dataset_final	EstimatedSalary	647.0	98690.8456	57627.4038	123.0	48913.0	95501.0	147136.0	180812.0	190860.0	199454.0

2026-10-30 21:07:12,930 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.DescriptiveStatistics:development_data does not exist in model's document

validmind.data_validation.TabularDescriptionTables:development_data

Tabular Description Tables Development Data

The Descriptive Statistics test evaluates the composition, completeness, and basic distributional properties of variables in the development data. The results summarize numerical and categorical fields for both train_dataset_final and test_dataset_final, including observation counts, means, minimum and maximum values, missingness, unique values, and data types. The development sample contains 2,585 training observations and 647 test observations across eight numerical variables and three categorical indicator variables. All reported fields show their recorded ranges and encodings directly in the summary tables.

Key insights:

• No missing values reported: Missing values are 0.0% for every numerical and categorical variable in both train_dataset_final and test_dataset_final, indicating complete populated fields in the reported development datasets.

• Train and test structures are aligned: Both datasets contain the same eight numerical variables (CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary, Exited) and the same three categorical variables (Geography_Germany, Geography_Spain, Gender_Male), with matching reported data types across splits.

• Binary fields are consistently encoded: HasCrCard, IsActiveMember, and Exited are stored as int64 with values ranging from 0 to 1, while the categorical indicator variables are stored as bool with two unique values, shown as False and True in both datasets.

• Numerical ranges are bounded and explicit: Reported ranges include CreditScore from 350 to 850 in training and 382 to 850 in test, Tenure from 0 to 10 in both datasets, NumOfProducts from 1 to 4 in both datasets, Balance from 0.0 to 250,898.09 in training and 222,267.63 in test, and EstimatedSalary from 11.58 to 199,929.17 in training and 123.07 to 199,454.37 in test.

• Central tendencies are similar across splits: Means are close between training and test for several variables, including CreditScore (647.0839 vs. 652.4992), Tenure (4.9431 vs. 5.0417), NumOfProducts (1.5304 vs. 1.4992), IsActiveMember (0.4592 vs. 0.4745), EstimatedSalary (99,635.2396 vs. 98,690.8456), and Exited (0.4967 vs. 0.5131).

The descriptive statistics show a development dataset with complete records, consistent feature structure across training and test splits, and clearly defined numerical and binary encodings. Variable ranges and means are reported explicitly for each split, with broadly similar central tendencies across most fields. The result provides a concise profile of the data used in development without indicating missing-data issues or schema inconsistencies in the summarized variables.

Tables

dataset	Numerical Variable	Num of Obs	Mean	Min	Max	Missing Values (%)	Data Type
train_dataset_final	CreditScore	2585	647.0839	350.00	850.00	0.0	int64
train_dataset_final	Tenure	2585	4.9431	0.00	10.00	0.0	int64
train_dataset_final	Balance	2585	80083.5093	0.00	250898.09	0.0	float64
train_dataset_final	NumOfProducts	2585	1.5304	1.00	4.00	0.0	int64
train_dataset_final	HasCrCard	2585	0.6851	0.00	1.00	0.0	int64
train_dataset_final	IsActiveMember	2585	0.4592	0.00	1.00	0.0	int64
train_dataset_final	EstimatedSalary	2585	99635.2396	11.58	199929.17	0.0	float64
train_dataset_final	Exited	2585	0.4967	0.00	1.00	0.0	int64
test_dataset_final	CreditScore	647	652.4992	382.00	850.00	0.0	int64
test_dataset_final	Tenure	647	5.0417	0.00	10.00	0.0	int64
test_dataset_final	Balance	647	85071.2238	0.00	222267.63	0.0	float64
test_dataset_final	NumOfProducts	647	1.4992	1.00	4.00	0.0	int64
test_dataset_final	HasCrCard	647	0.7342	0.00	1.00	0.0	int64
test_dataset_final	IsActiveMember	647	0.4745	0.00	1.00	0.0	int64
test_dataset_final	EstimatedSalary	647	98690.8456	123.07	199454.37	0.0	float64
test_dataset_final	Exited	647	0.5131	0.00	1.00	0.0	int64

dataset	Categorical Variable	Num of Obs	Num of Unique Values	Unique Values	Missing Values (%)	Data Type
train_dataset_final	Geography_Germany	2585.0	2.0	[False True]	0.0	bool
train_dataset_final	Geography_Spain	2585.0	2.0	[False True]	0.0	bool
train_dataset_final	Gender_Male	2585.0	2.0	[False True]	0.0	bool
test_dataset_final	Geography_Germany	647.0	2.0	[False True]	0.0	bool
test_dataset_final	Geography_Spain	647.0	2.0	[False True]	0.0	bool
test_dataset_final	Gender_Male	647.0	2.0	[ True False]	0.0	bool

2026-10-30 21:07:25,859 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularDescriptionTables:development_data does not exist in model's document

validmind.data_validation.ClassImbalance:development_data

✅ Class Imbalance Development Data

The Class Imbalance test evaluates the distribution of target classes in the development data by measuring the percentage share of each class and comparing it with the configured minimum threshold of 10%. Results are reported separately for train_dataset_final and test_dataset_final for the Exited target. In the training dataset, class 0 represents 50.33% of rows and class 1 represents 49.67%. In the test dataset, class 1 represents 51.31% of rows and class 0 represents 48.69%, and all class-level checks are marked as Pass.

Key insights:

• Near-even class distribution: The Exited target is distributed almost evenly in both datasets, with a 50.33%/49.67% split in train_dataset_final and a 51.31%/48.69% split in test_dataset_final.

• All classes exceed threshold: Every observed class proportion is well above the 10% minimum threshold, and each class-level result is recorded as Pass.

• Train-test balance is consistent: The relative class proportions remain closely aligned between training and test datasets, with only a small shift toward class 1 in test_dataset_final.

The results show that the target distribution is balanced across both development datasets under the configured 10% threshold. No class falls near the cutoff, and all evaluated class proportions pass the test. The observed train-test differences are limited to a small change in the majority class between datasets.

Parameters:

{
  "min_percent_threshold": 10
}
            

Tables

dataset	Exited	Percentage of Rows (%)	Pass/Fail
train_dataset_final	0	50.33%	Pass
train_dataset_final	1	49.67%	Pass
test_dataset_final	1	51.31%	Pass
test_dataset_final	0	48.69%	Pass

Figures

2026-10-30 21:07:35,969 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.ClassImbalance:development_data does not exist in model's document

validmind.data_validation.UniqueRows:development_data

❌ Unique Rows Development Data

The UniqueRows test evaluates column-level diversity by comparing the share of unique values in each dataset column against the minimum threshold of 1%. Results are reported separately for train_dataset_final and test_dataset_final, with each column showing its number of unique values, percentage of unique values, and pass/fail outcome. In the training dataset, 3 of 11 columns pass the threshold, while in the test dataset, 4 of 11 columns pass. The reported percentages range from 0.0774% to 100.0% in training and from 0.3091% to 100.0% in testing.

Key insights:

• EstimatedSalary is fully unique: EstimatedSalary records 100.0% unique values in both train_dataset_final and test_dataset_final, corresponding to 2,585 unique values in training and 647 in testing.
• Balance and CreditScore pass in both datasets: Balance passes with 66.6925% unique values in training and 70.6337% in testing, while CreditScore passes with 16.5571% in training and 46.677% in testing.
• Low-cardinality columns consistently fail: HasCrCard, IsActiveMember, Geography_Germany, Geography_Spain, Gender_Male, and Exited each have only 2 unique values and fail in both datasets, with percentages of 0.0774% in training and 0.3091% in testing.
• Tenure differs between train and test: Tenure fails in train_dataset_final at 0.4255% unique values but passes in test_dataset_final at 1.7002%, despite having 11 unique values in both datasets.
• NumOfProducts remains below threshold: NumOfProducts has 4 unique values in both datasets and fails in both cases, with 0.1547% unique values in training and 0.6182% in testing.

The results show that uniqueness is concentrated in a small subset of columns, primarily EstimatedSalary, Balance, and CreditScore, with Tenure meeting the threshold only in the test dataset. Most remaining columns fall below the 1% threshold, driven by very low counts of distinct values. Overall, the test output indicates a mixed uniqueness profile across the development data, with broad separation between highly unique continuous fields and low-cardinality binary or limited-category fields.

Parameters:

{
  "min_percent_threshold": 1
}
            

Tables

dataset	Column	Number of Unique Values	Percentage of Unique Values (%)	Pass/Fail
train_dataset_final	CreditScore	428	16.5571	Pass
train_dataset_final	Tenure	11	0.4255	Fail
train_dataset_final	Balance	1724	66.6925	Pass
train_dataset_final	NumOfProducts	4	0.1547	Fail
train_dataset_final	HasCrCard	2	0.0774	Fail
train_dataset_final	IsActiveMember	2	0.0774	Fail
train_dataset_final	EstimatedSalary	2585	100.0000	Pass
train_dataset_final	Geography_Germany	2	0.0774	Fail
train_dataset_final	Geography_Spain	2	0.0774	Fail
train_dataset_final	Gender_Male	2	0.0774	Fail
train_dataset_final	Exited	2	0.0774	Fail
test_dataset_final	CreditScore	302	46.6770	Pass
test_dataset_final	Tenure	11	1.7002	Pass
test_dataset_final	Balance	457	70.6337	Pass
test_dataset_final	NumOfProducts	4	0.6182	Fail
test_dataset_final	HasCrCard	2	0.3091	Fail
test_dataset_final	IsActiveMember	2	0.3091	Fail
test_dataset_final	EstimatedSalary	647	100.0000	Pass
test_dataset_final	Geography_Germany	2	0.3091	Fail
test_dataset_final	Geography_Spain	2	0.3091	Fail
test_dataset_final	Gender_Male	2	0.3091	Fail
test_dataset_final	Exited	2	0.3091	Fail

2026-10-30 21:07:46,304 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.UniqueRows:development_data does not exist in model's document

validmind.data_validation.TabularNumericalHistograms:development_data

Tabular Numerical Histograms Development Data

The TabularNumericalHistograms test evaluates the univariate distributions of numerical input features through histograms. The results present feature-level histograms for the train and test datasets covering CreditScore, Tenure, Balance, NumOfProducts, HasCrCard, IsActiveMember, EstimatedSalary, Geography_Germany, Geography_Spain, and Gender_Male. Across both datasets, the plots show the shape, range, and concentration of values for each feature, including continuous variables and binary or low-cardinality encoded variables.

Key insights:

• CreditScore is broadly bell-shaped: CreditScore in both train and test datasets is concentrated in the mid-range, with the highest density around roughly 600–700 and thinner tails extending toward lower and higher scores.

• Balance shows a large mass at zero: Balance exhibits a pronounced spike at 0 in both datasets, alongside a separate concentration across positive balances centered approximately around 100k–140k, indicating a zero-heavy distribution with a broad positive range.

• EstimatedSalary is relatively even across range: EstimatedSalary is distributed across nearly the full observed range up to about 200k in both train and test datasets, with bar heights that remain fairly even and no dominant concentration in a narrow interval.

• NumOfProducts is highly concentrated at 1 and 2: NumOfProducts takes discrete values from 1 to 4, with most observations at 1 and 2, fewer at 3, and only a very small number at 4 in both datasets.

• Tenure is spread across all discrete levels: Tenure appears across integer values from 0 to 10 in both train and test datasets, with counts present at each level and no single tenure value overwhelmingly dominating the distribution.

• Binary indicators are imbalanced to varying degrees: HasCrCard is concentrated at 1 more than 0 in both datasets, IsActiveMember is closer to balanced with a modestly higher count at 0 in train and near-balance in test, Geography_Germany and Geography_Spain both show more false than true values, and Gender_Male is close to evenly split.

The histograms show that the train and test datasets have closely aligned univariate feature distributions across the numerical inputs displayed. The most distinctive distributional characteristics are the zero-heavy Balance feature, the discrete concentration of NumOfProducts at lower values, and the mild class imbalance in several binary indicators. Continuous variables such as CreditScore and EstimatedSalary display stable shapes across train and test, with no visually distinct shifts between the two samples.

Figures

2026-10-30 21:08:28,088 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.TabularNumericalHistograms:development_data does not exist in model's document

validmind.data_validation.MutualInformation:development_data

Mutual Information Development Data

The Mutual Information test evaluates the statistical dependency between each feature and the target to quantify feature relevance in the development data. The results are presented as bar charts for both train_dataset_final and test_dataset_final, with mutual information scores shown against a minimum threshold of 0.01. In both datasets, NumOfProducts has the highest score by a wide margin, while the remaining features are distributed between modest positive values and near-zero values, with several features falling below the threshold.

Key insights:

• NumOfProducts is the dominant feature: NumOfProducts records the highest mutual information score in both datasets at approximately 0.10, materially exceeding all other features shown.

• Feature ranking differs between train and test: In train_dataset_final, Balance, Geography_Germany, Gender_Male, and IsActiveMember all appear above the 0.01 threshold, whereas in test_dataset_final, only CreditScore and IsActiveMember remain above that threshold alongside NumOfProducts.

• Several features show low information content: EstimatedSalary, HasCrCard, and Tenure are below the 0.01 threshold in the training results, and multiple features in the test results are at or near zero, including Tenure, HasCrCard, EstimatedSalary, Geography_Germany, and Gender_Male.

• CreditScore changes across datasets: CreditScore is below the 0.01 threshold in the training plot but rises above the threshold in the test plot, where it becomes the second-highest scoring feature at approximately 0.018.

The mutual information profile is concentrated in a small subset of features, with NumOfProducts contributing the strongest univariate relationship to the target in both train and test data. The training results show a broader set of features above the threshold than the test results, while the test profile is more compressed, with several variables at or near zero. The observed differences in ranking across the two datasets indicate that feature relevance is not uniform across samples, although the leading position of NumOfProducts remains consistent.

Parameters:

{
  "min_threshold": 0.01
}
            

Figures

2026-10-30 21:08:43,584 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.MutualInformation:development_data does not exist in model's document

validmind.data_validation.PearsonCorrelationMatrix:development_data

Pearson Correlation Matrix Development Data

The Pearson Correlation Matrix test evaluates linear dependency among numerical variables in the development dataset using pairwise Pearson correlation coefficients. The result is presented as heat maps for both the train and test splits, with coefficients ranging from -1 to 1 and diagonal values of 1.0. Across both matrices, most off-diagonal correlations are close to zero, with a limited number of moderate positive and negative relationships visible among balance, geography indicators, number of products, active membership, and the target variable Exited.

Key insights:

• No high-correlation pairs observed: All visible off-diagonal correlations remain well below the stated 0.7 absolute threshold in both train and test heat maps. The largest observed absolute correlations are approximately 0.44 in train and 0.40 in test.

• Balance and Geography_Germany show the strongest positive relationship: The correlation between Balance and Geography_Germany is 0.44 in the train split and 0.40 in the test split. This is the strongest positive association shown in the matrices.

• Geography indicators are moderately negatively related: Geography_Germany and Geography_Spain have correlations of -0.37 in train and -0.35 in test. This is the strongest negative relationship displayed in both splits.

• Exited has only weak to modest linear associations: In the train split, Exited is correlated at 0.20 with Geography_Germany, 0.18 with Balance, and -0.19 with IsActiveMember. In the test split, the corresponding values are 0.19, 0.10, and -0.18, while other associations with Exited remain near zero.

• Train and test correlation structure is broadly consistent: The main pairwise relationships appear with similar sign and magnitude across both splits, including Balance with Geography_Germany, Geography_Germany with Geography_Spain, and Exited with IsActiveMember and Geography_Germany. Differences between splits are present but limited in size.

The correlation analysis shows a predominantly low-dependency feature set, with no pairwise relationships approaching the high-correlation threshold identified in the test definition. The most prominent structure is confined to a small number of moderate associations, particularly those involving Balance and geography indicators. The target variable Exited exhibits only weak to modest linear relationships with individual variables, and the overall correlation pattern remains similar between the train and test development splits.

Figures

2026-10-30 21:08:59,472 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.PearsonCorrelationMatrix:development_data does not exist in model's document

validmind.data_validation.HighPearsonCorrelation:development_data

❌ High Pearson Correlation Development Data

The High Pearson Correlation test evaluates pairwise linear relationships between features to identify potentially redundant or strongly related variables using a correlation threshold of 0.3. The results table reports the top 10 correlations for both train_dataset_final and test_dataset_final, including each feature pair’s Pearson coefficient and Pass/Fail status. Across both datasets, the strongest observed relationships are concentrated in a small number of feature pairs, while the remaining listed correlations are below the configured threshold.

Key insights:

• Two pairs exceed the threshold: In both train_dataset_final and test_dataset_final, only (Balance, Geography_Germany) and (Geography_Germany, Geography_Spain) are marked as Fail. Their coefficients are 0.4371 and -0.3702 in train, and 0.4024 and -0.3526 in test.

• Correlation ranking is consistent across datasets: The same two feature pairs are the strongest correlations in both train and test, and the remaining top correlations also show similar ordering and magnitudes across the two datasets.

• Most reported relationships are modest: Aside from the two failing pairs, all listed coefficients fall within approximately -0.19 to 0.20 in train and -0.18 to 0.19 in test, remaining below the 0.3 threshold.

• Exited shows only weak-to-moderate pairwise relationships: Correlations involving Exited are all below the threshold in both datasets. The largest such values are (Geography_Germany, Exited) at 0.1983 in train and 0.1865 in test, and (IsActiveMember, Exited) at -0.1872 in train and -0.1829 in test.

The observed correlation structure is concentrated in two feature pairs that exceed the configured threshold in both development splits, while all other reported pairwise relationships remain below that cutoff. The similarity of coefficients between train_dataset_final and test_dataset_final indicates a stable correlation pattern across the sampled datasets. Overall, the results show limited high-correlation findings, with most listed relationships remaining comparatively small in magnitude.

Parameters:

{
  "max_threshold": 0.3,
  "top_n_correlations": 10
}
            

Tables

dataset	Columns	Coefficient	Pass/Fail
train_dataset_final	(Balance, Geography_Germany)	0.4371	Fail
train_dataset_final	(Geography_Germany, Geography_Spain)	-0.3702	Fail
train_dataset_final	(Geography_Germany, Exited)	0.1983	Pass
train_dataset_final	(Balance, Geography_Spain)	-0.1895	Pass
train_dataset_final	(IsActiveMember, Exited)	-0.1872	Pass
train_dataset_final	(Balance, Exited)	0.1755	Pass
train_dataset_final	(Balance, NumOfProducts)	-0.1728	Pass
train_dataset_final	(Gender_Male, Exited)	-0.1241	Pass
train_dataset_final	(NumOfProducts, Exited)	-0.0712	Pass
train_dataset_final	(Tenure, IsActiveMember)	-0.0529	Pass
test_dataset_final	(Balance, Geography_Germany)	0.4024	Fail
test_dataset_final	(Geography_Germany, Geography_Spain)	-0.3526	Fail
test_dataset_final	(Geography_Germany, Exited)	0.1865	Pass
test_dataset_final	(IsActiveMember, Exited)	-0.1829	Pass
test_dataset_final	(Balance, NumOfProducts)	-0.1826	Pass
test_dataset_final	(Balance, Geography_Spain)	-0.1320	Pass
test_dataset_final	(Gender_Male, Exited)	-0.1063	Pass
test_dataset_final	(NumOfProducts, Exited)	-0.1034	Pass
test_dataset_final	(Balance, Exited)	0.1034	Pass
test_dataset_final	(Geography_Spain, Exited)	-0.0861	Pass

2026-10-30 21:09:09,760 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.data_validation.HighPearsonCorrelation:development_data does not exist in model's document

validmind.model_validation.ModelMetadata

Model Metadata

The ModelMetadata test compares metadata across models to summarize their implementation characteristics, including modeling technique, framework, framework version, and programming language. The results are presented as a two-row comparison table for log_model_champion and rf_model. For each model, the table reports the same set of metadata fields and their recorded values, enabling direct comparison of the documented implementation attributes.

Key insights:

• Metadata is consistent across models: Both log_model_champion and rf_model are recorded with the modeling technique SKlearnModel, the modeling framework sklearn, framework version 1.7.2, and programming language Python.
• No cross-model framework variation observed: The framework and version fields are identical across both entries, with both models documented under sklearn version 1.7.2.
• Implementation language is uniform: Both models are documented as implemented in Python, indicating no language differences within the compared set.

The comparison table shows fully aligned metadata across the two documented models. No differences are present in modeling technique, framework, framework version, or programming language. Based on the reported fields, the compared models share the same high-level implementation metadata.

Tables

model	Modeling Technique	Modeling Framework	Framework Version	Programming Language
log_model_champion	SKlearnModel	sklearn	1.7.2	Python
rf_model	SKlearnModel	sklearn	1.7.2	Python

2026-10-30 21:09:16,386 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.ModelMetadata does not exist in model's document

validmind.model_validation.sklearn.ModelParameters

Model Parameters

The Model Parameters test extracts model configuration settings to document the parameterization used for reproducibility and transparency. The results present parameter tables for two models, log_model_champion and rf_model, with each row listing a parameter name and its recorded value. For log_model_champion, the table includes logistic regression settings such as regularization, solver, and iteration controls. For rf_model, the table includes random forest settings covering ensemble size, split criteria, sampling behavior, and reproducibility controls.

Key insights:

• Two distinct model configurations documented: The results capture parameter sets for both log_model_champion and rf_model, indicating that model configuration was recorded for two separate estimators within the test output.

• Logistic model uses L1 regularization: log_model_champion is configured with penalty = l1, solver = liblinear, and C = 1. The parameter set also shows fit_intercept = True, max_iter = 100, tol = 0.0001, and multi_class = auto.

• Random forest uses 50 trees: rf_model is configured with n_estimators = 50, criterion = gini, and max_features = sqrt. The table also shows bootstrap = True, min_samples_split = 2, min_samples_leaf = 1, and oob_score = False.

• Reproducibility controls are explicitly recorded: The random forest configuration includes random_state = 42, and both models record execution-related settings such as verbose = 0 and warm_start = False. These entries document the fixed and operational aspects of the model configurations captured by the test.

The results provide a structured record of the parameterization for both the logistic regression champion model and the random forest model. The documented settings identify the regularization and optimization choices for log_model_champion and the ensemble construction choices for rf_model, while also recording operational parameters such as verbosity, warm start behavior, and a fixed random seed for the random forest. Collectively, the output establishes a reproducible configuration snapshot for the tested estimators.

Tables

model	Parameter	Value
log_model_champion	C	1
log_model_champion	dual	False
log_model_champion	fit_intercept	True
log_model_champion	intercept_scaling	1
log_model_champion	max_iter	100
log_model_champion	multi_class	auto
log_model_champion	penalty	l1
log_model_champion	solver	liblinear
log_model_champion	tol	0.0001
log_model_champion	verbose	0
log_model_champion	warm_start	False
rf_model	bootstrap	True
rf_model	ccp_alpha	0.0
rf_model	criterion	gini
rf_model	max_features	sqrt
rf_model	min_impurity_decrease	0.0
rf_model	min_samples_leaf	1
rf_model	min_samples_split	2
rf_model	min_weight_fraction_leaf	0.0
rf_model	n_estimators	50
rf_model	oob_score	False
rf_model	random_state	42
rf_model	verbose	0
rf_model	warm_start	False

2026-10-30 21:09:25,239 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.sklearn.ModelParameters does not exist in model's document

validmind.model_validation.sklearn.ROCCurve

ROC Curve

The ROC Curve test evaluates classification performance by plotting the trade-off between true positive rate and false positive rate across decision thresholds and summarizing discrimination with the AUC statistic. The results show ROC curves for log_model_champion on both train_dataset_final and test_dataset_final, each compared against the diagonal random-classification baseline. The reported AUC is 0.68 on the training dataset and 0.66 on the test dataset, with both curves remaining above the random baseline across most of the false positive rate range.

Key insights:

• Similar train and test AUCs: The AUC values are 0.68 on the training dataset and 0.66 on the test dataset. The 0.02 difference indicates closely aligned discrimination performance across the two samples.
• Performance exceeds random baseline: Both ROC curves lie above the diagonal reference line associated with AUC = 0.5. This shows that the model separates the two classes better than random classification on both datasets.
• Moderate discrimination level: The observed AUC values remain in the mid-0.60 range on both datasets. The ROC curves rise progressively above the baseline rather than approaching the upper-left corner sharply.

The ROC results show that log_model_champion achieves consistent classification discrimination across training and test datasets, with AUC values of 0.68 and 0.66 respectively. The close alignment between these two values indicates limited change in ranking performance between samples. Overall, the model demonstrates better-than-random class separation, with observed discrimination at a moderate level rather than a strongly pronounced one.

Figures

2026-10-30 21:09:37,503 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.sklearn.ROCCurve does not exist in model's document

validmind.model_validation.sklearn.MinimumROCAUCScore

✅ Minimum ROCAUC Score

The Minimum ROC AUC Score test evaluates whether the model’s ROC AUC score meets or exceeds a predefined minimum threshold. Results are reported for both train_dataset_final and test_dataset_final, with each dataset showing the observed ROC AUC score, the applied threshold of 0.5, and the associated pass/fail outcome. The recorded scores are 0.6791 for the training dataset and 0.6604 for the test dataset, and both results are marked as passing.

Key insights:

• Both datasets passed threshold: The ROC AUC score exceeded the minimum threshold of 0.5 on both evaluated datasets. train_dataset_final achieved 0.6791 and test_dataset_final achieved 0.6604.
• Training score is higher: The training dataset score of 0.6791 is higher than the test dataset score of 0.6604. The difference between the two reported scores is 0.0187.
• Test performance remains above cutoff: The test dataset ROC AUC score remains above the defined minimum threshold, with a margin of 0.1604 above 0.5.

The results show that the model satisfied the Minimum ROC AUC Score requirement on both the training and test datasets. Performance is slightly higher on the training sample than on the test sample, while both scores remain above the configured threshold. The observed outcome indicates that the model met this specific discrimination criterion across both evaluated datasets.

Parameters:

{
  "min_threshold": 0.5
}
            

Tables

dataset	Score	Threshold	Pass/Fail
train_dataset_final	0.6791	0.5	Pass
test_dataset_final	0.6604	0.5	Pass

2026-10-30 21:09:44,535 - INFO(validmind.vm_models.result.result): Test driven block with result_id validmind.model_validation.sklearn.MinimumROCAUCScore does not exist in model's document

In summary

In this final notebook, you learned how to:

• Implement a custom inline test
• Run and log your custom inline tests
• Use external custom test providers
• Run and log tests from your custom test providers
• Re-run tests provided by your development team to verify that they were run and reported accurately

With our ValidMind for validation series of notebooks, you learned how to validate a record (model) end-to-end with the ValidMind Library by running through some common scenarios in a typical validation setting:

• Verifying the data quality steps performed by the development team
• Independently replicating the champion's results and conducting additional tests to assess performance, stability, and robustness
• Setting up test inputs and a challenger for comparative analysis
• Running validation tests, analyzing results, and logging artifacts to ValidMind

Next steps

Work with your validation report

Now that you've logged all your test results and verified the work done by the development team, head to the ValidMind Platform to wrap up your validation report. Continue to work on your validation report by:

• Inserting additional test results: Click Link Evidence under any Evidence panel of 2. Validation in your validation report. (Learn more: Link evidence to reports)

• Making qualitative edits to your test descriptions: Expand any linked evidence under Validator Evidence and click See evidence details to review and edit the ValidMind-generated test descriptions for quality and accuracy. (Learn more: Preparing validation reports)

• Adding more findings: Click Link Finding to Report in any validation report section, then click + Create New Finding. (Learn more: Add and manage artifacts)

• Adding risk assessment notes: Click under Risk Assessment Notes in any validation report section to access the text editor and content editing toolbar, including an option to generate a draft with AI. Once generated, edit your ValidMind-generated test descriptions to adhere to your organization's requirements. (Learn more: Work with content blocks)

• Assessing compliance: Under the Guideline for any validation report section, click Assessment and select the compliance status from the drop-down menu. (Learn more: Assign compliance assessments)

• Collaborate with other stakeholders: Use the ValidMind Platform's real-time collaborative features to work seamlessly together with the rest of your organization, including developers. Propose suggested changes in the documentation, work with versioned history, and use comments to discuss specific portions of the documentation. (Learn more: Collaborate with others)

When your validation report is complete and ready for review, submit it for approval from the same ValidMind Platform where you made your edits and collaborated with the rest of your organization, ensuring transparency and a thorough validation history. (Learn more: Submit documents)

Learn more

Now that you're familiar with the basics, you can explore the following notebooks to get a deeper understanding on how the ValidMind Library assists you in streamlining validation:

Use cases

• Validate an application scorecard model

Discover more learning resources

Learn more about the ValidMind Library tools we used in this notebook:

• Explore tests
• Run dataset-based tests
• Implement custom tests
• Integrate external test providers

We also offer many interactive notebooks to help you use the ValidMind Library to streamline your work:

• Run tests & test suites
• Use ValidMind Library features
• Code samples by use case

Or, visit our documentation to learn more about ValidMind.

---

Copyright © 2023-2026 ValidMind Inc. All rights reserved.
Refer to LICENSE for details.
SPDX-License-Identifier: AGPL-3.0 AND ValidMind Commercial