Configure connections

Connections link ​ValidMind with the external systems your teams already use.

Each connection enables you to authenticate and keep events flowing between platforms such as Jira, ServiceNow, Amazon Bedrock and SageMaker, MLflow, and GitLab.

Prerequisites

• You are logged in to ValidMind.
• You can manage workspace settings.¹
• The integration’s required secrets already exist in integration secrets.²
• You coordinated with the external system owner to obtain the required credentials and configuration details for the connection.

¹ Manage permissions

² Manage secrets

Supported connections

Jira

A project tracking tool that teams use to plan work, track progress, and manage issues across a project lifecycle.

Required configuration details:

jira instance url

The base URL for your Cloud or Server deployment, such as https://yourcompany.atlassian.net.

personal access token

The secret that stores a Jira PAT with read:jira-work and write:jira-work scopes.

jql query (optional)

Limits synced issues by project, status, or labels. Leave blank to sync all tickets.

ServiceNow

An IT service management platform that helps teams handle requests incidents assets and other operational workflows.

Required configuration details:

servicenow instance url

The base URL for your ServiceNow environment.

authentication method

Username and password or another supported method, such as OAuth.

username

The ServiceNow username.

password

The secret that stores the ServiceNow password.

servicenow board

The board or table that receives linked tickets.

Amazon SageMaker

An ML development service that lets teams build train and deploy models at scale.

Required configuration details:

aws region

The primary region where your SageMaker model registry lives, for example us-west-2.

aws access key id

The secret generated by AWS IAM with permissions to read the model registry.

aws secret access key

The secret generated by AWS IAM with permissions to read the model registry.

Amazon Bedrock

A managed service that provides access to foundation models and tools for building generative AI applications, including agents, flows, and AgentCore runtimes.

Required configuration details:

aws region

The primary region where your Bedrock resources are deployed, for example us-east-1.

aws access key id

The secret generated by AWS IAM with permissions to access Bedrock.

aws secret access key

The secret generated by AWS IAM with permissions to access Bedrock.

MLflow

An open-source toolset that tracks experiments packages models and manages their deployment.

Required configuration details:

tracking uri

The tracking server endpoint, such as https://mlflow.example.com:5000.

username

The MLflow username.

password

The secret that stores the MLflow password.

GitLab

Synchronize GitLab model registry with ValidMind model inventory for comprehensive model tracking.

Required configuration details:

project id

The GitLab project ID or path containing ML models (required).

gitlab instance url (optional)

Leave empty to use GitLab.com, or enter your self-hosted GitLab URL.

personal access token

Select a secret containing your GitLab Personal Access Token with api or read_api scope.

Databricks

A unified analytics platform for data engineering, data science, and machine learning, including Unity Catalog for centralized metadata management.

Required configuration details:

databricks host

The URL of your Databricks workspace, such as https://yourcompany.cloud.databricks.com.

client id

The ID of your SQL Warehouse (found in Databricks SQL Warehouses settings).

client secret

The secret that stores your Databricks personal access token (PAT).

Custom

A user-defined connection to a third-party system that implements the ​ValidMind reference API.³

³ Implement custom integrations

Use custom integrations to connect internal model registries, proprietary platforms, or any service that exposes data through a compatible API.

Required configuration details:

endpoint url

The base URL of your deployed integration that implements the reference API.

api key

A secret containing the API key for authenticating with your integration.

integration types

Select which types of objects this integration provides — Model Registry and/or Artifact / Ticket Source.

Add connections

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Click Add Connection.

4. In the modal that opens, select one of the supported connections, such as MLflow or Jira.

5. Enter in the:

   • integration name — How other admins can identify the connection.
   • description (optional) — The intended usage or additional details.
   • Connection-specific fields,⁴ such as:
     • tracking uri — The identifiers needed for the connection.
     • password — The secret that stores the secret key, password, or token required by the external service.
   • initial status — Set to Operational to enable immediately or Disabled if you plan to finish setup later.

6. Click Save Integration.

⁴ Supported connections

You can now test the connection to ensure it is working as expected.

View connection details

The connection detail page provides a dashboard-style view for managing a connection, its linked models, and metric sources.

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Click the connection you want to view.

The connection detail page displays:

• Connection status — Shows whether the connection is operational, disabled, or experiencing issues.
• Linked models — Lists all ​ValidMind inventory records linked to resources in this integration.
• Monitoring — For integrations that provide metrics (such as Amazon Bedrock), shows configured metric sources and their sync status.

Monitor connection health

The Connections page provides visibility into the health of your integrations at a glance.

Health indicators

Each connection displays a status indicator:

• Operational — The connection is active and working correctly.
• Needs Attention — The connection has issues that need to be resolved.
• Disabled — The connection has been manually disabled.

View connections requiring attention

Connections with errors or warnings display a status indicator, such as 1 link with sync errors. Review these connections to:

1. Identify the cause of the issue from the error details.
2. Check that required secrets are still valid.
3. Verify the external service is accessible.
4. Test the connection after making changes.

View linked models

To see which models use a specific connection:

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Click the connection to open its details page.

4. Review the linked models table, which shows the VM Model (​ValidMind inventory record) and External Name (the linked external model) for each connection.

View metric sources

For connections that provide monitoring metrics, such as Amazon Bedrock:

1. Open the connection details page.

2. Review the Metric Sources section to see available metrics.

3. These metrics can be added to monitoring documents for models linked to this connection.⁵

⁵ Work with metrics over time

Edit connections

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Click the connection you want to edit.

4. In the modal that appears, click Edit Connection.

5. Make your changes. For example:

   • tracking uri — Change the URI needed for the connection.
   • password — Change the secret that stores the secret key, password, or token required by the external service.

6. Click Update Integration to save your changes.

Test connections

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Hover over the connection you want to test.

4. When the menu appears, click on it and select Test Connection.

If the test is successful, the message Connection successful displays.

Delete connections

WarningDeleting a connection is permanent.

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Hover over the connection you want to remove.

4. When the menu appears, click on it and select Delete Connection.

5. In the confirmation dialog, select OK to confirm deletion.

View monitoring data

Some integrations, such as Amazon Bedrock, can provide metrics that sync into ​ValidMind for use in monitoring documents. Metric sources define which external evaluations or metrics are available for a linked model.

Access the Monitoring tab

1. In the left sidebar, click Settings.

2. Under Integrations, select Connections.

3. Click on a connection that supports metric sources, for example a Bedrock connection.

4. On the connection detail page, select the Monitoring tab.

   This section lists all configured metric sources, their sync status, and when they last synced.

How metric sources work

When a model in the ​ValidMind inventory is linked to an external resource, such as a Bedrock agent, available evaluations from that resource become metric sources. These metrics:

• Sync automatically on a scheduled interval.
• Appear in the From Integration tab when adding metrics over time to monitoring documents.⁶
• Support threshold configuration for alerting when values exceed acceptable limits.

⁶ Work with metrics over time

NoneBedrock Agent Evaluations

For Amazon Bedrock connections, metric sources include agent evaluation results such as accuracy, helpfulness, and other assessment metrics. Link a model to a Bedrock agent to enable these metrics.⁷

⁷ Link external models