Integrations and support

Published

November 20, 2024

Can you integrate with JIRA to connect with our Model Development pipeline?

ValidMind is planning to provide integration with JIRA tickets via the JIRA Python API. You will be able to configure ValidMind to update the status of a particular JIRA ticket when a specific state or approval is triggered from the workflow (roadmap item – Q3’2023).

What libraries beyond XGBoost are supported?

ValidMind supports the most popular open-source model development libraries in Python and R, such as:

  • scikit-learn
  • XGBoost
  • statsmodels
  • PyTorch
  • TensorFlow

ValidMind supports ingesting test results from your training and evaluation pipeline, such as using batch prediction or online prediction mechanisms. We are also implementing standard documentation via the library for additional modeling techniques, check Do you include explainability-related testing and documentation? for more information.

What other programming languages and development environments do you support beyond Python and Jupyter Notebook, such as R and SAS?

The ValidMind Library is designed to be platform-agnostic and compatible with the most popular open-source programming languages and model development environments.

Currently, we support Python ≧3.8 and <3.11 and the most popular AI/ML and data science libraries (scikit-learn, XGBoost, statsmodels, PyTorch, TensorFlow).

We are working on deploying support for R 4.0+ and associated libraries (roadmap item – Q2’2023).

Support for commercial and closed-source programming languages such as SAS and Matlab depends on specific deployment details and commercial agreements with customers.

Do you support integration with data lakes and ETL solutions?

Support for connecting to data lakes and data processing or ETL pipelines is on our roadmap (Q3’2023+).

We will be implementing connector interfaces allowing extraction of relationships between raw data sources and final post-processed datasets for preloaded session instances received from Spark and Snowflake.

Which model development packages/libraries are supported by the ValidMind Library? What about complex/distributed models built with TensorFlow?

ValidMind supports the most popular open-source model development libraries in Python and R, such as:

  • scikit-learn
  • XGBoost
  • statsmodels
  • PyTorch
  • TensorFlow

For distributed training pipelines built with frameworks like TensorFlow, ValidMind can directly access the trained model instance to extract metadata stored in the model object, if the library is imported from within the pipeline’s code. ValidMind can also ingest test results from the customer’s training or evaluation pipeline, using batch prediction or online prediction mechanisms.

Is it possible for us to integrate the tool with LLMs like GPT-3?

ValidMind is integrating LLMs tools into our documentation features, enabling the following documentation features:

  • Generating content recommendations (or “starting points”) for model developers for specific sections of the documentation, based on historical documentations (roadmap item — Q3’2023).
  • Providing insights to model developers and model reviewers on possible model risks, and mitigation actions/improvements to the model, based on historical model documentations (roadmap item currently in research – not scheduled).

Can you handle more sophisticated AI/ML libraries such as Pytorch, TensorFlow?

ValidMind supports the most popular open-source model development libraries in Python, R, such as :

  • scikit-learn
  • XGBoost
  • statsmodels
  • PyTorch
  • TensorFlow

For distributed training pipelines built with frameworks, such as TensorFlow, ValidMind can directly access the trained model instance to extract metadata stored in the model object if the ValidMind Library is imported from within the pipeline’s code. ValidMind can also ingest test results from the customer’s training/evaluation pipeline, such as using batch prediction or online prediction mechanisms.

Does ValidMind support data dictionaries?

You can pass a data dictionary to ValidMind via the library, such as in CSV format.