2.9.5 Release notes
This release includes support for the DeepEval dataset in LLM tests, a demo notebook for agent testing documentation, the ability to enable a custom test description output structure, and an optional torch import in the client module.
Highlights
Add support for the DeepEval dataset in LLM tests
This update enhances the integration between DeepEval and the ValidMind library by adding support for a new dataset type specific to large language models (LLMs). You can now use various LLM tests from the DeepEval library. We have introduced new row-level metrics that return arrays.
These metrics can be used in the assign_scores interface and stored in memory by the virtual machine (VM) dataset object. This enables you to use them in generalized plots and statistical functions, aiding in the documentation and interpretation of test results.
Enhancements
Enable custom test description output structure
This update offers enhanced customization for LLM-generated test descriptions in the ValidMind Library. You can now manage test descriptions using a new context parameter in run_test(). This parameter accepts a dictionary with three optional keys:
test_description: Use domain-specific content to override test documentation.additional_context: Add business context, real-time information, or other relevant background for analyzing test results.instructions: Control the output formatting and style.
These options provide detailed control over creating LLM-generated test descriptions. While the existing environment variable approach (VALIDMIND_LLM_DESCRIPTIONS_CONTEXT) is still supported for backward compatibility, the instructions parameter will take precedence if both are used.
The environment variable approach will be deprecated in future releases. It is recommended to migrate to the new parameter-based approach.
Documentation
Removed legacy notebooks
This update removes multiple legacy agent demo notebooks and associated code samples that were previously used to demonstrate the integration of LangChain, LangGraph, and banking-specific agent functionality with the ValidMind Library.