2.11.1 Release notes

This release includes the exposure of the log_test_result function in the API client and the addition of agentic metrics from DeepEval as scorers.

Enhancements

Expose log_test_result function in API client

You can now use the log_test_result function within the API client to log test result dictionaries directly and synchronously to the API.

This new capability is available via both validmind.log_test_result and the shorter vm.log_test_result alias, simplifying result logging in your workflows.

log_test_result() documentation

Add agentic metrics from DeepEval as scorers

Four new agentic evaluation metrics from DeepEval are now available to assess your LLM agents:

• ArgumentCorrectness: Checks if agents generate correct arguments for tool calls.
• PlanAdherence: Measures whether agents follow their own execution plans.
• PlanQuality: Evaluates the logical quality, completeness, and efficiency of agent-generated plans.
• ToolCorrectness: Verifies if agents invoke the right tools with appropriate arguments.

These metrics broaden evaluation coverage across both tool use (actions) and strategic reasoning (plans). They are accessible via validmind.scorers.llm.deepeval and require validmind[llm].

ValidMind Library Python API