GroundednessProEvaluator Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Evaluates service-based groundedness score for a given response, context, and query or a multi-turn conversation, including reasoning.

The groundedness measure calls Azure AI Evaluation service to assess how well the AI-generated answer is grounded in the source context. Even if the responses from LLM are factually correct, they'll be considered ungrounded if they can't be verified against the provided sources (such as your input source or your database).

Service-based groundedness scores are boolean values, where True indicates that the response is grounded.

Note

If this evaluator is supplied to the evaluate function, the aggregated metric

for the groundedness pro label will be "groundedness_pro_passing_rate".

Constructor

GroundednessProEvaluator(credential, azure_ai_project, *, threshold: int = 5, **kwargs)

Parameters

Name	Description
credential Required	TokenCredential The credential for connecting to Azure AI project. Required
azure_ai_project Required	AzureAIProject The scope of the Azure AI project. It contains subscription id, resource group, and project name.
threshold Required	int The threshold for the groundedness pro evaluator. Default is 5.
kwargs Required	Any Additional arguments to pass to the evaluator.

Keyword-Only Parameters

Name	Description
threshold	Default value: 5

Examples

Initialize with a specified threshold and call GroundednessProEvaluator with a query, response, and context.


   import os
   from azure.identity import DefaultAzureCredential
   from azure.ai.evaluation import GroundednessProEvaluator

   azure_ai_project = {
       "subscription_id": os.environ.get("AZURE_SUBSCRIPTION_ID"),
       "resource_group_name": os.environ.get("AZURE_RESOURCE_GROUP_NAME"),
       "project_name": os.environ.get("AZURE_PROJECT_NAME"),
   }
   credential = DefaultAzureCredential()

   groundedness_pro_eval = GroundednessProEvaluator(azure_ai_project=azure_ai_project, credential=credential, threshold=2)
   groundedness_pro_eval(
       query="What shape has 4 equilateral sides?",
       response="Rhombus",
       context="Rhombus is a shape with 4 equilateral sides.",
   )

Attributes

id

Evaluator identifier, experimental and to be used only with evaluation in cloud.

id = 'azureml://registries/azureml/models/Groundedness-Pro-Evaluator/versions/1'

Share via