Run evaluations and generate synthetic datasets
Learn how to run evaluations and generate synthetic datasets with the Azure AI Evaluation SDK.
Learning objectives
By the end of this module, you're able to:
- Assess a generative AI app response using performance and quality metrics
- Assess a generative AI app response using risk and safety metrics
- Run an evaluation and track the results in Azure AI Foundry
- Create a custom evaluator with Prompty
- Send queries to an endpoint and run evaluators on the resulting query and response
- Generate a synthetic dataset using conversation starters
Prerequisites
- An Azure subscription – Create one for free
- Familiarity with Azure and the Azure portal
- Ability to understand Python at a beginner level
- Developing in browser:
- Developing locally: