Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
The Azure Developer CLI (azd) Microsoft Foundry datasets extension (azure.ai.dataset) registers and manages versioned datasets in a Microsoft Foundry project from your terminal or editor. A dataset is the set of inputs an evaluation runs against, so keeping it versioned is what makes two evaluation results comparable. The extension publishes a local .jsonl file, or a folder that contains one, as a new version, and lists, inspects, and removes the versions already registered.
This article introduces the extension. For dataset concepts and how evaluations consume them, see the Microsoft Foundry documentation.
Key features
| Feature | Description |
|---|---|
| Versioned publishing | Registers a local .jsonl file, or a folder that contains one, as a dataset version, choosing the next version from what the project already carries. |
| Local validation | Refuses a malformed row, an empty dataset, a name the service doesn't accept, and a folder that could mean more than one dataset, before sending any request. |
| Version history | Lists the versions of a dataset, so you can see what an earlier evaluation ran against. |
| Content retrieval | Reads a dataset's contents back, whether the service hands out a file or the container holding it. |
| Scriptable output | Supports -o json on every command and --no-prompt for unattended use. |
Explore the workflow
- Prepare: Write or export the rows you want to evaluate against as a
.jsonlfile. - Register: Publish the file as the dataset's first version.
- Iterate: Publish a new version when the data changes; earlier versions stay addressable.
- Inspect: List versions and read back what a given version contains.
- Clean up: Delete versions you no longer need.
Try it
You need an existing Foundry project, permission to manage its datasets, and a nonempty local .jsonl file with one JSON object per line. The example uses your file at ./data/rows.jsonl; the extension doesn't create sample data for you.
Install or update the Azure Developer CLI, then install the standalone datasets extension.
azd extension install azure.ai.datasetThis article describes version
1.0.0-beta.2of the datasets extension, which requiresazdversion1.33.0or later. You don't need the evaluations or agents extension to register and manage datasets. If you also install the evaluations extension, version1.0.0-beta.2has the sameazdversion requirement. The agents extension is a separate install, and its1.0.0-beta.22release requiresazdversion1.37.0or later, as the evaluations overview explains.Verify the installed binary, not just the latest version advertised by a registry:
azd version azd extension list --installed -o json azd ai dataset versionIn the JSON listing, read
installedVersion, not the catalog'sversion. A newer extension can require a newerazdversion. To update an existing installation, runazd extension update azure.ai.dataset, then check the installed version again.Sign in. In an empty working folder, initialize a minimal
azdproject and set your Foundry project endpoint. If you already have anazdproject and environment, use them instead.azd auth login azd init --minimal --no-prompt -e dataset-dev azd env set FOUNDRY_PROJECT_ENDPOINT "https://<resource-name>.services.ai.azure.com/api/projects/<project-name>"Replace the endpoint with your own project endpoint. The following commands manage resources in that project.
Register a local file as a dataset.
azd ai dataset create my-dataset --from-file ./data/rows.jsonlPublish a new version after the data changes.
azd ai dataset update my-dataset --from-file ./data/rows.jsonlWithout
--version, the CLI chooses the next version after the latest registered version, for example,2.0after1.0. An explicit--versionpublishes that exact version instead.See what is registered, and what versions a dataset has.
azd ai dataset listazd ai dataset versions list my-datasetThe
listandversions listcommands show 20 rows by default. Use--limitto show a different number of rows, or--allto show every row.Read a specific version's contents back to disk.
azd ai dataset download my-dataset --version 1.0Creating a dataset without
--versionpublishes version1.0. Useversions listto find subsequent versions. Omitting--versionwhen downloading selects the latest version, so specify it when you need reproducible input data.Remove a version when you no longer need it.
azd ai dataset delete my-dataset --version 1.0The command asks you to confirm the delete. In scripts and pipelines, add
--force. With--no-promptor-o json, the command refuses to delete without it, andazdturns on--no-promptautomatically when it detects a CI/CD environment.Warning
Deleting a version removes it from the project. An evaluation pinned to that version can no longer run.
Choose a download destination
For a dataset registered as a single file, use --output-file to choose the exact destination. This option is valid only for a single-file dataset, in both azd ai dataset download and azd ai eval dataset download.
azd ai dataset download my-dataset --version 1.0 --output-file my-dataset-1.0.jsonl
For a folder dataset, use --output-dir downloads instead, even if the folder contains only one file. For container-backed data, single-file output requires single-file dataset metadata and a complete listing that resolves to exactly one file. Use --force only when you intend to overwrite an existing destination.
When to use the datasets extension
The azd datasets extension is the best fit when you want to:
- Work primarily from the terminal or editor, with a scriptable, repeatable workflow.
- Keep evaluation data under version control alongside the app it tests, and publish it the same way each time.
- Publish from CI/CD without a portal step.
The Microsoft Foundry evaluations extension can also declare datasets as part of an evaluation configuration and reconcile them for you. Use this extension when you want to manage datasets on their own, and that one when the dataset is one piece of a larger evaluation you keep in a config file.
The evaluations extension also has an azd ai eval dataset command group for managing datasets within an evaluation workflow. It and azd ai dataset work on the same dataset catalog when you configure them for the same Foundry project, so you don't copy a dataset between them. Their flags and output formats can differ, so check each command's --help output.
To generate data, use azd ai eval generate from the evaluations extension. The standalone datasets extension registers and manages data you supply; it doesn't generate evaluation rows, simulation seeds, or evaluators.
For evaluations that must retain the registered dataset's identity and version, see Preserve registered dataset identity. Those run-time guarantees and the unsupported-cap error come from the evaluations extension version 1.0.0-beta.2 or later, not from the standalone datasets extension.