Create and manage runtimes
Prompt flow's runtime provides the computing resources required for the application to run, including a Docker image that contains all necessary dependency packages. This reliable and scalable runtime environment enables prompt flow to efficiently execute its tasks and functions, ensuring a seamless user experience for users.
Permissions/roles for runtime management
To create and use a runtime for prompt flow authoring, you need to have the
AzureML Data Scientist role in the workspace. To learn more, see Prerequisites.
Permissions/roles for deployments
After deploying a prompt flow, the endpoint must be assigned the
AzureML Data Scientist role to the workspace for successful inferencing. This can be done at any point after the endpoint has been created.
Create runtime in UI
- You need
AzureML Data Scientistrole in the workspace to create a runtime.
- Make sure the default data store in your workspace is blob type.
- If you secure prompt flow with virtual network, please follow Network isolation in prompt flow to learn more detail.
Prompt flow is not supported in the workspace which has data isolation enabled. The enableDataIsolation flag can only be set at the workspace creation phase and can't be updated.
Prompt flow is not supported in the project workspace which was created with a workspace hub. The workspace hub is a private preview feature.
Create compute instance runtime in UI
If you do not have a compute instance, create a new one: Create and manage an Azure Machine Learning compute instance.
- Select add runtime in runtime list page.
- Select compute instance you want to use as runtime. Because compute instances are isolated by user, you can only see your own compute instances or the ones assigned to you. To learn more, see Create and manage an Azure Machine Learning compute instance.
- Authenticate on the compute instance. You only need to do auth one time per region in 6 month.
- Select create new custom application or existing custom application as runtime.
This is recommended for most users of prompt flow. The prompt flow system creates a new custom application on a compute instance as a runtime.
If you want to install other packages in your project, you should create a custom environment. To learn how to build your own custom environment, see Customize environment with docker context for runtime.
- We are going to perform an automatic restart of your compute instance. Please ensure that you do not have any tasks or jobs running on it, as they may be affected by the restart.
To use an existing custom application as a runtime, choose the option "existing". This option is available if you have previously created a custom application on a compute instance. For more information on how to create and use a custom application as a runtime, learn more about how to create custom application as runtime.
Grant sufficient permissions to use the runtime
After creating the runtime, you need to grant the necessary permissions to use it.
Permissions required to assign roles
To assign role, you need to have
owner or have
Microsoft.Authorization/roleAssignments/write permission on the resource.
To use the runtime, assigning the
AzureML Data Scientist role of workspace to user (if using Compute instance as runtime) or endpoint (if using managed online endpoint as runtime). To learn more, see Manage access to an Azure Machine Learning workspace
This operation may take several minutes to take effect.
Using runtime in prompt flow authoring
When you're authoring your prompt flow, you can select and change the runtime from left top corner of the flow page.
When performing evaluation, you can use the original runtime in the flow or change to a more powerful runtime.
Update runtime from UI
We regularly update our base image (
mcr.microsoft.com/azureml/promptflow/promptflow-runtime-stable) to include the latest features and bug fixes. We recommend that you update your runtime to the latest version if possible.
Every time you open the runtime details page, we'll check whether there are new versions of the runtime. If there are new versions available, you'll see a notification at the top of the page. You can also manually check the latest version by clicking the check version button.
Try to keep your runtime up to date to get the best experience and performance.
Go to the runtime details page and select the "Update" button at the top. Here you can update the environment to use in your runtime. If you select use default environment, system will attempt to update your runtime to the latest version.
If you used a custom environment, you need to rebuild it using the latest prompt flow image first, and then update your runtime with the new custom environment.