How to configure a managed network for Azure AI Studio hubs

We have two network isolation aspects. One is the network isolation to access an Azure AI Studio hub. Another is the network isolation of computing resources for both your hub and project (such as compute instance, serverless and managed online endpoint.) This document explains the latter highlighted in the diagram. You can use hub built-in network isolation to protect your computing resources.

Diagram of hub network isolation.

You need to configure following network isolation configurations.

  • Choose network isolation mode. You have two options: allow internet outbound mode or allow only approved outbound mode.
  • If you use Visual Studio Code integration with allow only approved outbound mode, create FQDN outbound rules described in the use Visual Studio Code section.
  • If you use HuggingFace models in Models with allow only approved outbound mode, create FQDN outbound rules described in the use HuggingFace models section.
  • If you use one of the open-source models with allow only approved outbound mode, create FQDN outbound rules described in the curated by Azure AI section.

Network isolation architecture and isolation modes

When you enable managed virtual network isolation, a managed virtual network is created for the hub. Managed compute resources you create for the hub automatically use this managed virtual network. The managed virtual network can use private endpoints for Azure resources that are used by your hub, such as Azure Storage, Azure Key Vault, and Azure Container Registry.

There are three different configuration modes for outbound traffic from the managed virtual network:

Outbound mode Description Scenarios
Allow internet outbound Allow all internet outbound traffic from the managed virtual network. You want unrestricted access to machine learning resources on the internet, such as python packages or pretrained models.1
Allow only approved outbound Outbound traffic is allowed by specifying service tags. * You want to minimize the risk of data exfiltration, but you need to prepare all required machine learning artifacts in your private environment.
* You want to configure outbound access to an approved list of services, service tags, or FQDNs.
Disabled Inbound and outbound traffic isn't restricted. You want public inbound and outbound from the hub.

1 You can use outbound rules with allow only approved outbound mode to achieve the same result as using allow internet outbound. The differences are:

  • Always use private endpoints to access Azure resources.
  • You must add rules for each outbound connection you need to allow.
  • Adding FQDN outbound rules increase your costs as this rule type uses Azure Firewall. If you use outbound FQDN rules, charges for Azure Firewall are included in your billing. For more information, see Pricing.
  • The default rules for allow only approved outbound are designed to minimize the risk of data exfiltration. Any outbound rules you add might increase your risk.

The managed virtual network is preconfigured with required default rules. It's also configured for private endpoint connections to your hub, the hub's default storage, container registry, and key vault if they're configured as private or the hub isolation mode is set to allow only approved outbound. After choosing the isolation mode, you only need to consider other outbound requirements you might need to add.

The following diagram shows a managed virtual network configured to allow internet outbound:

Diagram of managed virtual network isolation configured for internet outbound.

The following diagram shows a managed virtual network configured to allow only approved outbound:

Note

In this configuration, the storage, key vault, and container registry used by the hub are flagged as private. Since they are flagged as private, a private endpoint is used to communicate with them.

Diagram of managed virtual network isolation configured for allow only approved outbound.

Prerequisites

Before following the steps in this article, make sure you have the following prerequisites:

  • An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.

  • The Microsoft.Network resource provider must be registered for your Azure subscription. This resource provider is used by the hub when creating private endpoints for the managed virtual network.

    For information on registering resource providers, see Resolve errors for resource provider registration.

  • The Azure identity you use when deploying a managed network requires the following Azure role-based access control (Azure RBAC) actions to create private endpoints:

    • Microsoft.MachineLearningServices/workspaces/privateEndpointConnections/read
    • Microsoft.MachineLearningServices/workspaces/privateEndpointConnections/write

Limitations

  • Azure AI Studio currently doesn't support bringing your own virtual network, it only supports managed virtual network isolation.
  • Once you enable managed virtual network isolation of your Azure AI, you can't disable it.
  • Managed virtual network uses private endpoint connection to access your private resources. You can't have a private endpoint and a service endpoint at the same time for your Azure resources, such as a storage account. We recommend using private endpoints in all scenarios.
  • The managed virtual network is deleted when the Azure AI is deleted.
  • Data exfiltration protection is automatically enabled for the only approved outbound mode. If you add other outbound rules, such as to FQDNs, Microsoft can't guarantee that you're protected from data exfiltration to those outbound destinations.
  • Using FQDN outbound rules increases the cost of the managed virtual network because FQDN rules use Azure Firewall. For more information, see Pricing.
  • FQDN outbound rules only support ports 80 and 443.
  • When using a compute instance with a managed network, use the az ml compute connect-ssh command to connect to the compute using SSH.
  • If your managed network is configured to allow only approved outbound, you cannot use an FQDN rule to access Azure Storage Accounts. You must use a private endpoint instead.

Configure a managed virtual network to allow internet outbound

Tip

The creation of the managed VNet is deferred until a compute resource is created or provisioning is manually started. When allowing automatic creation, it can take around 30 minutes to create the first compute resource as it is also provisioning the network.

  • Create a new hub:

    1. Sign in to the Azure portal, and choose Azure AI Studio from Create a resource menu.

    2. Select + New Azure AI.

    3. Provide the required information on the Basics tab.

    4. From the Networking tab, select Private with Internet Outbound.

    5. To add an outbound rule, select Add user-defined outbound rules from the Networking tab. From the Outbound rules sidebar, provide the following information:

      • Rule name: A name for the rule. The name must be unique for this hub.
      • Destination type: Private Endpoint is the only option when the network isolation is private with internet outbound. Hub managed virtual network doesn't support creating a private endpoint to all Azure resource types. For a list of supported resources, see the Private endpoints section.
      • Subscription: The subscription that contains the Azure resource you want to add a private endpoint for.
      • Resource group: The resource group that contains the Azure resource you want to add a private endpoint for.
      • Resource type: The type of the Azure resource.
      • Resource name: The name of the Azure resource.
      • Sub Resource: The sub resource of the Azure resource type.

      Select Save to save the rule. You can continue using Add user-defined outbound rules to add rules.

    6. Continue creating the hub as normal.

  • Update an existing hub:

    1. Sign in to the Azure portal, and select the hub that you want to enable managed virtual network isolation for.

    2. Select Networking, then select Private with Internet Outbound.

      • To add an outbound rule, select Add user-defined outbound rules from the Networking tab. From the Outbound rules sidebar, provide the same information as used when creating a hub in the 'Create a new hub' section.

      • To delete an outbound rule, select delete for the rule.

    3. Select Save at the top of the page to save the changes to the managed virtual network.

Configure a managed virtual network to allow only approved outbound

Tip

The managed VNet is automatically provisioned when you create a compute resource. When allowing automatic creation, it can take around 30 minutes to create the first compute resource as it is also provisioning the network. If you configured FQDN outbound rules, the first FQDN rule adds around 10 minutes to the provisioning time.

  • Create a new hub:

    1. Sign in to the Azure portal, and choose Azure AI Studio from Create a resource menu.

    2. Select + New Azure AI.

    3. Provide the required information on the Basics tab.

    4. From the Networking tab, select Private with Approved Outbound.

    5. To add an outbound rule, select Add user-defined outbound rules from the Networking tab. From the Outbound rules sidebar, provide the following information:

      • Rule name: A name for the rule. The name must be unique for this hub.
      • Destination type: Private Endpoint, Service Tag, or FQDN. Service Tag and FQDN are only available when the network isolation is private with approved outbound.

      If the destination type is Private Endpoint, provide the following information:

      • Subscription: The subscription that contains the Azure resource you want to add a private endpoint for.
      • Resource group: The resource group that contains the Azure resource you want to add a private endpoint for.
      • Resource type: The type of the Azure resource.
      • Resource name: The name of the Azure resource.
      • Sub Resource: The sub resource of the Azure resource type.

      Tip

      The hub managed VNet doesn't support creating a private endpoint to all Azure resource types. For a list of supported resources, see the Private endpoints section.

      If the destination type is Service Tag, provide the following information:

      • Service tag: The service tag to add to the approved outbound rules.
      • Protocol: The protocol to allow for the service tag.
      • Port ranges: The port ranges to allow for the service tag.

      If the destination type is FQDN, provide the following information:

      • FQDN destination: The fully qualified domain name to add to the approved outbound rules.

      Select Save to save the rule. You can continue using Add user-defined outbound rules to add rules.

    6. Continue creating the hub as normal.

  • Update an existing hub:

    1. Sign in to the Azure portal, and select the hub that you want to enable managed virtual network isolation for.

    2. Select Networking, then select Private with Approved Outbound.

      • To add an outbound rule, select Add user-defined outbound rules from the Networking tab. From the Outbound rules sidebar, provide the same information as when creating a hub in the previous 'Create a new hub' section.

      • To delete an outbound rule, select delete for the rule.

    3. Select Save at the top of the page to save the changes to the managed virtual network.

Manually provision a managed VNet

The managed VNet is automatically provisioned when you create a compute instance. When you rely on automatic provisioning, it can take around 30 minutes to create the first compute instance as it is also provisioning the network. If you configured FQDN outbound rules (only available with allow only approved mode), the first FQDN rule adds around 10 minutes to the provisioning time. If you have a large set of outbound rules to be provisioned in the managed network, it can take longer for provisioning to complete. The increased provisioning time can cause your first compute instance creation to time out.

To reduce the wait time and avoid potential timeout errors, we recommend manually provisioning the managed network. Then wait until the provisioning completes before you create a compute instance.

Note

To create an online deployment, you must manually provision the managed network, or create a compute instance first which will automatically provision it.

Use the Azure CLI or Python SDK tabs to learn how to manually provision the managed VNet.

Manage outbound rules

  1. Sign in to the Azure portal, and select the hub that you want to enable managed virtual network isolation for.
  2. Select Networking. The Azure AI Outbound access section allows you to manage outbound rules.
  • To add an outbound rule, select Add user-defined outbound rules from the Networking tab. From the Azure AI outbound rules sidebar, provide the following information:

  • To enable or disable a rule, use the toggle in the Active column.

  • To delete an outbound rule, select delete for the rule.

List of required rules

Tip

These rules are automatically added to the managed VNet.

Private endpoints:

  • When the isolation mode for the managed virtual network is Allow internet outbound, private endpoint outbound rules are automatically created as required rules from the managed virtual network for the hub and associated resources with public network access disabled (Key Vault, Storage Account, Container Registry, hub).
  • When the isolation mode for the managed virtual network is Allow only approved outbound, private endpoint outbound rules are automatically created as required rules from the managed virtual network for the hub and associated resources regardless of public network access mode for those resources (Key Vault, Storage Account, Container Registry, hub).

Outbound service tag rules:

  • AzureActiveDirectory
  • Azure Machine Learning
  • BatchNodeManagement.region
  • AzureResourceManager
  • AzureFrontDoor.FirstParty
  • MicrosoftContainerRegistry
  • AzureMonitor

Inbound service tag rules:

  • AzureMachineLearning

List of scenario specific outbound rules

Scenario: Access public machine learning packages

To allow installation of Python packages for training and deployment, add outbound FQDN rules to allow traffic to the following host names:

Note

This is not a complete list of the hosts required for all Python resources on the internet, only the most commonly used. For example, if you need access to a GitHub repository or other host, you must identify and add the required hosts for that scenario.

Host name Purpose
anaconda.com
*.anaconda.com
Used to install default packages.
*.anaconda.org Used to get repo data.
pypi.org Used to list dependencies from the default index, if any, and the index isn't overwritten by user settings. If the index is overwritten, you must also allow *.pythonhosted.org.
pytorch.org
*.pytorch.org
Used by some examples based on PyTorch.
*.tensorflow.org Used by some examples based on Tensorflow.

Scenario: Use Visual Studio Code

Visual Studio Code relies on specific hosts and ports to establish a remote connection.

Hosts

If you plan to use Visual Studio Code with the hub, add outbound FQDN rules to allow traffic to the following hosts:

  • *.vscode.dev
  • vscode.blob.core.windows.net
  • *.gallerycdn.vsassets.io
  • raw.githubusercontent.com
  • *.vscode-unpkg.net
  • *.vscode-cdn.net
  • *.vscodeexperiments.azureedge.net
  • default.exp-tas.com
  • code.visualstudio.com
  • update.code.visualstudio.com
  • *.vo.msecnd.net
  • marketplace.visualstudio.com
  • pkg-containers.githubusercontent.com
  • github.com

Ports

You must allow network traffic to ports 8704 to 8710. The VS Code server dynamically selects the first available port within this range.

Scenario: Use HuggingFace models

If you plan to use HuggingFace models with the hub, add outbound FQDN rules to allow traffic to the following hosts:

  • docker.io
  • *.docker.io
  • *.docker.com
  • production.cloudflare.docker.com
  • cnd.auth0.com
  • cdn-lfs.huggingface.co

Scenario: Curated by Azure AI

These models involve dynamic installation of dependencies at runtime, and require outbound FQDN rules to allow traffic to the following hosts:

*.anaconda.org *.anaconda.com anaconda.com pypi.org *.pythonhosted.org *.pytorch.org pytorch.org

Private endpoints

Private endpoints are currently supported for the following Azure services:

  • AI Studio hub
  • Azure AI Search
  • Azure AI services
  • Azure API Management
  • Azure Container Registry
  • Azure Cosmos DB (all sub resource types)
  • Azure Data Factory
  • Azure Database for MariaDB
  • Azure Database for MySQL
  • Azure Database for PostgreSQL Single Server
  • Azure Database for PostgreSQL Flexible Server
  • Azure Databricks
  • Azure Event Hubs
  • Azure Key Vault
  • Azure Machine Learning
  • Azure Machine Learning registries
  • Azure Redis Cache
  • Azure SQL Server
  • Azure Storage (all sub resource types)

When you create a private endpoint, you provide the resource type and subresource that the endpoint connects to. Some resources have multiple types and subresources. For more information, see what is a private endpoint.

When you create a private endpoint for hub dependency resources, such as Azure Storage, Azure Container Registry, and Azure Key Vault, the resource can be in a different Azure subscription. However, the resource must be in the same tenant as the hub.

A private endpoint is automatically created for a connection if the target resource is an Azure resource listed above. A valid target ID is expected for the private endpoint. A valid target ID for the connection can be the Azure Resource Manager ID of a parent resource. The target ID is also expected in the target of the connection or in metadata.resourceid. For more on connections, see How to add a new connection in Azure AI Studio.

Pricing

The hub managed virtual network feature is free. However, you're charged for the following resources that are used by the managed virtual network:

  • Azure Private Link - Private endpoints used to secure communications between the managed virtual network and Azure resources relies on Azure Private Link. For more information on pricing, see Azure Private Link pricing.

  • FQDN outbound rules - FQDN outbound rules are implemented using Azure Firewall. If you use outbound FQDN rules, charges for Azure Firewall are included in your billing. Azure Firewall SKU is standard. Azure Firewall is provisioned per hub.

    Important

    The firewall isn't created until you add an outbound FQDN rule. If you don't use FQDN rules, you will not be charged for Azure Firewall. For more information on pricing, see Azure Firewall pricing.