Can't create Compute Instance in Azure Machine Learning Workspace when Storage Account has no public IP address

Question

Hi everyone,

I'm trying to create an Azure Machine Learning workspace to be able to develop, train and put in production custom made Python AI models using Scikit-Learn. I want the setup to be created from Terraform and I want to ensure no public IP addresses are available on the system to avoid sensitive data leakage.

The issue I have is the following : when the Storage Account used for the Azure Machine Learning Workspace has the following attributes set to false to disable all public IP addresses, I can create a compute instance.

public_network_access_enabled = false
allow_nested_items_to_be_public = false

It fails with following error message:

When I set those two attributes of the storage account to true, everything runs fine.

Here is a part of the Terraform configuration file used here:

resource "azurerm_storage_account" "amlstorage" {
  name                          = var.aml_storage_account_name
  resource_group_name           = local.ai_resource_group_name
  location                      = local.location
  account_tier                  = "Standard"
  account_replication_type = "LRS"
  public_network_access_enabled = false # Issue here!
  allow_nested_items_to_be_public = false # Issue here!
  infrastructure_encryption_enabled = true
  https_traffic_only_enabled = true

  network_rules {
    default_action             = "Deny"
    bypass                     = ["AzureServices"]
    virtual_network_subnet_ids = [local.default_subnet_id]
  }
}

# Create a dedicated container for AML model artifacts
resource "azurerm_storage_container" "aml_model_artifacts_container" {
  name                  = "azureml-model-artifacts"
  storage_account_id    = azurerm_storage_account.amlstorage.id
  container_access_type = "private"
}

# Private Endpoint for AML Storage Account
resource "azurerm_private_endpoint" "aml_storage_blob_pe" {
  name                = "${local.prefix}amlstorage-blob-pe"
  location            = local.location
  resource_group_name = local.ai_resource_group_name
  subnet_id           = local.default_subnet_id

  private_service_connection {
    name                           = "${local.prefix}amlstorage-blob-psc"
    private_connection_resource_id = azurerm_storage_account.amlstorage.id
    subresource_names              = ["blob"]
    is_manual_connection           = false
  }

  private_dns_zone_group {
    name                 = "${local.prefix}amlstorage-blob-dns-group"
    private_dns_zone_ids = [local.private_dns_zone_blob_id]
  }
}

# Private Endpoint for AML Storage Account FILE service
resource "azurerm_private_endpoint" "aml_storage_file_pe" {
  name                = "${local.prefix}amlstorage-file-pe"
  location            = local.location
  resource_group_name = local.ai_resource_group_name
  subnet_id           = local.default_subnet_id

  private_service_connection {
    name                           = "${local.prefix}amlstorage-file-psc"
    private_connection_resource_id = azurerm_storage_account.amlstorage.id
    subresource_names              = ["file"]
    is_manual_connection           = false
  }

  private_dns_zone_group {
    name                 = "${local.prefix}amlstorage-file-dns-group"
    private_dns_zone_ids = [local.private_dns_zone_file_id]
  }
}

resource "azurerm_container_registry" "main" {
  name                          = "prdamlcontainerregistry"
  resource_group_name           = local.ai_resource_group_name
  location                      = local.location
  sku                           = "Basic"
  admin_enabled                 = false
}

resource "azurerm_machine_learning_workspace" "amlws" {
  name                        = "${local.prefix}amlworkspace"
  location                    = local.location
  resource_group_name         = local.ai_resource_group_name
  sku_name                    = "Basic"
  storage_account_id          = azurerm_storage_account.amlstorage.id
  key_vault_id                = local.key_vault_id
  application_insights_id     = azurerm_application_insights.aml_ai.id
  container_registry_id       = azurerm_container_registry.main.id

  identity {
    type = "SystemAssigned"
  }

  public_network_access_enabled = true
}

resource "azurerm_role_assignment" "aml_workspace_storage_blob_data_contributor" {
  scope                = azurerm_storage_account.amlstorage.id
  role_definition_name = "Storage Blob Data Contributor"
  principal_id         = azurerm_machine_learning_workspace.amlws.identity[0].principal_id
}

resource "azurerm_role_assignment" "acr_pull_for_aml_workspace" {
  scope                = azurerm_container_registry.main.id
  role_definition_name = "AcrPull"
  principal_id         = azurerm_machine_learning_workspace.amlws.identity[0].principal_id
}

# Compute instance for training models
resource "azurerm_machine_learning_compute_instance" "aml_compute_instance" {
  name                          = "${local.prefix}training-inst"
  machine_learning_workspace_id = azurerm_machine_learning_workspace.amlws.id
  virtual_machine_size          = "Standard_DS3_v2"
  subnet_resource_id            = local.default_subnet_id
  node_public_ip_enabled        = false

  identity {
    type = "SystemAssigned"
  }

  depends_on = [
    # Depends on the PE for the workspace and BOTH PEs for storage
    azurerm_private_endpoint.aml_workspace_pe,
    azurerm_private_endpoint.aml_storage_blob_pe,
    azurerm_private_endpoint.aml_storage_file_pe,
    # Depends on the role assignment being active
    azurerm_role_assignment.aml_workspace_storage_blob_data_contributor
  ]
}

# Private Endpoint for Azure Machine Learning Workspace Control Plane (API & Notebooks)
resource "azurerm_private_endpoint" "aml_workspace_pe" {
  name                = "${local.prefix}aml-ws-pe"
  location            = local.location
  resource_group_name = local.ai_resource_group_name # Private Endpoint should be in the RG of the service or a dedicated PE RG
  subnet_id           = local.default_subnet_id # Connect to your main default subnet

  private_service_connection {
    name                           = "${local.prefix}aml-ws-psc"
    private_connection_resource_id = azurerm_machine_learning_workspace.amlws.id
    subresource_names              = ["amlworkspace"] # Standard subresource for the workspace control plane
    is_manual_connection           = false
  }

  private_dns_zone_group {
    name                 = "${local.prefix}aml-ws-dns-group"
    # Link to both the API and Notebooks private DNS zones for the workspace
    private_dns_zone_ids = [
      local.aml_workspace_api_dns_zone_id,
      local.aml_workspace_notebooks_dns_zone_id
    ]
  }

  # Ensure the AML workspace is created before attempting to create its private endpoint
  depends_on = [
    azurerm_machine_learning_workspace.amlws,
    local.aml_workspace_api_vnet_link_id,
    local.aml_workspace_notebooks_vnet_link_id
  ]
}

Can you help me to fix that issue?